From patchwork Mon Nov 7 23:09:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035444 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 649EDC433FE for ; Mon, 7 Nov 2022 23:10:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232083AbiKGXKD (ORCPT ); Mon, 7 Nov 2022 18:10:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231974AbiKGXKB (ORCPT ); Mon, 7 Nov 2022 18:10:01 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0ADE41FCD6 for ; Mon, 7 Nov 2022 15:10:01 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id v4-20020a17090a088400b00212cb0ed97eso11780788pjc.5 for ; Mon, 07 Nov 2022 15:10:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RnrYmyFxoGEM6OlddgT2gVaTsDC543yW1/JtZQJqlOM=; b=CCIf1Zl8L1pIbAa66zDav4h3KJfv9e29otLCp362cEdzu5UdHXXpr+kBtj38FKn8dp A2Oe9cmNGL8UDN+bIIbOLyJ7WKT3l0yNtcbnA4FKGg6yo6dGsJiqhr6IzrBiDsO1clrJ dz+Rv2Uj/eByaUB+8O3Xd5NT+yBmsxzjF+IYWOMAGIhDrC7ajoes3pJlE9euizIWPRpr qHzB5TC/rIuVcYiANpXFKSCiMa4/txovmNgYBA+ZZLp6xh8nLkmo0x1u32obOu3pjWgM dV6I8vfQGNIAwcToBYp03/JUW1P2VAZZ7TnFRmWpxYwMI87KpA/lqJ1PWN6y0CVO6FT8 tOCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RnrYmyFxoGEM6OlddgT2gVaTsDC543yW1/JtZQJqlOM=; b=z3j6f0D4NcHj3RJeHTfBWG7A2ykCqLQdy+7tLFUNy2tzrcQ0B7gOqJKtLhHoNK3ax9 5n6ZnRVql7U8OfTbcgIeF2Pz1JSDihdL84PQJSixmjyptl/wE9kzZfm6nrPKLx8YbO+4 kfbbhiGT3vd7laJAYvV8ljIzgFPIby3AIef3+ROLbfQcNXvSbUCBFyvFmu47o+XgvdnS Cu2xHbYYIm6JqdVh6iZVY2nNkEKqMAOGPTMKPlPQsni4O8j5+lfzODW0095w7DRuGd8w LDyTYZE4Xh+MClhLumyBAf8FJd9it+H1nEZUHf+dwxr88zz4D2+Wtz0ZqJWaOmCoo5xz Afug== X-Gm-Message-State: ACrzQf3CfrZuQi9y1QaS7pbE02MBKU2yyj/Ym+y0lqKaEkhH9QAu2HPg T+Qq7MbsVnVy+7rNfxaD02exClWvn2huFA== X-Google-Smtp-Source: AMsMyM5S/xFcXoNrA2pFpM8NOY+8A4Y59O7QidUZJd0jlgazxLxQKDtqwe4yAiEsFRVgCjCps8CgcQ== X-Received: by 2002:a17:903:40cb:b0:187:1d3:461a with SMTP id t11-20020a17090340cb00b0018701d3461amr51461392pld.155.1667862600295; Mon, 07 Nov 2022 15:10:00 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id 72-20020a62194b000000b00562677968aesm5015198pfz.72.2022.11.07.15.09.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:00 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 01/25] bpf: Remove BPF_MAP_OFF_ARR_MAX Date: Tue, 8 Nov 2022 04:39:26 +0530 Message-Id: <20221107230950.7117-2-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1926; i=memxor@gmail.com; h=from:subject; bh=ZTLueoxLQrjQVPDjxfufu3gtMYHhhFj4zDRgWCv34xw=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+1KwPAD2afMKABKxB3GU1fQw0uIn5UXRBoG2Y8 3eCPBMaJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtQAKCRBM4MiGSL8RyjRGD/ 9zWZyEgaQonvxWtrYbCRq/2YKaDZn7Rs5l+hDP4WwoevVpWs1djjvSaP/2aP80sxDZJOgmdMtZ1x1Y B0+K5nB7RrIQK4GpI+wayscJKe/aYETPS0kTApDoSe3j5fjuX9w91V+/JnC+Q+XO4GCWz55ANkjLa1 vWwd1PLVPOU/85364+y4NwPURQ6u5BoePev/hKIlMxQ5KHYvdxiaNYf8X/aza1eyznsuJlybLfqz5V QCrJtT/TPnKTzXeKrPZ77PXUr6ccVgq8vuXm2mARxjOHiVulfvZloQ3fe72aPO4dFs2oXG1gG/JoJS Yc55IuxJDkMlK3bJzWXxq51XULpS7ebiBM1gi9Ggl1KQz3bsTTTLahyEPnnMDepy5c95RjghMwtpc2 8tmjmuWixB4Smz2C8sQgC0nDnApm3p4ZqQnJiaKyGUstkqnOtS95icRTQZZwk5PXYoGk+enysvYbay BzgR4IFRPEccHhqwZj0fZyISl1eUZO3/8EmUXJbrg5Od/ige4IJEPCCbecnv7hm8dzVwE1VXIYzaF0 Zzdv/dmd/hgEIHelZT5U4iO8CEQpPF4pl1Ei0IlBWyJwXKxre6YQG1zmSr2PpSp4KSF0diCwtLxbfA jk6EYOWecoJI56NmH3kZlJIgzmVQtzUj4TSxG2uk5Uegq4DmEwgH6bckkIUw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net In f71b2f64177a ("bpf: Refactor map->off_arr handling"), map->off_arr was refactored to be btf_field_offs. The number of field offsets is equal to maximum possible fields limited by BTF_FIELDS_MAX. Hence, reuse BTF_FIELDS_MAX as spin_lock and timer no longer are to be handled specially for offset sorting, fix the comment, and remove incorrect WARN_ON as its rec->cnt can never exceed this value. The reason to keep separate constant was the it was always more 2 more than total kptrs. This is no longer the case. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 9 ++++----- kernel/bpf/btf.c | 2 +- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 798aec816970..1a66a1df1af1 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -165,9 +165,8 @@ struct bpf_map_ops { }; enum { - /* Support at most 8 pointers in a BTF type */ - BTF_FIELDS_MAX = 10, - BPF_MAP_OFF_ARR_MAX = BTF_FIELDS_MAX, + /* Support at most 10 fields in a BTF type */ + BTF_FIELDS_MAX = 10, }; enum btf_field_type { @@ -203,8 +202,8 @@ struct btf_record { struct btf_field_offs { u32 cnt; - u32 field_off[BPF_MAP_OFF_ARR_MAX]; - u8 field_sz[BPF_MAP_OFF_ARR_MAX]; + u32 field_off[BTF_FIELDS_MAX]; + u8 field_sz[BTF_FIELDS_MAX]; }; struct bpf_map { diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 5579ff3a5b54..12361d7b2498 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3584,7 +3584,7 @@ struct btf_field_offs *btf_parse_field_offs(struct btf_record *rec) u8 *sz; BUILD_BUG_ON(ARRAY_SIZE(foffs->field_off) != ARRAY_SIZE(foffs->field_sz)); - if (IS_ERR_OR_NULL(rec) || WARN_ON_ONCE(rec->cnt > sizeof(foffs->field_off))) + if (IS_ERR_OR_NULL(rec)) return NULL; foffs = kzalloc(sizeof(*foffs), GFP_KERNEL | __GFP_NOWARN); From patchwork Mon Nov 7 23:09:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035445 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBAE9C43217 for ; Mon, 7 Nov 2022 23:10:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231974AbiKGXKH (ORCPT ); Mon, 7 Nov 2022 18:10:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232236AbiKGXKF (ORCPT ); Mon, 7 Nov 2022 18:10:05 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 554D52018E for ; Mon, 7 Nov 2022 15:10:04 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id u8-20020a17090a5e4800b002106dcdd4a0so16271845pji.1 for ; Mon, 07 Nov 2022 15:10:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TNUXGrsIBZA47dfMoUXlsA9DJrJBOE3oKFjyxYnaenU=; b=KG2tB+J4CDuVHyz0XZ62AmbeoCSHuKqEV5RAWO2SoLuDn9+4JAwsLPbjWY70WME83M A+N/4QVwkXli11nK4f4LMwyAxVsG9u3f1T1VrMFzixnoD576Jb+pTU0Ve3lkNdruNBVF dNUuoZp2/jCmQBgqiC35mz2kfdNgSxYTTcQprL6vg+gKKiCbE9t7bsevfjplDAl/7xVK 8xlGulGgNP3R0RN3usRv0ev5PnllMPtqGHhhF0EjDbyrZBQ+BVasiiEm1G+Ne7zcJLtj lIinF7wv4meZJyyKogw6KouI0I2y8bpe5dy0vsCWzgF5InN54fSWLGgfdyU6mQcbJrwJ 2hDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TNUXGrsIBZA47dfMoUXlsA9DJrJBOE3oKFjyxYnaenU=; b=az+eHx3yM0CcFzq4a9tPO9EHU44qzD2JpX+JFOM7qAr9/rMAlCEfVv0a3/0Ih1QT0q AdMX2yA5HtPmnMOlJTKcuBOl1kNJnP6TmPX7eJCn697ZhY84W1WdYhV5TsvuQW5iDZBF 1gEcxl/+QAuKVvp7KfVQcYyVc8/hwWfa+NK5U+7G3qENKc02l+TmMQ62WESeFeMmz5hm hdZ71GTxYL6U0p4gpmNYn76hTmRgx8H5e5T+WAqJwFX375DxC1g5RMpcNrxnnvnfJ6eA jahanZwL/bJpuuqDvEuP0aWpMfTn9C2LNsZUaYvWZ4HRFguIun9IScUWrnM5dXhNtiHx ufZQ== X-Gm-Message-State: ACrzQf3wadkhV+tMT4k8UAl5CTrFZ1ru81oqzHnvjYKluaKenb/e0ms2 mzUAbxhTKGC7S9gK/Jn9ZA3vpDyV/jCyqg== X-Google-Smtp-Source: AMsMyM5Umn6tYQLm8jIhvKK+bsk5pDCd2cFXxDl9qVdhqdSmQMcR10YsCvK0u05dDV7mk8bHDs5y0w== X-Received: by 2002:a17:90a:b897:b0:213:d66b:4996 with SMTP id o23-20020a17090ab89700b00213d66b4996mr47372847pjr.3.1667862603628; Mon, 07 Nov 2022 15:10:03 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id p13-20020a654bcd000000b00434272fe870sm4645240pgr.88.2022.11.07.15.10.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:03 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 02/25] bpf: Fix copy_map_value, zero_map_value Date: Tue, 8 Nov 2022 04:39:27 +0530 Message-Id: <20221107230950.7117-3-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1622; i=memxor@gmail.com; h=from:subject; bh=6RvsvjttmdEULzDCNignBfVVDaAhe2Uld3KegARQNKU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+1NLiBPyhOAGswnMZ++VCv2z7GCXT00P33PG8B xurXpGSJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtQAKCRBM4MiGSL8RyoMnEA DCi33aLtqfapRge7eNVXAvdMgKMcxopbacWGncUGX4eWoJ4HU1A9ZMPf8fOBxV3+w3oIA7gGc8L9xO yQAu1K+sAOUMrtpqGyImkYqs8kgHNncE00gciCzz0yNOLpJXuwnaETXG0Ds/y751yz6SahHW5FX6gO HrUoRpD5PuETPy91cYHs3SlH30uCgy1CCLfrhUtrjVvH7gLACfbU1/PlNvYoT+6oERG+HdAm6ZGDpB Dgan6q7SniwLj+95TguzuxbMi0C4xXQ3kda3nMHdUe9/rLpieUhffziU1W8b4eV6qrenyWCWjm+yPd 1dIW2eJucbRWEgOVB176kNEnCgjl7oldOYgbwrWJmpR02l1fTA/nnvSLHZB6RMWT/jkSd3qXkiXGIs KNqG81iwp+kS+hKJO6hWTOrxcbjmSpDle13xrOt7b8diRL5pkAdcq0FdYLatgVvOoBY9Kzhf64lcVb 0C4sdPHCGithExsN1Fj5//utwSDEI7OkJbjIqL+slfPNdLfUDFZ6ONOGUFiEsV8O3qTrjVMJj94tdx +W37E3HK2hvZVddcrvltg2uRpcXm9kvSEl4LzCS0DOTDGfHWc+hCyzKmXi40/AfA0tcOEYAbB5PToW kS2X4i/xJM6/v2En9iFXU6eWv6/tkQY/QQ7yU8cEBKXrEJFFK+uoLJlhV86Q== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The current offset needs to also skip over the already copied region in addition to the size of the next field. This case manifests where there are gaps between adjacent special fields. It was observed that for a map value with size 48, having fields at: off: 0, 16, 32 size: 4, 16, 16 The current code does: memcpy(dst + 0, src + 0, 0) memcpy(dst + 4, src + 4, 12) memcpy(dst + 20, src + 20, 12) memcpy(dst + 36, src + 36, 12) With the fix, it is done correctly as: memcpy(dst + 0, src + 0, 0) memcpy(dst + 4, src + 4, 12) memcpy(dst + 32, src + 32, 0) memcpy(dst + 48, src + 48, 0) Fixes: 4d7d7f69f4b1 ("bpf: Adapt copy_map_value for multiple offset case") Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 1a66a1df1af1..f08eb2d27de0 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -360,7 +360,7 @@ static inline void bpf_obj_memcpy(struct btf_field_offs *foffs, u32 sz = next_off - curr_off; memcpy(dst + curr_off, src + curr_off, sz); - curr_off += foffs->field_sz[i]; + curr_off += foffs->field_sz[i] + sz; } memcpy(dst + curr_off, src + curr_off, size - curr_off); } @@ -390,7 +390,7 @@ static inline void bpf_obj_memzero(struct btf_field_offs *foffs, void *dst, u32 u32 sz = next_off - curr_off; memset(dst + curr_off, 0, sz); - curr_off += foffs->field_sz[i]; + curr_off += foffs->field_sz[i] + sz; } memset(dst + curr_off, 0, size - curr_off); } From patchwork Mon Nov 7 23:09:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035447 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A22BC4332F for ; Mon, 7 Nov 2022 23:10:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232420AbiKGXKL (ORCPT ); Mon, 7 Nov 2022 18:10:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232375AbiKGXKJ (ORCPT ); Mon, 7 Nov 2022 18:10:09 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96BAD24BE7 for ; Mon, 7 Nov 2022 15:10:07 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id b29so12171889pfp.13 for ; Mon, 07 Nov 2022 15:10:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VEFwB6dYPn2S6gtavk14IeHvIx35vWmMY6ZMPqt7ZFo=; b=bs34dEqzODn/nH7VMn82bHUFLkMa8z4THlyZ6azFoV7N5Ajp7XAY87DOzDxqBgceU5 rVwdw9buULEpZJdBRNpJaN3vyrnTZ5K9MOnQhKGEceOVvjibvb5fXr8HGx1liDnYgtmj x43xVYodTM+m7O58G0JrWZjMl09tUYHzGSeSTSL47hYTNWlMVLieXOgOMUaL6Exw8lNG /kkBqAbdUZHRua3J0bPMHRW8mOCx2dr2fz8FTckMCvvG2xZJxp+uqacOp6b3aGLO5IY6 HQPNBdpE40Oyupu6LSaXgWCgzPw76wZEAB6P0oE576scNFlvXSET62xZd7gWc3qDwhl8 zRbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VEFwB6dYPn2S6gtavk14IeHvIx35vWmMY6ZMPqt7ZFo=; b=fgBxGnkF/SsAqtwK13U01UrwVQy0DOlkWP6BdWFW1cC+5RS1+gEIpvnklT0DZWfI2x x228AtIWcTQ5j4ch7Hb9lLVmF6m5vsjHPHNNyM3bPTSZ6jmi5pwa1Jr1+3zsSDXw71+E 4mwYWYA4RGsD7VPY/Nds7CxW6rD+ISmZxK4aouy2lTL1gbo+Ce/Qccg/JhLcOJHQVDGd OCbIzWE5lYm0ivKISw6ud79jMITF4oOg1JrevgIcSKhbqblbJomt5QQ5yrqNrkDPQUPy X7GKUZ+oQFHOSJhmYlfX2S4XQfA5nNz+w+VxLM7s6sZh3uOQ9LVCf4L4N0cr96HKzFKZ /elA== X-Gm-Message-State: ACrzQf0COoOz9tllyFbegamLGZD3dryHGWzTmVYDJH+7oLQomYID9ZFE kiHyuAE978sd1wmk88IznnpAZMtofEwROQ== X-Google-Smtp-Source: AMsMyM5gOFguqm3DYiIhovjWjeCZtpBydypwg/yvkd0G9bP2KyO4KnrenCAfkKjf1QacnkkiMXR7Fg== X-Received: by 2002:a05:6a00:2285:b0:56d:5d42:3aa8 with SMTP id f5-20020a056a00228500b0056d5d423aa8mr45135888pfe.79.1667862606687; Mon, 07 Nov 2022 15:10:06 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id j21-20020a170902c3d500b00186c9d17af2sm5513167plj.17.2022.11.07.15.10.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:06 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 03/25] bpf: Support bpf_list_head in map values Date: Tue, 8 Nov 2022 04:39:28 +0530 Message-Id: <20221107230950.7117-4-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=16579; i=memxor@gmail.com; h=from:subject; bh=ftOaE2WRB3lp74ANQAmDow4oJIZ7qZn7IimT/1j3Ius=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+1f5mA2kLUZP6gRstqKMbLVe4mGWUOCqegDMcE T8mNi1yJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtQAKCRBM4MiGSL8RykV1D/ 9Z8YCpFcgImEVm5ixbCWVSdefB3Xc7RzHRGh+rPRX8JzFToSmMVqhJGe12dHtgTzu2Znl5BhSpBaJI nl4Ek6bE4eadcok4+FNOJEihkbcUT2LYQSqsxaaL0CU1cBSScCO63Ii3TxHDFNb2GiD8G8K9PgOeNq HHfTIDDa3p1FJ3tAz47rp1lM8QQxPqO3pqSyf+4BVMHpiO9DHZuAEWk/OlUghcprWOnh+y4kHLMmNh 5DqCrthVYPRaWdh7ji5qXrEvvGGOVYdOkBQQuKRBeWJwdkkJuV191r2n7kOdGF2BMrMWe5PzV/LBfb 9Bw/PFF6uBx0dS11Z6QPfiYe6Yu/gySOzddVo/7alO6asUlPd6yCmlzPyJKFuMV4O7mxoVYZ68FLnk 7c9Q+kpXgE182UBpnhG+3JNph+5FZ7OT033a5BeNYH8WQzTKxwuaz7tsibDZuaZSW6fkzhRTEOd278 vPLYdgDUNQhfgKuProRLQYv3CD0kYwN36DFjIstqaK/XliyfCixxyvvoN6oWr0A7ZTcDImiJd/W+oe g12wTzrMSaMYPCpMEpw6a+dNLxiLbnV95Fgvr9jDgX5ZqgDZz01y+bRCSGcaLoGi21piBtpoTLMreQ aoXN7+DXgI265A4QqjxG9IK8+hArQUb7zNTAF6VppJ80hckyCINDH1krAJtQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add the support on the map side to parse, recognize, verify, and build metadata table for a new special field of the type struct bpf_list_head. To parameterize the bpf_list_head for a certain value type and the list_node member it will accept in that value type, we use BTF declaration tags. The definition of bpf_list_head in a map value will be done as follows: struct foo { struct bpf_list_node node; int data; }; struct map_value { struct bpf_list_head head __contains(foo, node); }; Then, the bpf_list_head only allows adding to the list 'head' using the bpf_list_node 'node' for the type struct foo. The 'contains' annotation is a BTF declaration tag composed of four parts, "contains:name:node" where the name is then used to look up the type in the map BTF, with its kind hardcoded to BTF_KIND_STRUCT during the lookup. The node defines name of the member in this type that has the type struct bpf_list_node, which is actually used for linking into the linked list. For now, 'kind' part is hardcoded as struct. This allows building intrusive linked lists in BPF, using container_of to obtain pointer to entry, while being completely type safe from the perspective of the verifier. The verifier knows exactly the type of the nodes, and knows that list helpers return that type at some fixed offset where the bpf_list_node member used for this list exists. The verifier also uses this information to disallow adding types that are not accepted by a certain list. For now, no elements can be added to such lists. Support for that is coming in future patches, hence draining and freeing items is done with a TODO that will be resolved in a future patch. Note that the bpf_list_head_free function moves the list out to a local variable under the lock and releases it, doing the actual draining of the list items outside the lock. While this helps with not holding the lock for too long pessimizing other concurrent list operations, it is also necessary for deadlock prevention: unless every function called in the critical section would be notrace, a fentry/fexit program could attach and call bpf_map_update_elem again on the map, leading to the same lock being acquired if the key matches and lead to a deadlock. While this requires some special effort on part of the BPF programmer to trigger and is highly unlikely to occur in practice, it is always better if we can avoid such a condition. While notrace would prevent this, doing the draining outside the lock has advantages of its own, hence it is used to also fix the deadlock related problem. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 17 ++++ include/uapi/linux/bpf.h | 10 +++ kernel/bpf/btf.c | 143 ++++++++++++++++++++++++++++++++- kernel/bpf/helpers.c | 32 ++++++++ kernel/bpf/syscall.c | 22 ++++- kernel/bpf/verifier.c | 7 ++ tools/include/uapi/linux/bpf.h | 10 +++ 7 files changed, 237 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f08eb2d27de0..05f98e9e5c48 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -175,6 +175,7 @@ enum btf_field_type { BPF_KPTR_UNREF = (1 << 2), BPF_KPTR_REF = (1 << 3), BPF_KPTR = BPF_KPTR_UNREF | BPF_KPTR_REF, + BPF_LIST_HEAD = (1 << 4), }; struct btf_field_kptr { @@ -184,11 +185,18 @@ struct btf_field_kptr { u32 btf_id; }; +struct btf_field_list_head { + struct btf *btf; + u32 value_btf_id; + u32 node_offset; +}; + struct btf_field { u32 offset; enum btf_field_type type; union { struct btf_field_kptr kptr; + struct btf_field_list_head list_head; }; }; @@ -266,6 +274,8 @@ static inline const char *btf_field_type_name(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: return "kptr"; + case BPF_LIST_HEAD: + return "bpf_list_head"; default: WARN_ON_ONCE(1); return "unknown"; @@ -282,6 +292,8 @@ static inline u32 btf_field_type_size(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: return sizeof(u64); + case BPF_LIST_HEAD: + return sizeof(struct bpf_list_head); default: WARN_ON_ONCE(1); return 0; @@ -298,6 +310,8 @@ static inline u32 btf_field_type_align(enum btf_field_type type) case BPF_KPTR_UNREF: case BPF_KPTR_REF: return __alignof__(u64); + case BPF_LIST_HEAD: + return __alignof__(struct bpf_list_head); default: WARN_ON_ONCE(1); return 0; @@ -403,6 +417,9 @@ static inline void zero_map_value(struct bpf_map *map, void *dst) void copy_map_value_locked(struct bpf_map *map, void *dst, void *src, bool lock_src); void bpf_timer_cancel_and_free(void *timer); +void bpf_list_head_free(const struct btf_field *field, void *list_head, + struct bpf_spin_lock *spin_lock); + int bpf_obj_name_cpy(char *dst, const char *src, unsigned int size); struct bpf_offload_dev; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 94659f6b3395..dd381086bad9 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -6887,6 +6887,16 @@ struct bpf_dynptr { __u64 :64; } __attribute__((aligned(8))); +struct bpf_list_head { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + +struct bpf_list_node { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + struct bpf_sysctl { __u32 write; /* Sysctl is being read (= 0) or written (= 1). * Allows 1,2,4-byte read, but no write. diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 12361d7b2498..d8f083b09e5e 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3205,9 +3205,15 @@ enum { struct btf_field_info { enum btf_field_type type; u32 off; - struct { - u32 type_id; - } kptr; + union { + struct { + u32 type_id; + } kptr; + struct { + const char *node_name; + u32 value_btf_id; + } list_head; + }; }; static int btf_find_struct(const struct btf *btf, const struct btf_type *t, @@ -3261,6 +3267,63 @@ static int btf_find_kptr(const struct btf *btf, const struct btf_type *t, return BTF_FIELD_FOUND; } +static const char *btf_find_decl_tag_value(const struct btf *btf, + const struct btf_type *pt, + int comp_idx, const char *tag_key) +{ + int i; + + for (i = 1; i < btf_nr_types(btf); i++) { + const struct btf_type *t = btf_type_by_id(btf, i); + int len = strlen(tag_key); + + if (!btf_type_is_decl_tag(t)) + continue; + if (pt != btf_type_by_id(btf, t->type) || + btf_type_decl_tag(t)->component_idx != comp_idx) + continue; + if (strncmp(__btf_name_by_offset(btf, t->name_off), tag_key, len)) + continue; + return __btf_name_by_offset(btf, t->name_off) + len; + } + return NULL; +} + +static int btf_find_list_head(const struct btf *btf, const struct btf_type *pt, + const struct btf_type *t, int comp_idx, + u32 off, int sz, struct btf_field_info *info) +{ + const char *value_type; + const char *list_node; + s32 id; + + if (!__btf_type_is_struct(t)) + return BTF_FIELD_IGNORE; + if (t->size != sz) + return BTF_FIELD_IGNORE; + value_type = btf_find_decl_tag_value(btf, pt, comp_idx, "contains:"); + if (!value_type) + return -EINVAL; + list_node = strstr(value_type, ":"); + if (!list_node) + return -EINVAL; + value_type = kstrndup(value_type, list_node - value_type, GFP_KERNEL | __GFP_NOWARN); + if (!value_type) + return -ENOMEM; + id = btf_find_by_name_kind(btf, value_type, BTF_KIND_STRUCT); + kfree(value_type); + if (id < 0) + return id; + list_node++; + if (str_is_empty(list_node)) + return -EINVAL; + info->type = BPF_LIST_HEAD; + info->off = off; + info->list_head.value_btf_id = id; + info->list_head.node_name = list_node; + return BTF_FIELD_FOUND; +} + static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, int *align, int *sz) { @@ -3284,6 +3347,12 @@ static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, goto end; } } + if (field_mask & BPF_LIST_HEAD) { + if (!strcmp(name, "bpf_list_head")) { + type = BPF_LIST_HEAD; + goto end; + } + } /* Only return BPF_KPTR when all other types with matchable names fail */ if (field_mask & BPF_KPTR) { type = BPF_KPTR_REF; @@ -3317,6 +3386,8 @@ static int btf_find_struct_field(const struct btf *btf, return field_type; off = __btf_member_bit_offset(t, member); + if (i && !off) + return -EFAULT; if (off % 8) /* valid C code cannot generate such BTF */ return -EINVAL; @@ -3339,6 +3410,12 @@ static int btf_find_struct_field(const struct btf *btf, if (ret < 0) return ret; break; + case BPF_LIST_HEAD: + ret = btf_find_list_head(btf, t, member_type, i, off, sz, + idx < info_cnt ? &info[idx] : &tmp); + if (ret < 0) + return ret; + break; default: return -EFAULT; } @@ -3373,6 +3450,8 @@ static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, return field_type; off = vsi->offset; + if (i && !off) + return -EFAULT; if (vsi->size != sz) continue; if (off % align) @@ -3393,6 +3472,12 @@ static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, if (ret < 0) return ret; break; + case BPF_LIST_HEAD: + ret = btf_find_list_head(btf, var, var_type, -1, off, sz, + idx < info_cnt ? &info[idx] : &tmp); + if (ret < 0) + return ret; + break; default: return -EFAULT; } @@ -3491,6 +3576,46 @@ static int btf_parse_kptr(const struct btf *btf, struct btf_field *field, return ret; } +static int btf_parse_list_head(const struct btf *btf, struct btf_field *field, + struct btf_field_info *info) +{ + const struct btf_type *t, *n = NULL; + const struct btf_member *member; + u32 offset; + int i; + + t = btf_type_by_id(btf, info->list_head.value_btf_id); + /* We've already checked that value_btf_id is a struct type. We + * just need to figure out the offset of the list_node, and + * verify its type. + */ + for_each_member(i, t, member) { + if (strcmp(info->list_head.node_name, __btf_name_by_offset(btf, member->name_off))) + continue; + /* Invalid BTF, two members with same name */ + if (n) + return -EINVAL; + n = btf_type_by_id(btf, member->type); + if (!__btf_type_is_struct(n)) + return -EINVAL; + if (strcmp("bpf_list_node", __btf_name_by_offset(btf, n->name_off))) + return -EINVAL; + offset = __btf_member_bit_offset(n, member); + if (offset % 8) + return -EINVAL; + offset /= 8; + if (offset % __alignof__(struct bpf_list_node)) + return -EINVAL; + + field->list_head.btf = (struct btf *)btf; + field->list_head.value_btf_id = info->list_head.value_btf_id; + field->list_head.node_offset = offset; + } + if (!n) + return -ENOENT; + return 0; +} + struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t, u32 field_mask, u32 value_size) { @@ -3539,12 +3664,24 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type if (ret < 0) goto end; break; + case BPF_LIST_HEAD: + ret = btf_parse_list_head(btf, &rec->fields[i], &info_arr[i]); + if (ret < 0) + goto end; + break; default: ret = -EFAULT; goto end; } rec->cnt++; } + + /* bpf_list_head requires bpf_spin_lock */ + if (btf_record_has_field(rec, BPF_LIST_HEAD) && rec->spin_lock_off < 0) { + ret = -EINVAL; + goto end; + } + return rec; end: btf_record_free(rec); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 283f55bbeb70..7bc71995f17c 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1706,6 +1706,38 @@ bpf_base_func_proto(enum bpf_func_id func_id) } } +void bpf_list_head_free(const struct btf_field *field, void *list_head, + struct bpf_spin_lock *spin_lock) +{ + struct list_head *head = list_head, *orig_head = list_head; + + BUILD_BUG_ON(sizeof(struct list_head) > sizeof(struct bpf_list_head)); + BUILD_BUG_ON(__alignof__(struct list_head) > __alignof__(struct bpf_list_head)); + + /* Do the actual list draining outside the lock to not hold the lock for + * too long, and also prevent deadlocks if tracing programs end up + * executing on entry/exit of functions called inside the critical + * section, and end up doing map ops that call bpf_list_head_free for + * the same map value again. + */ + __bpf_spin_lock_irqsave(spin_lock); + if (!head->next || list_empty(head)) + goto unlock; + head = head->next; +unlock: + INIT_LIST_HEAD(orig_head); + __bpf_spin_unlock_irqrestore(spin_lock); + + while (head != orig_head) { + void *obj = head; + + obj -= field->list_head.node_offset; + head = head->next; + /* TODO: Rework later */ + kfree(obj); + } +} + BTF_SET8_START(tracing_btf_ids) #ifdef CONFIG_KEXEC_CORE BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 85532d301124..fdbae52f463f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -536,6 +536,9 @@ void btf_record_free(struct btf_record *rec) module_put(rec->fields[i].kptr.module); btf_put(rec->fields[i].kptr.btf); break; + case BPF_LIST_HEAD: + /* Nothing to release for bpf_list_head */ + break; default: WARN_ON_ONCE(1); continue; @@ -578,6 +581,9 @@ struct btf_record *btf_record_dup(const struct btf_record *rec) goto free; } break; + case BPF_LIST_HEAD: + /* Nothing to acquire for bpf_list_head */ + break; default: ret = -EFAULT; WARN_ON_ONCE(1); @@ -637,6 +643,11 @@ void bpf_obj_free_fields(const struct btf_record *rec, void *obj) case BPF_KPTR_REF: field->kptr.dtor((void *)xchg((unsigned long *)field_ptr, 0)); break; + case BPF_LIST_HEAD: + if (WARN_ON_ONCE(rec->spin_lock_off < 0)) + continue; + bpf_list_head_free(field, field_ptr, obj + rec->spin_lock_off); + break; default: WARN_ON_ONCE(1); continue; @@ -965,7 +976,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, if (!value_type || value_size != map->value_size) return -EINVAL; - map->record = btf_parse_fields(btf, value_type, BPF_SPIN_LOCK | BPF_TIMER | BPF_KPTR, + map->record = btf_parse_fields(btf, value_type, + BPF_SPIN_LOCK | BPF_TIMER | BPF_KPTR | BPF_LIST_HEAD, map->value_size); if (!IS_ERR_OR_NULL(map->record)) { int i; @@ -1012,6 +1024,14 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, goto free_map_tab; } break; + case BPF_LIST_HEAD: + if (map->map_type != BPF_MAP_TYPE_HASH && + map->map_type != BPF_MAP_TYPE_LRU_HASH && + map->map_type != BPF_MAP_TYPE_ARRAY) { + ret = -EOPNOTSUPP; + goto free_map_tab; + } + break; default: /* Fail if map_type checks are missing for a field type */ ret = -EOPNOTSUPP; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d3b75aa0c54d..0374f03d1f56 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12805,6 +12805,13 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env, { enum bpf_prog_type prog_type = resolve_prog_type(prog); + if (btf_record_has_field(map->record, BPF_LIST_HEAD)) { + if (is_tracing_prog_type(prog_type)) { + verbose(env, "tracing progs cannot use bpf_list_head yet\n"); + return -EINVAL; + } + } + if (btf_record_has_field(map->record, BPF_SPIN_LOCK)) { if (prog_type == BPF_PROG_TYPE_SOCKET_FILTER) { verbose(env, "socket filter progs cannot use bpf_spin_lock yet\n"); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 94659f6b3395..dd381086bad9 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -6887,6 +6887,16 @@ struct bpf_dynptr { __u64 :64; } __attribute__((aligned(8))); +struct bpf_list_head { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + +struct bpf_list_node { + __u64 :64; + __u64 :64; +} __attribute__((aligned(8))); + struct bpf_sysctl { __u32 write; /* Sysctl is being read (= 0) or written (= 1). * Allows 1,2,4-byte read, but no write. From patchwork Mon Nov 7 23:09:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035446 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 425F2C433FE for ; Mon, 7 Nov 2022 23:10:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232141AbiKGXKM (ORCPT ); Mon, 7 Nov 2022 18:10:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232274AbiKGXKL (ORCPT ); Mon, 7 Nov 2022 18:10:11 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 849311FCD6 for ; Mon, 7 Nov 2022 15:10:10 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id b11so12135344pjp.2 for ; Mon, 07 Nov 2022 15:10:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X/wrTWjeoGJ5PcE2HVDewWAhvJm6XCc8VxMDXoI4aI8=; b=TKY/6ZiMWSVXINrDrMw+1bKv2OGBLjM2cf0VZTcokKASpFTd/uE6BU1GqD4klC0J40 WCtTgeShKo9gi6gLE6TZOe+d0GNLgTXtIHkrOnuzBF+KulX+yn3itfn8g3JKu59un67v LXiZDSbCnVr3GDHSzO2erAIpX9o0AsT46KEGgiVXr672vV8OVFdkQE1DOpVnb41US4Fb rv5XRAKiKnXyCX0X+29oZKTfXoc7SVc3DzHN1tAHoxpSHk8y7QrwQY8rFWqaL3lsZtlM pnDAZ9uAAepMDMOGsKNPCYAszfuf1r4CC33Ql667J2hOQU1hGp6BYXPRINpv1R8hOeOx /Dbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X/wrTWjeoGJ5PcE2HVDewWAhvJm6XCc8VxMDXoI4aI8=; b=Z9TJKtvmfZzAaPQDZj1LlpQFsPTNbbIocVJg80tjQD6h6mSxvHeuwu1LqyNNlo9rIu 7yfRRvf/nU/Kaduvq4Edtvwi5Yf7Qxqob6CjalwBlY7oWf+3FI7me+WQVDckGWuggycW z5qEiKMSYGn4QGfAmBaw45UoM7Q8t84/mLV3rjzDbubX5gd5fsJLkgJOibVMxHBfWHzT sxWyJeSkVkIgvUlG4lW6FKEIeCtKHRQwmGHjCE59uAaJG4iWRQoslQNNddj2NBAfjoKZ AoMvp5CnmTM6sdRwtotFVcT63ikx+eZK+rU8W+fkBEg0PvCf7ywiq2pG8KsqpTXIJvKx C5zA== X-Gm-Message-State: ACrzQf26tvzbeyRXu/FgQPwweme+1xY1XtFtzt7bFgwapBWBNO7pIJoN FTKyxCXRkqzv1E2fG8oSYItDdYua0EwDfg== X-Google-Smtp-Source: AMsMyM4GLxlyFuVh0IH6aoemCUBE94p6EaCWpxxl1qI2c+CLy0WMuwUOK+s0u/iJ1SXa8l4XA8ROgw== X-Received: by 2002:a17:902:8f91:b0:187:4acf:861c with SMTP id z17-20020a1709028f9100b001874acf861cmr30434687plo.166.1667862609831; Mon, 07 Nov 2022 15:10:09 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id a24-20020a170902b59800b001788ccecbf5sm5471761pls.31.2022.11.07.15.10.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:09 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 04/25] bpf: Rename RET_PTR_TO_ALLOC_MEM Date: Tue, 8 Nov 2022 04:39:29 +0530 Message-Id: <20221107230950.7117-5-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2687; i=memxor@gmail.com; h=from:subject; bh=0z92ryWA6XsveKbJHr6S6jTfA+C5JIJe/ym7BPLG/iY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+1HRkIjZt8mnhFlc+mlBlx/wckOGz9VYukDvsY WaIsVzOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtQAKCRBM4MiGSL8Ryjm1EA CH2llLPGUpEU3ExJsOC1bTJEvvwWnWfZJTaAyhRMKKu9J4WcMmqJPlWodC3tjNdzFHddyQ5235oimJ uFsc32qpbEum+6+5kTV7NpHMLlXgJLj7U+oH6wWdJieC1MnvW6R2NsPkfUiVKU6MyBZqG2kjWJMLfU wNcJ5Amz9TaRE01VQ3zCZ+VWM7zM/pZRj+a4iLgGTarQuzMpW6qwlB13dmIVei3ggPsNhsq6FsDP1c m52+F3bBQMfP9nUoD3wdAzYpip8riiBeqC1uIok0dWLNPnLW9AqDZb61gpjV7llioXbn6yAwnf8dFB pgt2voZKFRhZpJlCG5Dp3qqwKRDHfpVgG1XMNLZx//9fXR4ppP1LM9a6AJ3xTcwN3LczK1kVldZAXU BRU6cqi06+PshXW9tF0UBywnD47vgywzTV3afHbgsAtCfCY/CVnI3LKJsj1uPt5rMQfoXf+wFt+Wo4 /C0n73z4DXkOHB1lhht3MR2IxjBLXq6gRlLstNbgd5+HYc8O2g3pTEc1Tv8jxb5it0SxmCanAMJ0Ci LRbzzn4gm3wc62Yg1TUbs+K1KTLjnYxRUAD4TGKeVXdsmkcSTY+Ke+HYAIWJf0bVEp8ijv5nt1YHAS tiIRfaU9KhnL1cM3tKxAX6sC20E3EpchuEmm59BUjX436e3nPVaVfF/t7nOw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, the verifier has two return types, RET_PTR_TO_ALLOC_MEM, and RET_PTR_TO_ALLOC_MEM_OR_NULL, however the former is confusingly named to imply that it carries MEM_ALLOC, while only the latter does. This causes confusion during code review leading to conclusions like that the return value of RET_PTR_TO_DYNPTR_MEM_OR_NULL (which is RET_PTR_TO_ALLOC_MEM | PTR_MAYBE_NULL) may be consumable by bpf_ringbuf_{submit,commit}. Rename it to make it clear MEM_ALLOC needs to be tacked on top of RET_PTR_TO_MEM. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 6 +++--- kernel/bpf/verifier.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 05f98e9e5c48..2fe3ec620d54 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -607,7 +607,7 @@ enum bpf_return_type { RET_PTR_TO_SOCKET, /* returns a pointer to a socket */ RET_PTR_TO_TCP_SOCK, /* returns a pointer to a tcp_sock */ RET_PTR_TO_SOCK_COMMON, /* returns a pointer to a sock_common */ - RET_PTR_TO_ALLOC_MEM, /* returns a pointer to dynamically allocated memory */ + RET_PTR_TO_MEM, /* returns a pointer to memory */ RET_PTR_TO_MEM_OR_BTF_ID, /* returns a pointer to a valid memory or a btf_id */ RET_PTR_TO_BTF_ID, /* returns a pointer to a btf_id */ __BPF_RET_TYPE_MAX, @@ -617,8 +617,8 @@ enum bpf_return_type { RET_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCKET, RET_PTR_TO_TCP_SOCK_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_TCP_SOCK, RET_PTR_TO_SOCK_COMMON_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCK_COMMON, - RET_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_ALLOC | RET_PTR_TO_ALLOC_MEM, - RET_PTR_TO_DYNPTR_MEM_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_ALLOC_MEM, + RET_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_ALLOC | RET_PTR_TO_MEM, + RET_PTR_TO_DYNPTR_MEM_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_MEM, RET_PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_BTF_ID, /* This must be the last entry. Its purpose is to ensure the enum is diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 0374f03d1f56..2407e3b179ec 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7621,7 +7621,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn mark_reg_known_zero(env, regs, BPF_REG_0); regs[BPF_REG_0].type = PTR_TO_TCP_SOCK | ret_flag; break; - case RET_PTR_TO_ALLOC_MEM: + case RET_PTR_TO_MEM: mark_reg_known_zero(env, regs, BPF_REG_0); regs[BPF_REG_0].type = PTR_TO_MEM | ret_flag; regs[BPF_REG_0].mem_size = meta.mem_size; From patchwork Mon Nov 7 23:09:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035448 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DA57C4332F for ; Mon, 7 Nov 2022 23:10:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232375AbiKGXKQ (ORCPT ); Mon, 7 Nov 2022 18:10:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232274AbiKGXKO (ORCPT ); Mon, 7 Nov 2022 18:10:14 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A20171FCD6 for ; Mon, 7 Nov 2022 15:10:13 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id k15so12212361pfg.2 for ; Mon, 07 Nov 2022 15:10:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7Qn946/Dfi7ihlJL944GILOqS6Benn+3+4NY3sXoxuY=; b=XMwK30ZTTwl1vVsJa6DnR4Hfu9pWzDKwaCk7c7NHjZ8NkN2rBcDCqVDIxBk4mRrlTc Tf5KgH59BOb884gu+epRQ9ZqQgU1rwrm/8VYEBINo+umYEF2WGV/caIXcYKyO5GZtssv JkHpxg5zSMuqmA9/oMG7guJtkiGRyJ0Gxg0A4IP4SP2FoythCAZ85GNRcCayW/QgfB2O 1VWUDmj9IPwpU3SLmqL6Q3Rt4xwyUboiAGSmKlAMYifQBS7EAuvAa0ftd0DkgIj5lkW7 lp+0/y7vV4LkEsdCIAXB5DLoh0sR1AmFYdYdccLfubJBHrQiU/qJFYbC9zuF8w7A+7ZB 3FfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7Qn946/Dfi7ihlJL944GILOqS6Benn+3+4NY3sXoxuY=; b=czecHbh9dFLiaTQ7+Pp3WFSBDNxPoLLjQKTF7r5ZrvxnaFy/uJ2BZsq9tLee5bjmHm 4PRs/dtnDoqNEoRphZXLa83OXU+tZpHwbhTC73JmAzE1j7ZConhCHy50SBo7ojCuYcuK m0dmvGl27pLjZFQORWzCGraemjqx7HL/ponAIZjOsT+9xD1Sn/IZ2Pc/oQi7qkXg9ocD AoJbDQdtFxXF3MEbpIkXjmgPrRh7lfgLbg7FfKnkDMeAFSP/IqCpQe3E+1lACgSVxOXf ewknvqxZLjWbZCedB3ki5hjAxJ74LRXrFvSHPu2e/RTT4LRV5m/HsHJ1w3UQGLmCC+ms tG8A== X-Gm-Message-State: ACrzQf1XkEKANRkz6TUy3/Jd+XZKC7icfI2OM+yw8e03tZro90b7mHW5 nZwlKabtIjtzL0se5qwEel/OvvJEwsuhBw== X-Google-Smtp-Source: AMsMyM79IR6O7oohYfE264eixE1mbMnECOnodhqANdwKw0qvBzPHHfl2uqplnl7iENOJixC8aXN1Zw== X-Received: by 2002:a62:ea0d:0:b0:55f:8624:4d8b with SMTP id t13-20020a62ea0d000000b0055f86244d8bmr52836284pfh.74.1667862612840; Mon, 07 Nov 2022 15:10:12 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id b4-20020a170902650400b001869079d083sm5520176plk.90.2022.11.07.15.10.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:12 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 05/25] bpf: Rename MEM_ALLOC to MEM_RINGBUF Date: Tue, 8 Nov 2022 04:39:30 +0530 Message-Id: <20221107230950.7117-6-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=9098; i=memxor@gmail.com; h=from:subject; bh=QiF5aiMWdeVPrX2X+RNmqO7yjZn/FhBvFcVkHpPSfBU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+16ZPke4vV7BNRjXfeS4Pi5fu52IQiEddTUb6G YQ5duCiJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtQAKCRBM4MiGSL8RyhNhD/ oCCPkYYpmE1Zear4NJ/VDyUdlW8fz0ddybq/sGhmRDvtia58pKWM18LIWD7LNWbBluiQxmWq3FWf+j w2UdWO93j6DfK9ytrhNmaxTIUicpFqrjjtE8FAnK+1e7YmTBicvcEAdxsZkPMJVWmhVTBDneofKwBH lOIDUmsn2Py6LJMGTG28AX2Kt1ybEoLx1a016SlPfIoCzMBJyW83HHDTN8kGa3skAEsLVDy0BOIhau 0Ae3EW6nDlVKzHcqIQP09/XxLXD7b80FHZxP+vDf/rePFRfY0lrA2yyfjRMfCdOwFldKkG8fX3+ldi gsdi020DJg0KdNNAIFkRbS9ljs1nDvJL2pmlo91DySemrbFLWrmuBE2bfZZEVZ2H8ZV8m7lLoYSdw6 87QYL3WRetQLNh8wGUKj7znv8Ykg9S7pEJgFZVtQb2Mddh7usfmepDZ38WDubrDWaJCT3ezHn4hrJB 0iZQt835KmrSNAHwU9c7IYhpyivLpzn5mkyM0kzdqd/kv7RiodwkqCJ/4ZIDD4te3Qe5atMIjC57R8 K5BxJk4hMpGSMvN8aPLkwx9jbRqRDp/sN18zYjZ906+BpvRoT6qkMgTNdfkRgGzOCDO4fg7/Ahph/z x4VCk3L5lul7BwpvUfqcnz6ZWHnpprYtzw3UuR2KL/r2UolPNSGFsReLy7hg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, verifier uses MEM_ALLOC type tag to specially tag memory returned from bpf_ringbuf_reserve helper. However, this is currently only used for this purpose and there is an implicit assumption that it only refers to ringbuf memory (e.g. the check for ARG_PTR_TO_ALLOC_MEM in check_func_arg_reg_off). Hence, rename MEM_ALLOC to MEM_RINGBUF to indicate this special relationship and instead open the use of MEM_ALLOC for more generic allocations made for user types. Also, since ARG_PTR_TO_ALLOC_MEM_OR_NULL is unused, simply drop it. Finally, update selftests using 'alloc_' verifier string to 'ringbuf_'. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 11 ++++------- kernel/bpf/ringbuf.c | 6 +++--- kernel/bpf/verifier.c | 14 +++++++------- tools/testing/selftests/bpf/prog_tests/dynptr.c | 2 +- tools/testing/selftests/bpf/verifier/ringbuf.c | 2 +- tools/testing/selftests/bpf/verifier/spill_fill.c | 2 +- 6 files changed, 17 insertions(+), 20 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 2fe3ec620d54..afc1c51b59ff 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -488,10 +488,8 @@ enum bpf_type_flag { */ MEM_RDONLY = BIT(1 + BPF_BASE_TYPE_BITS), - /* MEM was "allocated" from a different helper, and cannot be mixed - * with regular non-MEM_ALLOC'ed MEM types. - */ - MEM_ALLOC = BIT(2 + BPF_BASE_TYPE_BITS), + /* MEM points to BPF ring buffer reservation. */ + MEM_RINGBUF = BIT(2 + BPF_BASE_TYPE_BITS), /* MEM is in user address space. */ MEM_USER = BIT(3 + BPF_BASE_TYPE_BITS), @@ -565,7 +563,7 @@ enum bpf_arg_type { ARG_PTR_TO_LONG, /* pointer to long */ ARG_PTR_TO_SOCKET, /* pointer to bpf_sock (fullsock) */ ARG_PTR_TO_BTF_ID, /* pointer to in-kernel struct */ - ARG_PTR_TO_ALLOC_MEM, /* pointer to dynamically allocated memory */ + ARG_PTR_TO_RINGBUF_MEM, /* pointer to dynamically reserved ringbuf memory */ ARG_CONST_ALLOC_SIZE_OR_ZERO, /* number of allocated bytes requested */ ARG_PTR_TO_BTF_ID_SOCK_COMMON, /* pointer to in-kernel sock_common or bpf-mirrored bpf_sock */ ARG_PTR_TO_PERCPU_BTF_ID, /* pointer to in-kernel percpu type */ @@ -582,7 +580,6 @@ enum bpf_arg_type { ARG_PTR_TO_MEM_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_MEM, ARG_PTR_TO_CTX_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_CTX, ARG_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_SOCKET, - ARG_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_ALLOC_MEM, ARG_PTR_TO_STACK_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_STACK, ARG_PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | ARG_PTR_TO_BTF_ID, /* pointer to memory does not need to be initialized, helper function must fill @@ -617,7 +614,7 @@ enum bpf_return_type { RET_PTR_TO_SOCKET_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCKET, RET_PTR_TO_TCP_SOCK_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_TCP_SOCK, RET_PTR_TO_SOCK_COMMON_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_SOCK_COMMON, - RET_PTR_TO_ALLOC_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_ALLOC | RET_PTR_TO_MEM, + RET_PTR_TO_RINGBUF_MEM_OR_NULL = PTR_MAYBE_NULL | MEM_RINGBUF | RET_PTR_TO_MEM, RET_PTR_TO_DYNPTR_MEM_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_MEM, RET_PTR_TO_BTF_ID_OR_NULL = PTR_MAYBE_NULL | RET_PTR_TO_BTF_ID, diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index 9e832acf4692..80f4b4d88aaf 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -447,7 +447,7 @@ BPF_CALL_3(bpf_ringbuf_reserve, struct bpf_map *, map, u64, size, u64, flags) const struct bpf_func_proto bpf_ringbuf_reserve_proto = { .func = bpf_ringbuf_reserve, - .ret_type = RET_PTR_TO_ALLOC_MEM_OR_NULL, + .ret_type = RET_PTR_TO_RINGBUF_MEM_OR_NULL, .arg1_type = ARG_CONST_MAP_PTR, .arg2_type = ARG_CONST_ALLOC_SIZE_OR_ZERO, .arg3_type = ARG_ANYTHING, @@ -490,7 +490,7 @@ BPF_CALL_2(bpf_ringbuf_submit, void *, sample, u64, flags) const struct bpf_func_proto bpf_ringbuf_submit_proto = { .func = bpf_ringbuf_submit, .ret_type = RET_VOID, - .arg1_type = ARG_PTR_TO_ALLOC_MEM | OBJ_RELEASE, + .arg1_type = ARG_PTR_TO_RINGBUF_MEM | OBJ_RELEASE, .arg2_type = ARG_ANYTHING, }; @@ -503,7 +503,7 @@ BPF_CALL_2(bpf_ringbuf_discard, void *, sample, u64, flags) const struct bpf_func_proto bpf_ringbuf_discard_proto = { .func = bpf_ringbuf_discard, .ret_type = RET_VOID, - .arg1_type = ARG_PTR_TO_ALLOC_MEM | OBJ_RELEASE, + .arg1_type = ARG_PTR_TO_RINGBUF_MEM | OBJ_RELEASE, .arg2_type = ARG_ANYTHING, }; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 2407e3b179ec..5fca156eca43 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -577,8 +577,8 @@ static const char *reg_type_str(struct bpf_verifier_env *env, if (type & MEM_RDONLY) strncpy(prefix, "rdonly_", 32); - if (type & MEM_ALLOC) - strncpy(prefix, "alloc_", 32); + if (type & MEM_RINGBUF) + strncpy(prefix, "ringbuf_", 32); if (type & MEM_USER) strncpy(prefix, "user_", 32); if (type & MEM_PERCPU) @@ -5780,7 +5780,7 @@ static const struct bpf_reg_types mem_types = { PTR_TO_MAP_KEY, PTR_TO_MAP_VALUE, PTR_TO_MEM, - PTR_TO_MEM | MEM_ALLOC, + PTR_TO_MEM | MEM_RINGBUF, PTR_TO_BUF, }, }; @@ -5798,7 +5798,7 @@ static const struct bpf_reg_types int_ptr_types = { static const struct bpf_reg_types fullsock_types = { .types = { PTR_TO_SOCKET } }; static const struct bpf_reg_types scalar_types = { .types = { SCALAR_VALUE } }; static const struct bpf_reg_types context_types = { .types = { PTR_TO_CTX } }; -static const struct bpf_reg_types alloc_mem_types = { .types = { PTR_TO_MEM | MEM_ALLOC } }; +static const struct bpf_reg_types ringbuf_mem_types = { .types = { PTR_TO_MEM | MEM_RINGBUF } }; static const struct bpf_reg_types const_map_ptr_types = { .types = { CONST_PTR_TO_MAP } }; static const struct bpf_reg_types btf_ptr_types = { .types = { PTR_TO_BTF_ID } }; static const struct bpf_reg_types spin_lock_types = { .types = { PTR_TO_MAP_VALUE } }; @@ -5831,7 +5831,7 @@ static const struct bpf_reg_types *compatible_reg_types[__BPF_ARG_TYPE_MAX] = { [ARG_PTR_TO_BTF_ID] = &btf_ptr_types, [ARG_PTR_TO_SPIN_LOCK] = &spin_lock_types, [ARG_PTR_TO_MEM] = &mem_types, - [ARG_PTR_TO_ALLOC_MEM] = &alloc_mem_types, + [ARG_PTR_TO_RINGBUF_MEM] = &ringbuf_mem_types, [ARG_PTR_TO_INT] = &int_ptr_types, [ARG_PTR_TO_LONG] = &int_ptr_types, [ARG_PTR_TO_PERCPU_BTF_ID] = &percpu_btf_ptr_types, @@ -5952,14 +5952,14 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, case PTR_TO_MAP_VALUE: case PTR_TO_MEM: case PTR_TO_MEM | MEM_RDONLY: - case PTR_TO_MEM | MEM_ALLOC: + case PTR_TO_MEM | MEM_RINGBUF: case PTR_TO_BUF: case PTR_TO_BUF | MEM_RDONLY: case SCALAR_VALUE: /* Some of the argument types nevertheless require a * zero register offset. */ - if (base_type(arg_type) != ARG_PTR_TO_ALLOC_MEM) + if (base_type(arg_type) != ARG_PTR_TO_RINGBUF_MEM) return 0; break; /* All the rest must be rejected, except PTR_TO_BTF_ID which allows diff --git a/tools/testing/selftests/bpf/prog_tests/dynptr.c b/tools/testing/selftests/bpf/prog_tests/dynptr.c index 8fc4e6c02bfd..b0c06f821cb8 100644 --- a/tools/testing/selftests/bpf/prog_tests/dynptr.c +++ b/tools/testing/selftests/bpf/prog_tests/dynptr.c @@ -17,7 +17,7 @@ static struct { {"ringbuf_missing_release2", "Unreleased reference id=2"}, {"ringbuf_missing_release_callback", "Unreleased reference id"}, {"use_after_invalid", "Expected an initialized dynptr as arg #3"}, - {"ringbuf_invalid_api", "type=mem expected=alloc_mem"}, + {"ringbuf_invalid_api", "type=mem expected=ringbuf_mem"}, {"add_dynptr_to_map1", "invalid indirect read from stack"}, {"add_dynptr_to_map2", "invalid indirect read from stack"}, {"data_slice_out_of_bounds_ringbuf", "value is outside of the allowed memory range"}, diff --git a/tools/testing/selftests/bpf/verifier/ringbuf.c b/tools/testing/selftests/bpf/verifier/ringbuf.c index b64d33e4833c..84838feba47f 100644 --- a/tools/testing/selftests/bpf/verifier/ringbuf.c +++ b/tools/testing/selftests/bpf/verifier/ringbuf.c @@ -28,7 +28,7 @@ }, .fixup_map_ringbuf = { 1 }, .result = REJECT, - .errstr = "dereference of modified alloc_mem ptr R1", + .errstr = "dereference of modified ringbuf_mem ptr R1", }, { "ringbuf: invalid reservation offset 2", diff --git a/tools/testing/selftests/bpf/verifier/spill_fill.c b/tools/testing/selftests/bpf/verifier/spill_fill.c index e23f07175e1b..9bb302dade23 100644 --- a/tools/testing/selftests/bpf/verifier/spill_fill.c +++ b/tools/testing/selftests/bpf/verifier/spill_fill.c @@ -84,7 +84,7 @@ }, .fixup_map_ringbuf = { 1 }, .result = REJECT, - .errstr = "R0 pointer arithmetic on alloc_mem_or_null prohibited", + .errstr = "R0 pointer arithmetic on ringbuf_mem_or_null prohibited", }, { "check corrupted spill/fill", From patchwork Mon Nov 7 23:09:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035449 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75DD5C433FE for ; Mon, 7 Nov 2022 23:10:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232484AbiKGXKU (ORCPT ); Mon, 7 Nov 2022 18:10:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232448AbiKGXKS (ORCPT ); Mon, 7 Nov 2022 18:10:18 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACD4823164 for ; Mon, 7 Nov 2022 15:10:16 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id d13-20020a17090a3b0d00b00213519dfe4aso11802042pjc.2 for ; Mon, 07 Nov 2022 15:10:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ye/v2RFv95ygSo6ebZbkDhxhdMhepOxDfkIMAVMhXHE=; b=VtYaM2h/6ucNK5VNMwstE25MpF1heWTtwhTJA0TjFPW2VzCqznaorQjdA0qtcSMx6q wYTMallvDEXQnZaqmX2mU7NqgwpukD/qYnECVPb6X8fR9LVZuKYji873MV4ZNUGY00mr tl1mQOkdHd+x8kndP3kwMg30qANA6gPD7DKM+WDR3n91J0hMj2hDIUwQaTli72DxiR2L dWpxpmx9TWKqGyrQJs7miyjMygIgLY37rkVd5a1nwoOqKqAFCfE4XA+DT3rcWhQXF4ck LMyOW2zaNQLu21eph/ldcaa6MVMCcrNT9eYrWLfFkU9iUceLozOE2iyxzpy/LvZSwBYd i44A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ye/v2RFv95ygSo6ebZbkDhxhdMhepOxDfkIMAVMhXHE=; b=UkwIP8jWV1Bp1kYMJxbyRnDfBZjzKKAHvhvxfhV+A0a6K/CVSKf6QRhpQh+AOViYcj j0mm7r/S24e5s+6x6mdMo+6KV1+oPlCte9CsI0lY0rWEUvmQA9yCk737TWfsMPFfHHIn Sxpf1FvQ0/ZDdTcP7KbUgWqTR8p1YutXUiNyvw4L9U3wfnE47C6ftiN018JS0jaHTKkH e26k05yRA6zYfR7BMppxSO3YlI8X4cNC/CqQYccywaloGGZ4UJfOB0vQqDiGj8WWuMPc PBSAqD9Mj0WkCQ3A7yhWhUbw6rGHBedoFtR9ZPO9H+NkvLfrWiAS9zIPqkwe9KgS7tM7 OiWw== X-Gm-Message-State: ACrzQf16mZhWsVvOKoaK1ErCzmBP3pYCrPVqW5ZHI7Hz1fuiT4CZPZ29 eOrUNWmjTU9G6nM2GdTaXu/WyrPv2y+uog== X-Google-Smtp-Source: AMsMyM6w1TGQ55enXMY3NyiaYB6PTgiRegfCaxf8odqKykQJEREDSYHFeOICMILaUIfcQROoG1uhQg== X-Received: by 2002:a17:903:11c7:b0:178:af17:e93e with SMTP id q7-20020a17090311c700b00178af17e93emr52470650plh.78.1667862615843; Mon, 07 Nov 2022 15:10:15 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id t11-20020a170902b20b00b0018703bf42desm5515223plr.159.2022.11.07.15.10.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:15 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 06/25] bpf: Introduce local kptrs Date: Tue, 8 Nov 2022 04:39:31 +0530 Message-Id: <20221107230950.7117-7-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=16362; i=memxor@gmail.com; h=from:subject; bh=Dja4sjb7SgEYFlHvZ5lFyq1ddKX2AatjroTIv5xwmOs=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+2EFhi7MtRk0H1+8OSskhXalY+kkbrRPVg9KSO 2uC+qEiJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8RyofvEA CoZIpUOspkixl5GNtdI9agEEsQHXboq2tRs0JO+qCWdfWWFMBdtgV5A/jEOqAUorJqLSRRsUk7qFtU Bw5e2K9UR8Dfs2AwlPQGqOaEnBl8wJpuhqpXwBVbVipJInEOvdFYLE1Httag2kcBJ3H/ECGEC1shpP 6YMtvFmS1i2FWoZ/y/Cu1WXv64pqtT3SImSmxQrlUQCX9P5i5swh6Le6FpfnsJbg37PKKHLv/uqlXx rd4t96KX0fxsC77mgvIpXrahxB0TIucPco0KT59Cy4tixWcTUjfseekUNrth8o7eGpHd642z19fiEU lICWzYNU3NBlAs2ivSaNUkRdzV6QXnibS2+j5Ritu8ZD5wnf8HHcyhEKQHQ4CHd+8+4xJBNXUsG+5Y QZm8oqv72y17S8rGgt1D2jMBA60F2bw8kZ6dDG/2CdCziewRljrmyLq5fxqlj+xQ7KbV1UpNyhF6od s7hLtzU1vxpATMhTrnfM/jPoebFky5f2UxeLuLGqGEjHHd0gtNPoDg0Z/9kPHsRS7YQqMs8bShEF2/ C7N0I3noI18cgwX2ayNyKrLdMgBEztVmDMGPW9vtCH7UrxE+ghq6qbpmv4LT3q3JeuNzB9L+O5QRrR rCbGhQaQXiSXAHinuwRrX8qZ2fBBko/I4cdkAPxSgnQGHhGbyi4n2ge2OZnw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Introduce local kptrs, i.e. PTR_TO_BTF_ID that point to a type in program BTF. This is indicated by the presence of MEM_ALLOC type flag in reg->type to avoid having to check btf_is_kernel when trying to match argument types in helpers. Refactor btf_struct_access callback to just take bpf_reg_state instead of btf and btf_type paramters. Note that the call site in check_map_access now simulates access to a PTR_TO_BTF_ID by creating a dummy reg on stack. Since only the type, btf, and btf_id of the register matter for the checks, it can be done so without complicating the usual cases elsewhere in the verifier where reg->btf and reg->btf_id is used verbatim. Whenever walking such types, any pointers being walked will always yield a SCALAR instead of pointer. In the future we might permit kptr inside local kptr (either kernel or local), and it would be permitted only in that case. For now, these local kptrs will always be referenced in verifier context, hence ref_obj_id == 0 for them is a bug. It is allowed to write to such objects, as long fields that are special are not touched (support for which will be added in subsequent patches). Note that once such a local kptr is marked PTR_UNTRUSTED, it is no longer allowed to write to it. No PROBE_MEM handling is therefore done for loads into this type unless PTR_UNTRUSTED is part of the register type, since they can never be in an undefined state, and their lifetime will always be valid. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 28 ++++++++++++++++-------- include/linux/filter.h | 8 +++---- kernel/bpf/btf.c | 16 ++++++++++---- kernel/bpf/verifier.c | 37 ++++++++++++++++++++++++++------ net/bpf/bpf_dummy_struct_ops.c | 14 ++++++------ net/core/filter.c | 34 ++++++++++++----------------- net/ipv4/bpf_tcp_ca.c | 13 ++++++----- net/netfilter/nf_conntrack_bpf.c | 17 ++++++--------- 8 files changed, 99 insertions(+), 68 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index afc1c51b59ff..75dbd2ecf80a 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -524,6 +524,11 @@ enum bpf_type_flag { /* Size is known at compile time. */ MEM_FIXED_SIZE = BIT(10 + BPF_BASE_TYPE_BITS), + /* MEM is of a type from program BTF, not kernel BTF. This is used to + * tag PTR_TO_BTF_ID allocated using bpf_obj_new. + */ + MEM_ALLOC = BIT(11 + BPF_BASE_TYPE_BITS), + __BPF_TYPE_FLAG_MAX, __BPF_TYPE_LAST_FLAG = __BPF_TYPE_FLAG_MAX - 1, }; @@ -771,6 +776,7 @@ struct bpf_prog_ops { union bpf_attr __user *uattr); }; +struct bpf_reg_state; struct bpf_verifier_ops { /* return eBPF function prototype for verification */ const struct bpf_func_proto * @@ -792,9 +798,8 @@ struct bpf_verifier_ops { struct bpf_insn *dst, struct bpf_prog *prog, u32 *target_size); int (*btf_struct_access)(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag); }; @@ -2080,9 +2085,9 @@ static inline bool bpf_tracing_btf_ctx_access(int off, int size, return btf_ctx_access(off, size, type, prog, info); } -int btf_struct_access(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, +int btf_struct_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag); bool btf_struct_ids_match(struct bpf_verifier_log *log, const struct btf *btf, u32 id, int off, @@ -2333,9 +2338,8 @@ static inline struct bpf_prog *bpf_prog_by_id(u32 id) } static inline int btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag) { return -EACCES; @@ -2792,4 +2796,10 @@ struct bpf_key { bool has_ref; }; #endif /* CONFIG_KEYS */ + +static inline bool type_is_local_kptr(u32 type) +{ + return type & MEM_ALLOC; +} + #endif /* _LINUX_BPF_H */ diff --git a/include/linux/filter.h b/include/linux/filter.h index efc42a6e3aed..787d35dbf5b0 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -568,10 +568,10 @@ struct sk_filter { DECLARE_STATIC_KEY_FALSE(bpf_stats_enabled_key); extern struct mutex nf_conn_btf_access_lock; -extern int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, u32 *next_btf_id, - enum bpf_type_flag *flag); +extern int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag); typedef unsigned int (*bpf_dispatcher_fn)(const void *ctx, const struct bpf_insn *insnsi, diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index d8f083b09e5e..4d6c8577bf17 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6015,20 +6015,28 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf, return -EINVAL; } -int btf_struct_access(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype __maybe_unused, +int btf_struct_access(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype __maybe_unused, u32 *next_btf_id, enum bpf_type_flag *flag) { + const struct btf *btf = reg->btf; enum bpf_type_flag tmp_flag = 0; + const struct btf_type *t; + u32 id = reg->btf_id; int err; - u32 id; + t = btf_type_by_id(btf, id); do { err = btf_struct_walk(log, btf, t, off, size, &id, &tmp_flag); switch (err) { case WALK_PTR: + /* For local types, the destination register cannot + * become a pointer again. + */ + if (type_is_local_kptr(reg->type)) + return SCALAR_VALUE; /* If we found the pointer or scalar on t+off, * we're done. */ diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5fca156eca43..7dcb4629f764 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4682,17 +4682,28 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env, return -EACCES; } - if (env->ops->btf_struct_access) { - ret = env->ops->btf_struct_access(&env->log, reg->btf, t, - off, size, atype, &btf_id, &flag); + if (env->ops->btf_struct_access && !type_is_local_kptr(reg->type)) { + if (!btf_is_kernel(reg->btf)) { + verbose(env, "verifier internal error: reg->btf must be kernel btf\n"); + return -EFAULT; + } + ret = env->ops->btf_struct_access(&env->log, reg, off, size, atype, &btf_id, &flag); } else { - if (atype != BPF_READ) { + /* Writes are permitted with default btf_struct_access for local + * kptrs (which always have ref_obj_id > 0), but not for + * _untrusted_ local kptrs. + */ + if (atype != BPF_READ && reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { verbose(env, "only read is supported\n"); return -EACCES; } - ret = btf_struct_access(&env->log, reg->btf, t, off, size, - atype, &btf_id, &flag); + if (type_is_local_kptr(reg->type) && !reg->ref_obj_id) { + verbose(env, "verifier internal error: ref_obj_id for local kptr must be non-zero\n"); + return -EFAULT; + } + + ret = btf_struct_access(&env->log, reg, off, size, atype, &btf_id, &flag); } if (ret < 0) @@ -4718,6 +4729,7 @@ static int check_ptr_to_map_access(struct bpf_verifier_env *env, { struct bpf_reg_state *reg = regs + regno; struct bpf_map *map = reg->map_ptr; + struct bpf_reg_state map_reg; enum bpf_type_flag flag = 0; const struct btf_type *t; const char *tname; @@ -4756,7 +4768,10 @@ static int check_ptr_to_map_access(struct bpf_verifier_env *env, return -EACCES; } - ret = btf_struct_access(&env->log, btf_vmlinux, t, off, size, atype, &btf_id, &flag); + /* Simulate access to a PTR_TO_BTF_ID */ + memset(&map_reg, 0, sizeof(map_reg)); + mark_btf_ld_reg(env, &map_reg, 0, PTR_TO_BTF_ID, btf_vmlinux, *map->ops->map_btf_id, 0); + ret = btf_struct_access(&env->log, &map_reg, off, size, atype, &btf_id, &flag); if (ret < 0) return ret; @@ -5966,6 +5981,7 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, * fixed offset. */ case PTR_TO_BTF_ID: + case PTR_TO_BTF_ID | MEM_ALLOC: /* When referenced PTR_TO_BTF_ID is passed to release function, * it's fixed offset must be 0. In the other cases, fixed offset * can be non-zero. @@ -13648,6 +13664,13 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) break; case PTR_TO_BTF_ID: case PTR_TO_BTF_ID | PTR_UNTRUSTED: + /* PTR_TO_BTF_ID | MEM_ALLOC always has a valid lifetime, unlike + * PTR_TO_BTF_ID, and an active ref_obj_id, but the same cannot + * be said once it is marked PTR_UNTRUSTED, hence we must handle + * any faults for loads into such types. BPF_WRITE is disallowed + * for this case. + */ + case PTR_TO_BTF_ID | MEM_ALLOC | PTR_UNTRUSTED: if (type == BPF_READ) { insn->code = BPF_LDX | BPF_PROBE_MEM | BPF_SIZE((insn)->code); diff --git a/net/bpf/bpf_dummy_struct_ops.c b/net/bpf/bpf_dummy_struct_ops.c index e78dadfc5829..2d434c1f4617 100644 --- a/net/bpf/bpf_dummy_struct_ops.c +++ b/net/bpf/bpf_dummy_struct_ops.c @@ -156,29 +156,29 @@ static bool bpf_dummy_ops_is_valid_access(int off, int size, } static int bpf_dummy_ops_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, u32 *next_btf_id, enum bpf_type_flag *flag) { const struct btf_type *state; + const struct btf_type *t; s32 type_id; int err; - type_id = btf_find_by_name_kind(btf, "bpf_dummy_ops_state", + type_id = btf_find_by_name_kind(reg->btf, "bpf_dummy_ops_state", BTF_KIND_STRUCT); if (type_id < 0) return -EINVAL; - state = btf_type_by_id(btf, type_id); + t = btf_type_by_id(reg->btf, reg->btf_id); + state = btf_type_by_id(reg->btf, type_id); if (t != state) { bpf_log(log, "only access to bpf_dummy_ops_state is supported\n"); return -EACCES; } - err = btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + err = btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); if (err < 0) return err; diff --git a/net/core/filter.c b/net/core/filter.c index cb3b635e35be..199632e6a7cb 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -8651,28 +8651,25 @@ static bool tc_cls_act_is_valid_access(int off, int size, DEFINE_MUTEX(nf_conn_btf_access_lock); EXPORT_SYMBOL_GPL(nf_conn_btf_access_lock); -int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, const struct btf *btf, - const struct btf_type *t, int off, int size, - enum bpf_access_type atype, u32 *next_btf_id, - enum bpf_type_flag *flag); +int (*nfct_btf_struct_access)(struct bpf_verifier_log *log, + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag); EXPORT_SYMBOL_GPL(nfct_btf_struct_access); static int tc_cls_act_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { int ret = -EACCES; if (atype == BPF_READ) - return btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + return btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_lock(&nf_conn_btf_access_lock); if (nfct_btf_struct_access) - ret = nfct_btf_struct_access(log, btf, t, off, size, atype, next_btf_id, flag); + ret = nfct_btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_unlock(&nf_conn_btf_access_lock); return ret; @@ -8738,21 +8735,18 @@ void bpf_warn_invalid_xdp_action(struct net_device *dev, struct bpf_prog *prog, EXPORT_SYMBOL_GPL(bpf_warn_invalid_xdp_action); static int xdp_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { int ret = -EACCES; if (atype == BPF_READ) - return btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + return btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_lock(&nf_conn_btf_access_lock); if (nfct_btf_struct_access) - ret = nfct_btf_struct_access(log, btf, t, off, size, atype, next_btf_id, flag); + ret = nfct_btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); mutex_unlock(&nf_conn_btf_access_lock); return ret; diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 6da16ae6a962..d15c91de995f 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -69,18 +69,17 @@ static bool bpf_tcp_ca_is_valid_access(int off, int size, } static int bpf_tcp_ca_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { + const struct btf_type *t; size_t end; if (atype == BPF_READ) - return btf_struct_access(log, btf, t, off, size, atype, next_btf_id, - flag); + return btf_struct_access(log, reg, off, size, atype, next_btf_id, flag); + t = btf_type_by_id(reg->btf, reg->btf_id); if (t != tcp_sock_type) { bpf_log(log, "only read is supported\n"); return -EACCES; diff --git a/net/netfilter/nf_conntrack_bpf.c b/net/netfilter/nf_conntrack_bpf.c index 8639e7efd0e2..24002bc61e07 100644 --- a/net/netfilter/nf_conntrack_bpf.c +++ b/net/netfilter/nf_conntrack_bpf.c @@ -191,19 +191,16 @@ BTF_ID(struct, nf_conn___init) /* Check writes into `struct nf_conn` */ static int _nf_conntrack_btf_struct_access(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int off, - int size, enum bpf_access_type atype, - u32 *next_btf_id, - enum bpf_type_flag *flag) + const struct bpf_reg_state *reg, + int off, int size, enum bpf_access_type atype, + u32 *next_btf_id, enum bpf_type_flag *flag) { - const struct btf_type *ncit; - const struct btf_type *nct; + const struct btf_type *ncit, *nct, *t; size_t end; - ncit = btf_type_by_id(btf, btf_nf_conn_ids[1]); - nct = btf_type_by_id(btf, btf_nf_conn_ids[0]); - + ncit = btf_type_by_id(reg->btf, btf_nf_conn_ids[1]); + nct = btf_type_by_id(reg->btf, btf_nf_conn_ids[0]); + t = btf_type_by_id(reg->btf, reg->btf_id); if (t != nct && t != ncit) { bpf_log(log, "only read is supported\n"); return -EACCES; From patchwork Mon Nov 7 23:09:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035451 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3AD5C433FE for ; Mon, 7 Nov 2022 23:10:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232236AbiKGXKY (ORCPT ); Mon, 7 Nov 2022 18:10:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232448AbiKGXKW (ORCPT ); Mon, 7 Nov 2022 18:10:22 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E835323164 for ; Mon, 7 Nov 2022 15:10:20 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id y13so12196463pfp.7 for ; Mon, 07 Nov 2022 15:10:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XUm3eU1Buz+0ts0Dif5SZ61dZ8dUqnINfHNkZQXYpU8=; b=qu8lonhYtyC/zECoKYCfYf1uBhPW7zdUbHIi86jGYWEfLsHrclSpnZq42FL3A8iYw2 mxjTFtUf0egqIWbCy2yY5tpguYoBXnZrfb7XhJg/X5dSo6NIv3qiV14egzQrzo/PvEiG b1ou+aJy31FBiAGbpOyat1MxfxEAZU5HOBifYEvFQRZbwq3WISp7PMwed5BMrizrmyAL jzuzwL/V0fEH8vdsiKZYRzvrjq4QGPvNpLc/RHoB6vsNbrt3oLM9fa3Ydi9xs+WA3nxK a1XmEJz3a1XMbOmxPIXoTk/nP/DCl/9EmEEAcHb8eL9+qxgcArhLMptI+hinXwRMGKIy UzWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XUm3eU1Buz+0ts0Dif5SZ61dZ8dUqnINfHNkZQXYpU8=; b=MwJMgXnqtUxWIKlCdoqLr4r75k5USEO+wqco5m5XeaJgMnTIwP6PfqSfVaS3+6KBeF aCzXazHdO8QUh9CZiHN0vPqnd4uVJZsxidhoXAcS4s+DkIj5VzJ47UaTiXA9JEm4fiZ/ PwtD+TQvpLpvcLpv0n0YaXibp3wm4Wp1HuFQGnBGwYa4LuxiteQ+iGFpvmKS74M5fawK gFq6Gu3A/4O2zNJtpiqL0RmcVI9DfjZiSyObIo0ipy1HeaDVWYc5P8FQZ9AdL8hK9Hiu G64bENHekBVCJwBIWwIMqrYtKQth2o8zTMgeRDaz+TFWIIEsCm8BzCNZhDEYFGJ83Sy5 U5Yg== X-Gm-Message-State: ACrzQf2KPOpQNaCEll/cZ2tIuponk2qDBrgVSZ7FSo7GBmByvP9Fc4eH QCa704rqyBGIhZDO+EwQXFecCqVvXT9FaA== X-Google-Smtp-Source: AMsMyM74jB+esTc3yDWKccOD/OzefKI7IAoXk2Bu1NoM2ASC/PzJ5Pa4wLF1p6MgM78+xu1eym6GEw== X-Received: by 2002:a05:6a00:1c82:b0:562:7bed:9676 with SMTP id y2-20020a056a001c8200b005627bed9676mr914694pfw.13.1667862620064; Mon, 07 Nov 2022 15:10:20 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id b3-20020a62cf03000000b00562a526cd2esm5037265pfg.55.2022.11.07.15.10.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:19 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 07/25] bpf: Recognize bpf_{spin_lock,list_head,list_node} in local kptrs Date: Tue, 8 Nov 2022 04:39:32 +0530 Message-Id: <20221107230950.7117-8-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=14726; i=memxor@gmail.com; h=from:subject; bh=IDI88PXg+i1c6BSagE19MIWh+rcW6bvnq7C0li9H3zg=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+2DtgpCrh/1XYlRSdfHfzgfq9er1vp7DC7oZqz sw9QDQKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8RyuXeD/ wK/ezWE4qjwrPYgytP5LUGz2LXJrT2G2yOUoUhB0qa0D3HmG98HCpEsPJMwpf+5ee9KkCXWZaDTtmy JKFU/IHIqc9oiZCCTEF7ZOdmV4sJO6X3y7nkSTDvzeA4DT+L7HtMOqkB3VOENY9STlKBno4RF4VxgD 4mrlFFqQOJdT2IPKGIcqA55mHjwTaH+T1DW1WNkqtYH7vYCuaa5eRPvRqd2oT9ZXi06lIAlhFXWt3o /w1HuAcHgWk8SnxAMBuCuZnpe1UpQMDX6jlryMaXHs7DCPsHIu07EjrJt3sKlvtbBe9X8NVpD9FvKc Hd2uiUWGy84GZ4BzUN35KGmHLqhI4LeEm2F5khsWm3pzpaJ8nnf+HQSMYIWabRZBY76f5w6qmBL92E IiIVbQnqe2jSk8vf3deJ5Ir7l598Si332HBylZtcWrvBwF0f+mIdxz5wSJBg43+l1h4xYXEH5s2X7L hQeYf0KBBZ9gRFyQ4Xerao4z3fxt/xo5Uk8aFY0trQ50HK8BV0NM2p2Hx8XDYNRZ4D3MvLkmH1kfL/ pxr526m0zorlAEvyGWvm2UbMfyvaIIbcAn3CWyNnD+3V7xS1MkMH+0WqwGtLqOILRnuGj6d986Lo9+ dlBNkazVYWvM47EUYZ8lcHatxi8Atm+gBND0eyMVScd6Js18vcGdTWQARUpQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Allow specifying bpf_spin_lock, bpf_list_head, bpf_list_node fields in a local kptr. Update btf_struct_access callback to reject direct access to these special fields in local kptrs. A bpf_list_head allows implementing map-in-map style use cases, where local kptr with bpf_list_head is linked into a list in a map value. This would require embedding a bpf_list_node, support for which is also included. Lastly, while we strictly don't require to hold a bpf_spin_lock while manipulating the bpf_list_head of a local kptr, as when have access to it, we have complete ownership of the object, the locking constraint is still kept and may be conditionally lifted in the future. Note that the specification of such types can be done just like map values, e.g.: struct bar { struct bpf_list_node node; }; struct foo { struct bpf_spin_lock lock; struct bpf_list_head head __contains(bar, node); struct bpf_list_node node; }; struct map_value { struct bpf_spin_lock lock; struct bpf_list_head head __contains(foo, node); }; To recognize such types in user BTF, we build a btf_struct_metas array of metadata items corresponding to each BTF ID. This is done once during the btf_parse stage to avoid having to do it each time during the verification process's requirement to inspect the metadata. Moreover, the computed metadata needs to be passed to some helpers in future patches which requires allocating them and storing them in the BTF that is pinned by the program itself, so that valid access can be assumed to such data during program runtime. A key thing to note is that once a btf_struct_meta is available for a type, both the btf_record and btf_field_offs should be available. It is critical that btf_field_offs is available in case special fields are present, as we extensively rely on special fields being zeroed out in map values and local kptrs in later patches. The code ensures that by bailing out in case of errors and ensuring both are available together. If the record is not available, the special fields won't be recognized, so not having both is also fine (in terms of being a verification error and not a runtime bug). Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 7 ++ include/linux/btf.h | 35 ++++++++ kernel/bpf/btf.c | 196 +++++++++++++++++++++++++++++++++++++++---- kernel/bpf/syscall.c | 4 + 4 files changed, 224 insertions(+), 18 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 75dbd2ecf80a..dfcd34e36025 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -176,6 +176,7 @@ enum btf_field_type { BPF_KPTR_REF = (1 << 3), BPF_KPTR = BPF_KPTR_UNREF | BPF_KPTR_REF, BPF_LIST_HEAD = (1 << 4), + BPF_LIST_NODE = (1 << 5), }; struct btf_field_kptr { @@ -276,6 +277,8 @@ static inline const char *btf_field_type_name(enum btf_field_type type) return "kptr"; case BPF_LIST_HEAD: return "bpf_list_head"; + case BPF_LIST_NODE: + return "bpf_list_node"; default: WARN_ON_ONCE(1); return "unknown"; @@ -294,6 +297,8 @@ static inline u32 btf_field_type_size(enum btf_field_type type) return sizeof(u64); case BPF_LIST_HEAD: return sizeof(struct bpf_list_head); + case BPF_LIST_NODE: + return sizeof(struct bpf_list_node); default: WARN_ON_ONCE(1); return 0; @@ -312,6 +317,8 @@ static inline u32 btf_field_type_align(enum btf_field_type type) return __alignof__(u64); case BPF_LIST_HEAD: return __alignof__(struct bpf_list_head); + case BPF_LIST_NODE: + return __alignof__(struct bpf_list_node); default: WARN_ON_ONCE(1); return 0; diff --git a/include/linux/btf.h b/include/linux/btf.h index d80345fa566b..a01a8da20021 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -6,6 +6,8 @@ #include #include +#include +#include #include #include @@ -78,6 +80,17 @@ struct btf_id_dtor_kfunc { u32 kfunc_btf_id; }; +struct btf_struct_meta { + u32 btf_id; + struct btf_record *record; + struct btf_field_offs *field_offs; +}; + +struct btf_struct_metas { + u32 cnt; + struct btf_struct_meta types[]; +}; + typedef void (*btf_dtor_kfunc_t)(void *); extern const struct file_operations btf_fops; @@ -408,6 +421,23 @@ static inline struct btf_param *btf_params(const struct btf_type *t) return (struct btf_param *)(t + 1); } +static inline int btf_id_cmp_func(const void *a, const void *b) +{ + const int *pa = a, *pb = b; + + return *pa - *pb; +} + +static inline bool btf_id_set_contains(const struct btf_id_set *set, u32 id) +{ + return bsearch(&id, set->ids, set->cnt, sizeof(u32), btf_id_cmp_func) != NULL; +} + +static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id) +{ + return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func); +} + #ifdef CONFIG_BPF_SYSCALL struct bpf_prog; @@ -423,6 +453,7 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type, s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id); int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt, struct module *owner); +struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id); #else static inline const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id) @@ -454,6 +485,10 @@ static inline int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dt { return 0; } +static inline struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id) +{ + return NULL; +} #endif static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type *t) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 4d6c8577bf17..5e3cffe4bc37 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -237,6 +237,7 @@ struct btf { struct rcu_head rcu; struct btf_kfunc_set_tab *kfunc_set_tab; struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab; + struct btf_struct_metas *struct_meta_tab; /* split BTF support */ struct btf *base_btf; @@ -1642,8 +1643,30 @@ static void btf_free_dtor_kfunc_tab(struct btf *btf) btf->dtor_kfunc_tab = NULL; } +static void btf_struct_metas_free(struct btf_struct_metas *tab) +{ + int i; + + if (!tab) + return; + for (i = 0; i < tab->cnt; i++) { + btf_record_free(tab->types[i].record); + kfree(tab->types[i].field_offs); + } + kfree(tab); +} + +static void btf_free_struct_meta_tab(struct btf *btf) +{ + struct btf_struct_metas *tab = btf->struct_meta_tab; + + btf_struct_metas_free(tab); + btf->struct_meta_tab = NULL; +} + static void btf_free(struct btf *btf) { + btf_free_struct_meta_tab(btf); btf_free_dtor_kfunc_tab(btf); btf_free_kfunc_set_tab(btf); kvfree(btf->types); @@ -3353,6 +3376,12 @@ static int btf_get_field_type(const char *name, u32 field_mask, u32 *seen_mask, goto end; } } + if (field_mask & BPF_LIST_NODE) { + if (!strcmp(name, "bpf_list_node")) { + type = BPF_LIST_NODE; + goto end; + } + } /* Only return BPF_KPTR when all other types with matchable names fail */ if (field_mask & BPF_KPTR) { type = BPF_KPTR_REF; @@ -3398,6 +3427,7 @@ static int btf_find_struct_field(const struct btf *btf, switch (field_type) { case BPF_SPIN_LOCK: case BPF_TIMER: + case BPF_LIST_NODE: ret = btf_find_struct(btf, member_type, off, sz, field_type, idx < info_cnt ? &info[idx] : &tmp); if (ret < 0) @@ -3460,6 +3490,7 @@ static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, switch (field_type) { case BPF_SPIN_LOCK: case BPF_TIMER: + case BPF_LIST_NODE: ret = btf_find_struct(btf, var_type, off, sz, field_type, idx < info_cnt ? &info[idx] : &tmp); if (ret < 0) @@ -3669,6 +3700,8 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type if (ret < 0) goto end; break; + case BPF_LIST_NODE: + break; default: ret = -EFAULT; goto end; @@ -5139,6 +5172,118 @@ static int btf_parse_hdr(struct btf_verifier_env *env) return btf_check_sec_info(env, btf_data_size); } +static const char *local_kptr_fields[] = { + "bpf_spin_lock", + "bpf_list_head", + "bpf_list_node", +}; + +static struct btf_struct_metas * +btf_parse_struct_metas(struct bpf_verifier_log *log, struct btf *btf) +{ + union { + struct btf_id_set set; + struct { + u32 _cnt; + u32 _ids[ARRAY_SIZE(local_kptr_fields)]; + } _arr; + } lkf; + struct btf_struct_metas *tab = NULL; + int i, n, id, ret; + + memset(&lkf, 0, sizeof(lkf)); + + for (i = 0; i < ARRAY_SIZE(local_kptr_fields); i++) { + /* Try to find whether this special type exists in user BTF, and + * if so remember its ID so we can easily find it among members + * of structs that we iterate in the next loop. + */ + id = btf_find_by_name_kind(btf, local_kptr_fields[i], BTF_KIND_STRUCT); + if (id < 0) + continue; + lkf.set.ids[lkf.set.cnt++] = id; + } + + if (!lkf.set.cnt) + return NULL; + sort(&lkf.set.ids, lkf.set.cnt, sizeof(lkf.set.ids[0]), btf_id_cmp_func, NULL); + + n = btf_nr_types(btf); + for (i = 1; i < n; i++) { + const struct btf_member *member; + struct btf_field_offs *foffs; + struct btf_struct_meta *type; + struct btf_record *record; + const struct btf_type *t; + int j; + + t = btf_type_by_id(btf, i); + if (!t) { + ret = -EINVAL; + goto free; + } + if (!__btf_type_is_struct(t)) + continue; + + cond_resched(); + + for_each_member(j, t, member) { + if (btf_id_set_contains(&lkf.set, member->type)) + goto parse; + } + continue; + parse: + if (!tab) { + tab = kzalloc(offsetof(struct btf_struct_metas, types[1]), + GFP_KERNEL | __GFP_NOWARN); + if (!tab) + return ERR_PTR(-ENOMEM); + } else { + struct btf_struct_metas *new_tab; + + new_tab = krealloc(tab, offsetof(struct btf_struct_metas, types[tab->cnt + 1]), + GFP_KERNEL | __GFP_NOWARN); + if (!new_tab) { + ret = -ENOMEM; + goto free; + } + tab = new_tab; + } + type = &tab->types[tab->cnt]; + + type->btf_id = i; + record = btf_parse_fields(btf, t, BPF_SPIN_LOCK | BPF_LIST_HEAD | BPF_LIST_NODE, t->size); + if (IS_ERR_OR_NULL(record)) { + ret = PTR_ERR_OR_ZERO(record) ?: -EFAULT; + goto free; + } + foffs = btf_parse_field_offs(record); + if (WARN_ON_ONCE(IS_ERR_OR_NULL(foffs))) { + btf_record_free(record); + ret = -EFAULT; + goto free; + } + type->record = record; + type->field_offs = foffs; + tab->cnt++; + } + return tab; +free: + btf_struct_metas_free(tab); + return ERR_PTR(ret); +} + +struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id) +{ + struct btf_struct_metas *tab; + + BUILD_BUG_ON(offsetof(struct btf_struct_meta, btf_id) != 0); + tab = btf->struct_meta_tab; + if (!tab) + return NULL; + return bsearch(&btf_id, tab->types, tab->cnt, sizeof(tab->types[0]), btf_id_cmp_func); +} + static int btf_check_type_tags(struct btf_verifier_env *env, struct btf *btf, int start_id) { @@ -5189,6 +5334,7 @@ static int btf_check_type_tags(struct btf_verifier_env *env, static struct btf *btf_parse(bpfptr_t btf_data, u32 btf_data_size, u32 log_level, char __user *log_ubuf, u32 log_size) { + struct btf_struct_metas *struct_meta_tab; struct btf_verifier_env *env = NULL; struct bpf_verifier_log *log; struct btf *btf = NULL; @@ -5257,15 +5403,24 @@ static struct btf *btf_parse(bpfptr_t btf_data, u32 btf_data_size, if (err) goto errout; + struct_meta_tab = btf_parse_struct_metas(log, btf); + if (IS_ERR(struct_meta_tab)) { + err = PTR_ERR(struct_meta_tab); + goto errout; + } + btf->struct_meta_tab = struct_meta_tab; + if (log->level && bpf_verifier_log_full(log)) { err = -ENOSPC; - goto errout; + goto errout_meta; } btf_verifier_env_free(env); refcount_set(&btf->refcnt, 1); return btf; +errout_meta: + btf_free_struct_meta_tab(btf); errout: btf_verifier_env_free(env); if (btf) @@ -6026,6 +6181,28 @@ int btf_struct_access(struct bpf_verifier_log *log, u32 id = reg->btf_id; int err; + while (type_is_local_kptr(reg->type)) { + struct btf_struct_meta *meta; + struct btf_record *rec; + int i; + + meta = btf_find_struct_meta(btf, id); + if (!meta) + break; + rec = meta->record; + for (i = 0; i < rec->cnt; i++) { + struct btf_field *field = &rec->fields[i]; + u32 offset = field->offset; + if (off < offset + btf_field_type_size(field->type) && offset < off + size) { + bpf_log(log, + "direct access to %s is disallowed\n", + btf_field_type_name(field->type)); + return -EACCES; + } + } + break; + } + t = btf_type_by_id(btf, id); do { err = btf_struct_walk(log, btf, t, off, size, &id, &tmp_flag); @@ -7267,23 +7444,6 @@ bool btf_is_module(const struct btf *btf) return btf->kernel_btf && strcmp(btf->name, "vmlinux") != 0; } -static int btf_id_cmp_func(const void *a, const void *b) -{ - const int *pa = a, *pb = b; - - return *pa - *pb; -} - -bool btf_id_set_contains(const struct btf_id_set *set, u32 id) -{ - return bsearch(&id, set->ids, set->cnt, sizeof(u32), btf_id_cmp_func) != NULL; -} - -static void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id) -{ - return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func); -} - enum { BTF_MODULE_F_LIVE = (1 << 0), }; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index fdbae52f463f..c96039a4e57f 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -537,6 +537,7 @@ void btf_record_free(struct btf_record *rec) btf_put(rec->fields[i].kptr.btf); break; case BPF_LIST_HEAD: + case BPF_LIST_NODE: /* Nothing to release for bpf_list_head */ break; default: @@ -582,6 +583,7 @@ struct btf_record *btf_record_dup(const struct btf_record *rec) } break; case BPF_LIST_HEAD: + case BPF_LIST_NODE: /* Nothing to acquire for bpf_list_head */ break; default: @@ -648,6 +650,8 @@ void bpf_obj_free_fields(const struct btf_record *rec, void *obj) continue; bpf_list_head_free(field, field_ptr, obj + rec->spin_lock_off); break; + case BPF_LIST_NODE: + break; default: WARN_ON_ONCE(1); continue; From patchwork Mon Nov 7 23:09:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035450 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9839AC4332F for ; Mon, 7 Nov 2022 23:10:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232506AbiKGXK0 (ORCPT ); Mon, 7 Nov 2022 18:10:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47096 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232448AbiKGXKZ (ORCPT ); Mon, 7 Nov 2022 18:10:25 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C01651FCD6 for ; Mon, 7 Nov 2022 15:10:23 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id 130so12199086pfu.8 for ; Mon, 07 Nov 2022 15:10:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rhA06fQZ2maF9obKz7l2Sp1w6d7RA8pXSzWK85L2jyU=; b=QHGMYFjzwmEXWDVZenwE6xV/M/K0Ia7iFZ7ESbsfqOK6dys0IsF6duGwz8j7BpwFdb xWNzYkWeeTjjR09/0gQsIT+gJJLdZsDmoxJOlv+AC3B0GuW8Ls9yFy2dhylWUOa2ASVA LjTSkIKxy08/eo94UmZg33gRIFPPYHusOdzlFP0omhWdAFdlwxxlnC30L0NXEymq7cyh qx1MFp1Vflm2FWwf34Cs7bCZYGACFHirEhHWk/+8ADSWFWbPENCARsGlfmjqKg3QbkKL donisfv63b3HBhFA/YQqpO2roi/axPOQPnJbPhcUa5aKGhK/L4Vst47UurQLnVxM9CZu tIsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rhA06fQZ2maF9obKz7l2Sp1w6d7RA8pXSzWK85L2jyU=; b=LhDHfg2g7rKYSdiQ//PKgDfpgxIfQT4/DKhV6+gRG5OQI6y2aqGlyQOsNXxSP+Fw7v KI61h3EXNbQyIngtOrewI9+HD1oMYN+4LhwR3ugcDNeYLschKWia9eLAjHbK9qjOrtn+ t+fqQMI/QolktD9/vKeVrBhaEE39ufl+rHYNBqyvsGWEDNHwU84OoQjj4LOxHjir7FXs GANYvlCUBNLX1ywWMq40n/08QtYSY+tsudlLR2RjC91HixeR6S1W8h1ZiO7F8gEpMss/ +MsfOTaa8Y8qKFQClRSKtFiZ8apNpZs98vCb/YnBkWIsuDljps7CPsiS0CK6JV/3njay OfYg== X-Gm-Message-State: ACrzQf1CwQ2IvA5kvpKcWRs8NcqlLc3c/ryu1ssMFguE50c6QygJ+NG6 y0TnJb+JN50ON2J1DVsbdjCcT6zrNHdd5w== X-Google-Smtp-Source: AMsMyM43P9h+kn2wkFYqwdr9GQl3GPUEWXiL7VouQ+L70gpFg3vPwyNtPToMpzzSonYWo7hd+o+Urg== X-Received: by 2002:a63:dd16:0:b0:46f:9c0b:7f82 with SMTP id t22-20020a63dd16000000b0046f9c0b7f82mr40185810pgg.161.1667862622968; Mon, 07 Nov 2022 15:10:22 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id f7-20020a170902ce8700b0018691ce1696sm5532564plg.131.2022.11.07.15.10.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:22 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 08/25] bpf: Verify ownership relationships for user BTF types Date: Tue, 8 Nov 2022 04:39:33 +0530 Message-Id: <20221107230950.7117-9-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5565; i=memxor@gmail.com; h=from:subject; bh=dbxNcx/sah2xoNQU8s4VBd7QRq4f8gXikWmJQ28jzUY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+24y7s5OT1ntUl2PXNooZ2kqLUPBbzeUAc6Lkh x+6Q19uJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8RytW/D/ 99RP7Dit8WDiTaWhaGOmos1rAdxrXm1TAeduRVHzlbFdKW/crNcffKTmgAvi6RTwlJ0rUyANCe4OhQ L37UCinnkjLIjjFu91p8750n/2/ZorPVPpH59hRpM3Xbc9STxjbaDXFA7hoyaRG4pKrl1s5lEMONO2 GOklBPDv1mqaSjGw4gNSNrYAtjdypUBvbDyu1F19tnImQUqiawPS0rDeieqbfQKcSJPGxi61pLyIve uFtNa3Ujr17x69JQK0K9HAlwES8s1PGH0L4xJOipwjsSLRZkAxhtlGBAit9RBcQ8o9oXDWEvaXvqbp 0dSkMjn2hQX6iTdv7q+lUzljA3cHfQQ/Vuq0mRzJXh7oD8kK6z7Cx1td21mIITzRZir6U9IOxRtD3f jVLxVZiB1w/iEzcxqABJW7lkloCkqCEfAK48WRa6YKWZqdABexTEO9WtgErcQvTzeYGruaqpWRyPe8 vlrlpLQhVgPFhkF8ULCcsDs2MUISl2UsagJNpmWoED5TqOQvlv+RNqOMVgZTVnmRnCYuMOWM4XfBAh 0YWwSwvGjqLnuZC+gdEXf0etzW26gwzX1kNduClOmnC7smqW1ynRHxbPlAMsiMl9fnnn46ZnDoD0IY kmZMMUvEOgGG9GslBLZ4aA8OpPJ59wqzcE4vXfLj9QsPNQKltLBFPgE+3Fmw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Ensure that there can be no ownership cycles among different types by way of having owning objects that can hold some other type as their element. For instance, a map value can only hold local kptrs, but these are allowed to have another bpf_list_head. To prevent unbounded recursion while freeing resources, elements of bpf_list_head in local kptrs can never have a bpf_list_head which are part of list in a map value. Later patches will verify this by having dedicated BTF selftests. Also, to make runtime destruction easier, once btf_struct_metas is fully populated, we can stash the metadata of the value type directly in the metadata of the list_head fields, as that allows easier access to the value type's layout to destruct it at runtime from the btf_field entry of the list head itself. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 1 + include/linux/btf.h | 1 + kernel/bpf/btf.c | 71 ++++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 4 +++ 4 files changed, 77 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index dfcd34e36025..88ac4c12bab0 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -190,6 +190,7 @@ struct btf_field_list_head { struct btf *btf; u32 value_btf_id; u32 node_offset; + struct btf_record *value_rec; }; struct btf_field { diff --git a/include/linux/btf.h b/include/linux/btf.h index a01a8da20021..42d8f3730a8d 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -178,6 +178,7 @@ int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t); int btf_find_timer(const struct btf *btf, const struct btf_type *t); struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t, u32 field_mask, u32 value_size); +int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec); struct btf_field_offs *btf_parse_field_offs(struct btf_record *rec); bool btf_type_is_void(const struct btf_type *t); s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind); diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 5e3cffe4bc37..685a6998432b 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3721,6 +3721,67 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type return ERR_PTR(ret); } +int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec) +{ + int i; + + /* There are two owning types, kptr_ref and bpf_list_head. The former + * only supports storing kernel types, which can never store references + * to program allocated local types, atleast not yet. Hence we only need + * to ensure that bpf_list_head ownership does not form cycles. + */ + if (IS_ERR_OR_NULL(rec) || !(rec->field_mask & BPF_LIST_HEAD)) + return 0; + for (i = 0; i < rec->cnt; i++) { + struct btf_struct_meta *meta; + u32 btf_id; + + if (!(rec->fields[i].type & BPF_LIST_HEAD)) + continue; + btf_id = rec->fields[i].list_head.value_btf_id; + meta = btf_find_struct_meta(btf, btf_id); + if (!meta) + return -EFAULT; + rec->fields[i].list_head.value_rec = meta->record; + + if (!(rec->field_mask & BPF_LIST_NODE)) + continue; + + /* We need to ensure ownership acyclicity among all types. The + * proper way to do it would be to topologically sort all BTF + * IDs based on the ownership edges, since there can be multiple + * bpf_list_head in a type. Instead, we use the following + * reasoning: + * + * - A type can only be owned by another type in user BTF if it + * has a bpf_list_node. + * - A type can only _own_ another type in user BTF if it has a + * bpf_list_head. + * + * We ensure that if a type has both bpf_list_head and + * bpf_list_node, its element types cannot be owning types. + * + * To ensure acyclicity: + * + * When A only has bpf_list_head, ownership chain can be: + * A -> B -> C + * Where: + * - B has both bpf_list_head and bpf_list_node. + * - C only has bpf_list_node. + * + * When A has both bpf_list_head and bpf_list_node, some other + * type already owns it in the BTF domain, hence it can not own + * another owning type through any of the bpf_list_head edges. + * A -> B + * Where: + * - B only has bpf_list_node. + */ + if (meta->record->field_mask & BPF_LIST_HEAD) + return -ELOOP; + } + return 0; +} + static int btf_field_offs_cmp(const void *_a, const void *_b, const void *priv) { const u32 a = *(const u32 *)_a; @@ -5410,6 +5471,16 @@ static struct btf *btf_parse(bpfptr_t btf_data, u32 btf_data_size, } btf->struct_meta_tab = struct_meta_tab; + if (struct_meta_tab) { + int i; + + for (i = 0; i < struct_meta_tab->cnt; i++) { + err = btf_check_and_fixup_fields(btf, struct_meta_tab->types[i].record); + if (err < 0) + goto errout_meta; + } + } + if (log->level && bpf_verifier_log_full(log)) { err = -ENOSPC; goto errout_meta; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index c96039a4e57f..4669020bb47d 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1044,6 +1044,10 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, } } + ret = btf_check_and_fixup_fields(btf, map->record); + if (ret < 0) + goto free_map_tab; + if (map->ops->map_check_btf) { ret = map->ops->map_check_btf(map, btf, key_type, value_type); if (ret < 0) From patchwork Mon Nov 7 23:09:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035452 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8EC3C4332F for ; Mon, 7 Nov 2022 23:10:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232740AbiKGXK3 (ORCPT ); Mon, 7 Nov 2022 18:10:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232539AbiKGXK1 (ORCPT ); Mon, 7 Nov 2022 18:10:27 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 785262018E for ; Mon, 7 Nov 2022 15:10:26 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id s196so11869400pgs.3 for ; Mon, 07 Nov 2022 15:10:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oQF9hfPM6rxjnO+dc+1qEodgsXV3OQMlHxdznDYwqLY=; b=UMOlLqp5VWS45L/6NLvW0sA7a3rC3EvYkqIcDMhJFtPVdNgI2eAQPrnzfmevHL63+n OWhCZT+LQWH2BEd4P/Wm9B9fj9GbxLuYHkRn16drkvIZU/79xDcDNCCIR3elW5uEDZvN f3CKOEy6CH27OIUnmEQVjYNfMCyg+m3DDVodWEMBIYYk22oq6cPWVyqoOvv595Oa7N5U 5Y+xibOuDBXDwVMRsd3G7yUng6sD2bQC/K1xCAYcjfK2su7rVBBK3nOpHcBAp540cQfj t9rQqCVVRwTf2IoZEEwnvoDrI+fNsoDpSaRD5hJ9SmINL8AdCkceqGCPSsrwIBGEqwWw LgYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oQF9hfPM6rxjnO+dc+1qEodgsXV3OQMlHxdznDYwqLY=; b=rwUzeeACWYXRFcHxjdL2e+LQ7OGMjtm3DrZOVpC+iHS2z6X5UGwl68VTwYzk4miZ0A G51GsQ81QjPyQC9ceEUawaAgY4N5T5nZJE9+cTcoxCB9Z38u7xULFivZD+t0oVt/gNdr RZGO/1U6pfLl/sopNICATxm2aAhWO4aKN/Z+LR2uvLB+pDNfid8pv9hFARngPvVF77oh V4FKVzi0mcj5B/npFRrXttxRuOEpN3A0uXJwG1jhAl38J2CDhnG1Uq/LmMclG0WJqoS7 T05QhPhEN1z/STVMtt2EkHG9M727NwQSYIL4/+q4CMJGCPDq06ha1b9UNesM+yzb8UpK Rg0w== X-Gm-Message-State: ACrzQf2/2SYEptU82MULVQ/ri1kp5t2GyTM4pZBSUWdYU+mQ/NWjlMm1 e5IxWnSrPcF19sBEJtqU/kdt3Dc653ePgw== X-Google-Smtp-Source: AMsMyM5aTHtcEzH+jkky/+oLh9a4mbt/CW1xngdhedJNsseNEUWGV+lt78W/w/nT1HnM/vZdBtGOqQ== X-Received: by 2002:a65:5386:0:b0:46e:dbd3:413 with SMTP id x6-20020a655386000000b0046edbd30413mr44058866pgq.240.1667862625740; Mon, 07 Nov 2022 15:10:25 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id h11-20020a170902f70b00b00182a9c27acfsm5463171plo.227.2022.11.07.15.10.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:25 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 09/25] bpf: Allow locking bpf_spin_lock in local kptr Date: Tue, 8 Nov 2022 04:39:34 +0530 Message-Id: <20221107230950.7117-10-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6766; i=memxor@gmail.com; h=from:subject; bh=MVQAS2kfnE7vi+V2YwtVErsQEqRsBY+jluWi6u8U8YI=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+2H8ILifZnISrzCP3Ph14a4AqtzweF6qpXkAOp 1quM72qJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8RynCHD/ 9td926X5UvGy4dyKVQDCS8A/XF//m9/mVx1BozmK90YPjgXC3s6ZbxKjoiZE7JGcHsnKVMDDOGl3A0 h/ATq8fJaPL2yreUcuB0VjPJXWaQ/0XzDr2JcBJ88V6ek6UOM/XauTFEQC2WsAoG15k7NxXaz/RThN sIZuARE9SKDF1CC9Z5D04SnhDa2t4uO5YK6/RdcueUI6dO8yHsXfEDz3xfWMKPS8TXPuknOpLGHpS6 Zua+jmOm/bTirkYABSoVDdh62kql9ttGRANvbHcefWJ+fKHBcE3IRac3T3jXZd9YpU9xB2IAlhRcnV /Qgleal06P3mO4xpNhREIOxNBqQHmpp2MWLXL02z26F2utDETp/frloQLy3X532R0LfYSCXAj/OBmX zOqGl0f6jOz/5XA2uG9y4c1hYYv1ihHzy4myQ//4zRExv3hJKP2xf6jrqzksK1shR3jKrzvPr77fvp g/zM5WpoWi01oZ762aImbi3+8udsOobjQ0iIZR9Kkch8RComttLSJqQJ6ZIr8UBuiVwc62TuQL7+wn O+GwTf7sGE65opYMq4Vir1akGZSYN0HR20G1BYXtRYgXDXkbe/2gvzaYX/JkcvDbe4f7tR+kUwCstW z/jssnK+WQnev+KJpS//pkYyzBgRE6CN1igQD8/VUuIM1SY5vBnTEQtOQPGg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Allow locking a bpf_spin_lock embedded in local kptr, in addition to already support map value pointers. The handling is similar to that of map values, by just preserving the reg->id of local kptrs as well, and adjusting process_spin_lock to work with non-PTR_TO_MAP_VALUE and remember the id in verifier state. Refactor the existing process_spin_lock to work with PTR_TO_BTF_ID | MEM_ALLOC in addition to PTR_TO_MAP_VALUE. We need to update the reg_may_point_to_spin_lock which is used in mark_ptr_or_null_reg to preserve reg->id, that will be used in env->cur_state->active_spin_lock to remember the currently held spin lock. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/helpers.c | 2 ++ kernel/bpf/verifier.c | 70 ++++++++++++++++++++++++++++++++----------- 2 files changed, 55 insertions(+), 17 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 7bc71995f17c..5bc0b9f0f306 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -336,6 +336,7 @@ const struct bpf_func_proto bpf_spin_lock_proto = { .gpl_only = false, .ret_type = RET_VOID, .arg1_type = ARG_PTR_TO_SPIN_LOCK, + .arg1_btf_id = BPF_PTR_POISON, }; static inline void __bpf_spin_unlock_irqrestore(struct bpf_spin_lock *lock) @@ -358,6 +359,7 @@ const struct bpf_func_proto bpf_spin_unlock_proto = { .gpl_only = false, .ret_type = RET_VOID, .arg1_type = ARG_PTR_TO_SPIN_LOCK, + .arg1_btf_id = BPF_PTR_POISON, }; void copy_map_value_locked(struct bpf_map *map, void *dst, void *src, diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7dcb4629f764..f1170e9db699 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -453,8 +453,16 @@ static bool reg_type_not_null(enum bpf_reg_type type) static bool reg_may_point_to_spin_lock(const struct bpf_reg_state *reg) { - return reg->type == PTR_TO_MAP_VALUE && - btf_record_has_field(reg->map_ptr->record, BPF_SPIN_LOCK); + struct btf_record *rec = NULL; + + if (reg->type == PTR_TO_MAP_VALUE) { + rec = reg->map_ptr->record; + } else if (reg->type == (PTR_TO_BTF_ID | MEM_ALLOC)) { + struct btf_struct_meta *meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (meta) + rec = meta->record; + } + return btf_record_has_field(rec, BPF_SPIN_LOCK); } static bool type_is_rdonly_mem(u32 type) @@ -5583,8 +5591,10 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[regno]; struct bpf_verifier_state *cur = env->cur_state; bool is_const = tnum_is_const(reg->var_off); - struct bpf_map *map = reg->map_ptr; u64 val = reg->var_off.value; + struct bpf_map *map = NULL; + struct btf_record *rec; + struct btf *btf = NULL; if (!is_const) { verbose(env, @@ -5592,19 +5602,32 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, regno); return -EINVAL; } - if (!map->btf) { - verbose(env, - "map '%s' has to have BTF in order to use bpf_spin_lock\n", - map->name); - return -EINVAL; + if (reg->type == PTR_TO_MAP_VALUE) { + map = reg->map_ptr; + if (!map->btf) { + verbose(env, + "map '%s' has to have BTF in order to use bpf_spin_lock\n", + map->name); + return -EINVAL; + } + rec = map->record; + } else { + struct btf_struct_meta *meta; + + btf = reg->btf; + meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (meta) + rec = meta->record; } - if (!btf_record_has_field(map->record, BPF_SPIN_LOCK)) { - verbose(env, "map '%s' has no valid bpf_spin_lock\n", map->name); + + if (!btf_record_has_field(rec, BPF_SPIN_LOCK)) { + verbose(env, "%s '%s' has no valid bpf_spin_lock\n", map ? "map" : "local", + map ? map->name : "kptr"); return -EINVAL; } - if (map->record->spin_lock_off != val + reg->off) { + if (rec->spin_lock_off != val + reg->off) { verbose(env, "off %lld doesn't point to 'struct bpf_spin_lock' that is at %d\n", - val + reg->off, map->record->spin_lock_off); + val + reg->off, rec->spin_lock_off); return -EINVAL; } if (is_lock) { @@ -5810,13 +5833,19 @@ static const struct bpf_reg_types int_ptr_types = { }, }; +static const struct bpf_reg_types spin_lock_types = { + .types = { + PTR_TO_MAP_VALUE, + PTR_TO_BTF_ID | MEM_ALLOC, + } +}; + static const struct bpf_reg_types fullsock_types = { .types = { PTR_TO_SOCKET } }; static const struct bpf_reg_types scalar_types = { .types = { SCALAR_VALUE } }; static const struct bpf_reg_types context_types = { .types = { PTR_TO_CTX } }; static const struct bpf_reg_types ringbuf_mem_types = { .types = { PTR_TO_MEM | MEM_RINGBUF } }; static const struct bpf_reg_types const_map_ptr_types = { .types = { CONST_PTR_TO_MAP } }; static const struct bpf_reg_types btf_ptr_types = { .types = { PTR_TO_BTF_ID } }; -static const struct bpf_reg_types spin_lock_types = { .types = { PTR_TO_MAP_VALUE } }; static const struct bpf_reg_types percpu_btf_ptr_types = { .types = { PTR_TO_BTF_ID | MEM_PERCPU } }; static const struct bpf_reg_types func_ptr_types = { .types = { PTR_TO_FUNC } }; static const struct bpf_reg_types stack_ptr_types = { .types = { PTR_TO_STACK } }; @@ -5941,6 +5970,11 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, return -EACCES; } } + } else if (type_is_local_kptr(reg->type)) { + if (meta->func_id != BPF_FUNC_spin_lock && meta->func_id != BPF_FUNC_spin_unlock) { + verbose(env, "verifier internal error: unimplemented handling of local kptr\n"); + return -EFAULT; + } } return 0; @@ -6057,7 +6091,8 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, goto skip_type_check; /* arg_btf_id and arg_size are in a union. */ - if (base_type(arg_type) == ARG_PTR_TO_BTF_ID) + if (base_type(arg_type) == ARG_PTR_TO_BTF_ID || + base_type(arg_type) == ARG_PTR_TO_SPIN_LOCK) arg_btf_id = fn->arg_btf_id[arg]; err = check_reg_type(env, regno, arg_type, arg_btf_id, meta); @@ -6675,9 +6710,10 @@ static bool check_btf_id_ok(const struct bpf_func_proto *fn) int i; for (i = 0; i < ARRAY_SIZE(fn->arg_type); i++) { - if (base_type(fn->arg_type[i]) == ARG_PTR_TO_BTF_ID && !fn->arg_btf_id[i]) - return false; - + if (base_type(fn->arg_type[i]) == ARG_PTR_TO_BTF_ID) + return !!fn->arg_btf_id[i]; + if (base_type(fn->arg_type[i]) == ARG_PTR_TO_SPIN_LOCK) + return fn->arg_btf_id[i] == BPF_PTR_POISON; if (base_type(fn->arg_type[i]) != ARG_PTR_TO_BTF_ID && fn->arg_btf_id[i] && /* arg_btf_id and arg_size are in a union. */ (base_type(fn->arg_type[i]) != ARG_PTR_TO_MEM || From patchwork Mon Nov 7 23:09:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035453 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7572C433FE for ; Mon, 7 Nov 2022 23:10:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232890AbiKGXKf (ORCPT ); Mon, 7 Nov 2022 18:10:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232540AbiKGXKa (ORCPT ); Mon, 7 Nov 2022 18:10:30 -0500 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC9CF21255 for ; Mon, 7 Nov 2022 15:10:29 -0800 (PST) Received: by mail-pj1-x1041.google.com with SMTP id l6so12165185pjj.0 for ; Mon, 07 Nov 2022 15:10:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MIhIL4dohOPWzRl2T6wUmZxNEb9GITrVnG/bgpRugSQ=; b=WLRRZ6+WeqHicYGu0U+xoh9qk3/p56N3w+uxJjdIx6Me6Dm3w32BxKX0T0SLgofi+M MnI1XVq50i+1BDzId6us2PLoPlXUNaP45Cu1aOvCKTrh1Cn3Q/tTFSv8V+mgFjG97qfP RPlQ6sH8Gh5iEvuV02OiwqZN7/r8mFCbTeCcYseZ60iJNvMBnhhH98hi1xUP4pZctC+t qxJ5mfw4pKDERl1lLISg57bXuO5gmHpIta8O2ujtiEi8roPJ0P4T1iRmSw/cfKlGI5bx BYjKesfGoe1Xz/UI2Izet0zsiJIAq4AuiJWT6Xbh3KxDi8cD5QgvdtOOhFyD3Uw+igbe y4MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MIhIL4dohOPWzRl2T6wUmZxNEb9GITrVnG/bgpRugSQ=; b=ZO0Q7moJLEmNxxyLlpo01VRV5d4YIzFgLtStwtX2iBvrpAgtECLaYCm8rnpELMFSlu LFsGx3o82o/HTr9tr+hC+SscbkRCTDu3Wbz1pIEnX3AEMn0Q2+dSmkIOZCcjwso2YXlF I2PdxropWIXL0Ofm5qwJMQc/hDHX4bHuiAduJjWOnguNlpYc7lEDI5GxyO+WHU3T4N1D imhBaYSzr4qwTesOyyox2rUP4v3y2u2vPMY0nTZG3UYezgzXcA3314xFMGksnD956DjE QBa00DJSLvlt+heza1dq9Ls6SMxpXL0bF+x+LwruhDyn6Ik3QaQlmfzttVu+zLW/+59Y Culg== X-Gm-Message-State: ACrzQf1aiD8BHtYak6TP3i0uvN7m7w3oVRfuzV4zMYMVacGVPCFSxOxk xRTLmqq1bX87wWIU3mitDc8nhBZLE06zvA== X-Google-Smtp-Source: AMsMyM4iOspFN7t12MUXGMe9ghtYJLmDHe6DhI6t4lAX2XZlcpnoy2VRT9tUgeTjhglnwv8hcgVD7Q== X-Received: by 2002:a17:90b:4ac5:b0:213:e936:a843 with SMTP id mh5-20020a17090b4ac500b00213e936a843mr44487853pjb.156.1667862628971; Mon, 07 Nov 2022 15:10:28 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id d20-20020a170902e15400b00182d25a1e4bsm5455414pla.259.2022.11.07.15.10.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:28 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 10/25] bpf: Allow locking bpf_spin_lock global variables Date: Tue, 8 Nov 2022 04:39:35 +0530 Message-Id: <20221107230950.7117-11-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6765; i=memxor@gmail.com; h=from:subject; bh=UFBOFJul7eTIVrFTDVKBsNyvEBE0cydPvbeqN/Sbkyg=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+2P4r9T5QYfKewSp5U3NNvqyeqM8Tgt0RtJfIm K06P7vqJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8RyugTD/ 9cPEzwBc7gyeLMXytoL6DZQuzQoweFQX4faoVJKfSQyEGncqMBZBMNs9k95HO+GmAlVneJIEbK55sE n+zryU9FizhAQR4kxw+Jzvm/GMsr4bQUO497hMWoVF2GXmPqXSg+fF+ZBJrDTjB1N4/Ht6rWnlORsx PaNcM9jTaMgMQDKoKqKQ7t974ZeMPpY35q4EaTvd1PMGXU927Jk4pwUQUQ0gWaLbh1nsZx4ym6TN9y W2PvAa/jdhPwBU2Uk65FRb8J16EiaDaE3GIx1uWwt7PvyrZI/AzSkJeFOpKCBKVLpl8/efoPKx+M/R mlgpBI0Kwav2BE6aErVwyAN0l9vm/A5zQ9Zoqy5JbZxxSooIwsskUzjCQjX6zBKNu1BX7TxhWtecam 72ZJVCKTmbIWWZ7BHO+go9MBbDfMVB1efQ/+J5mWrsmIdvuDxwbSQGmauaRetS0AlnYFZbRsXN4ryT aVf5LqCeEUuC7HPjjW+pwNgJPADESPJNFAo1/zuIEcDpgFnvNIqgpsZtE4tJvp54z7yLx2IWNLoBQY uYam5hdUQIF3/CuPmt7VUl8ILpvQpPB+s1YeA2KGMYszPkVjyP7PSff5m0Hs2zAqBohhz6T6Pl3FSO C7g7ZxKH45jdjVDyBQJHYSTb/uzXvShoJtQ4EvAw+r4fpLZnqSndriMVO1sg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Global variables reside in maps accessible using direct_value_addr callbacks, so giving each load instruction's rewrite a unique reg->id disallows us from holding locks which are global. The reason for preserving reg->id as a unique value for registers that may point to spin lock is that two separate lookups are treated as two separate memory regions, and any possible aliasing is ignored for the purposes of spin lock correctness. This is not great especially for the global variable case, which are served from maps that have max_entries == 1, i.e. they always lead to map values pointing into the same map value. So refactor the active_spin_lock into a 'active_lock' structure which represents the lock identity, and instead of the reg->id, remember two fields, a pointer and the reg->id. The pointer will store reg->map_ptr or reg->btf. It's only necessary to distinguish for the id == 0 case of global variables, but always setting the pointer to a non-NULL value and using the pointer to check whether the lock is held simplifies code in the verifier. This is generic enough to allow it for global variables, map lookups, and local kptr registers at the same time. Note that while whether a lock is held can be answered by just comparing active_lock.ptr to NULL, to determine whether the register is pointing to the same held lock requires comparing _both_ ptr and id. Finally, as a result of this refactoring, pseudo load instructions are not given a unique reg->id, as they are doing lookup for the same map value (max_entries is never greater than 1). Essentially, we consider that the tuple of (ptr, id) will always be unique for any kind of argument to bpf_spin_{lock,unlock}. Note that this can be extended in the future to also remember offset used for locking, so that we can introduce multiple bpf_spin_lock fields in the same allocation. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf_verifier.h | 5 ++++- kernel/bpf/verifier.c | 41 ++++++++++++++++++++++++------------ 2 files changed, 32 insertions(+), 14 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 1a32baa78ce2..70cccac62a15 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -323,7 +323,10 @@ struct bpf_verifier_state { u32 branches; u32 insn_idx; u32 curframe; - u32 active_spin_lock; + struct { + void *ptr; + u32 id; + } active_lock; bool speculative; /* first and last insn idx of this verifier state */ diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f1170e9db699..281a6a04a0d8 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1210,7 +1210,8 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state, } dst_state->speculative = src->speculative; dst_state->curframe = src->curframe; - dst_state->active_spin_lock = src->active_spin_lock; + dst_state->active_lock.ptr = src->active_lock.ptr; + dst_state->active_lock.id = src->active_lock.id; dst_state->branches = src->branches; dst_state->parent = src->parent; dst_state->first_insn_idx = src->first_insn_idx; @@ -5582,7 +5583,7 @@ int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state * Since only one bpf_spin_lock is allowed the checks are simpler than * reg_is_refcounted() logic. The verifier needs to remember only * one spin_lock instead of array of acquired_refs. - * cur_state->active_spin_lock remembers which map value element got locked + * cur_state->active_lock remembers which map value element got locked * and clears it after bpf_spin_unlock. */ static int process_spin_lock(struct bpf_verifier_env *env, int regno, @@ -5631,22 +5632,35 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, return -EINVAL; } if (is_lock) { - if (cur->active_spin_lock) { + if (cur->active_lock.ptr) { verbose(env, "Locking two bpf_spin_locks are not allowed\n"); return -EINVAL; } - cur->active_spin_lock = reg->id; + if (map) + cur->active_lock.ptr = map; + else + cur->active_lock.ptr = btf; + cur->active_lock.id = reg->id; } else { - if (!cur->active_spin_lock) { + void *ptr; + + if (map) + ptr = map; + else + ptr = btf; + + if (!cur->active_lock.ptr) { verbose(env, "bpf_spin_unlock without taking a lock\n"); return -EINVAL; } - if (cur->active_spin_lock != reg->id) { + if (cur->active_lock.ptr != ptr || + cur->active_lock.id != reg->id) { verbose(env, "bpf_spin_unlock of different lock\n"); return -EINVAL; } - cur->active_spin_lock = 0; + cur->active_lock.ptr = NULL; + cur->active_lock.id = 0; } return 0; } @@ -10573,8 +10587,8 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn) insn->src_reg == BPF_PSEUDO_MAP_IDX_VALUE) { dst_reg->type = PTR_TO_MAP_VALUE; dst_reg->off = aux->map_off; - if (btf_record_has_field(map->record, BPF_SPIN_LOCK)) - dst_reg->id = ++env->id_gen; + WARN_ON_ONCE(map->max_entries != 1); + /* We want reg->id to be same (0) as map_value is not distinct */ } else if (insn->src_reg == BPF_PSEUDO_MAP_FD || insn->src_reg == BPF_PSEUDO_MAP_IDX) { dst_reg->type = CONST_PTR_TO_MAP; @@ -10652,7 +10666,7 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn) return err; } - if (env->cur_state->active_spin_lock) { + if (env->cur_state->active_lock.ptr) { verbose(env, "BPF_LD_[ABS|IND] cannot be used inside bpf_spin_lock-ed region\n"); return -EINVAL; } @@ -11918,7 +11932,8 @@ static bool states_equal(struct bpf_verifier_env *env, if (old->speculative && !cur->speculative) return false; - if (old->active_spin_lock != cur->active_spin_lock) + if (old->active_lock.ptr != cur->active_lock.ptr || + old->active_lock.id != cur->active_lock.id) return false; /* for states to be equal callsites have to be the same @@ -12563,7 +12578,7 @@ static int do_check(struct bpf_verifier_env *env) return -EINVAL; } - if (env->cur_state->active_spin_lock && + if (env->cur_state->active_lock.ptr && (insn->src_reg == BPF_PSEUDO_CALL || insn->imm != BPF_FUNC_spin_unlock)) { verbose(env, "function calls are not allowed while holding a lock\n"); @@ -12600,7 +12615,7 @@ static int do_check(struct bpf_verifier_env *env) return -EINVAL; } - if (env->cur_state->active_spin_lock) { + if (env->cur_state->active_lock.ptr) { verbose(env, "bpf_spin_unlock is missing\n"); return -EINVAL; } From patchwork Mon Nov 7 23:09:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035454 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19945C4332F for ; Mon, 7 Nov 2022 23:10:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232858AbiKGXKi (ORCPT ); Mon, 7 Nov 2022 18:10:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232862AbiKGXKe (ORCPT ); Mon, 7 Nov 2022 18:10:34 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF0E826481 for ; Mon, 7 Nov 2022 15:10:32 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id b1-20020a17090a7ac100b00213fde52d49so11791896pjl.3 for ; Mon, 07 Nov 2022 15:10:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hYvHeXAuVygr4ZcQHncrz4UIBOo5UsmMrsEx+4QLuhc=; b=KgOT7Q9/WhcFpx7HmJQCURFKisq8GuHdB+gcUvUiBH8DeD/u1BC0rhrFgGZOSSELuy jLXZzyUsX4Z0OlGTswbRCvfKlaqAkmrJLx9PfmyrvL/uP1W7GCga9V8J+5N8fR5ZBc73 tAhSznP6D+KFhlPyCWT4RTTG4Eo3hNHKz0zmow1Py3JjbMzQ/BgA09A37VDtGUlw+BZD n1ZpRuQC/0CGBUqDvLVAnNp9+T+5lqMuGKRoQ1UP/Lqlc4+XjoyBLlSMZPjUfEDsmoD+ OpmZE0VhEIwhm4jugiV4fvDlA+GAgHNi9LqsaXoNNg/Iqd8F6dPGeWHSgc3JfPtEP/Ja MDPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hYvHeXAuVygr4ZcQHncrz4UIBOo5UsmMrsEx+4QLuhc=; b=0UnOM3g4pbmReRO0fa0LL22zOhbJ6MjLay/1T7iOs1TZ3C2U5q4WhJv+lDOcTc9YMS 1OpLBcmPSZ7yMeE1KpoLOQWkcHoZQVrfF2L/KAHajgTIy6MxCLt09e5+uwzxYyrVoNtc F1H4JR0Hvqei1FsKhS46dgKO5iHVKFN5AuO8BA1uc4WC2wlmwZy5bcTA16J3C4oFZ9WI boopywiMPtuyvn1le7D5KWY3SYiUqYIzfnort6xLoAq7XoeKCg93JXpY4Jelau4QCoxL 1xQl/qndBPwACqR9/zAqDRfBFx+cmvqpxgJ0z2AzGJRWgSDnsjj/+dYseyTXR9XMV7GO CxsA== X-Gm-Message-State: ACrzQf2HpcJQrdgX66XPrQ+Kf8tpdhBBAYkfPY8zW3Z83Md7rN3hp4TE 9QCF/hoO7ROacQF/qnUorYkmiPR/91j5cw== X-Google-Smtp-Source: AMsMyM7j+ETxjKj9zo/dEf4RapWDjtEuoM4oYw/z5ZS2H+XrLRmPG8lAvel1GnxZm7Hkgbu0tpmGPw== X-Received: by 2002:a17:90a:8a16:b0:213:bc0c:74cd with SMTP id w22-20020a17090a8a1600b00213bc0c74cdmr48425600pjn.28.1667862631949; Mon, 07 Nov 2022 15:10:31 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id x190-20020a6231c7000000b0056bbebbcafbsm5030954pfx.100.2022.11.07.15.10.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:31 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 11/25] bpf: Allow locking bpf_spin_lock in inner map values Date: Tue, 8 Nov 2022 04:39:36 +0530 Message-Id: <20221107230950.7117-12-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1237; i=memxor@gmail.com; h=from:subject; bh=vEeT2/iSWZHC3KMjxfZP18u7ktP/IseF8QZTy1MJn24=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+2Hwor9e+vDGcaQsSs/hJmvUZYJzEVob7xslb7 HsZAtuOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8Rym53D/ 9qlYUDJT1wnzaBE+mzYdRvr3Vrqic+Flle5d8c5gvvBL8zG131Q5J94G7w4guPFVhd7PbnJ7kEX6Xm Ub8pyKdJPACZyyz1j7K8OcKn26SPlqIrG91HIYVBaUKGxkNTPYkeJRJ4GRluScvFCdEBa6ghMm4I95 4oO2rIyPq4Umf0gBM50n8tkYBgWXp7l3YhybUUPfXxBz9ixn9Y1GAYqt9m4EZgAC224F9FnJe1bham LnEEh7o3DqAfR8RBJNalM+J3P+YZf9QCbvFXZQS1fZZlYde2+EsTC/v8qVEw/jJODqMGFpZ6TKj3ce b05I5Yo5XmgsnxSgEbWXsqhm0seaZ1ni05tWSw0kRvyZa7Vg/VprxjXNzgddOHOjjuoRhkqnPyxnah kATuejJsz38ytwGCSsx8mvLTodEglaF4P7M3QBeqSa87deP+dIoJy7DJg+vw0C2upwmBpbtIe+M7Q0 492/KVPMESi5tm917rNYl6oNG+GOB1MvkyiPURuBBjIeaJjSmb5+dawwExpRSuJpoYdDjSUft+Yszh amxAZCNl+jWwN6hyiZEfLO0V5pG5p57hMw7ATWHCQpOAsvhrc5QsuQIq445qLk3J3n6pgA1i4vRWT+ xB9ApQhQ9+m6S/TaW0MFxdWHrr0m7MpULQg8aYo87J20HX6T1kBPJ0ASnnKA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net There is no need to restrict users from locking bpf_spin_lock in map values of inner maps. Each inner map lookup gets a unique reg->id assigned to the returned PTR_TO_MAP_VALUE which will be preserved after the NULL check. Distinct lookups into different inner map get unique IDs, and distinct lookups into same inner map also get unique IDs. Hence, lift the restriction by removing the check return -ENOTSUPP in map_in_map.c. Later commits will add comprehensive test cases to ensure that invalid cases are rejected. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/map_in_map.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/kernel/bpf/map_in_map.c b/kernel/bpf/map_in_map.c index 8ca0cca39d49..f31893a123a2 100644 --- a/kernel/bpf/map_in_map.c +++ b/kernel/bpf/map_in_map.c @@ -29,11 +29,6 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) return ERR_PTR(-ENOTSUPP); } - if (btf_record_has_field(inner_map->record, BPF_SPIN_LOCK)) { - fdput(f); - return ERR_PTR(-ENOTSUPP); - } - inner_map_meta_size = sizeof(*inner_map_meta); /* In some cases verifier needs to access beyond just base map. */ if (inner_map->ops == &array_map_ops) From patchwork Mon Nov 7 23:09:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035455 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 324E9C4332F for ; Mon, 7 Nov 2022 23:10:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232979AbiKGXKu (ORCPT ); Mon, 7 Nov 2022 18:10:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232924AbiKGXKi (ORCPT ); Mon, 7 Nov 2022 18:10:38 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4573628735 for ; Mon, 7 Nov 2022 15:10:36 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id 64so11863508pgc.5 for ; Mon, 07 Nov 2022 15:10:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3x3WNlnnfuF9cDYsyQ/W3xmEQeb6IjCRTBikfVfgTDs=; b=NM6D/f2TLQ+P064iaAvgXZQv5Cjch4VmQr4kj+WO61R01ni2vyUTjBB+RdjCn8YYRS cOyCRAHS17oXqOTw4RAb3J3JPZEikci9UMlHM9IA4GCKqUkO5O+kKoMqZLx+hEXhKb+p 8coCUDHD9WgPr89bu7B5BJwGg7AAEcejNvlMXgNSPzSInvZShAU7ePS7CrcPzx3pul1F qqldq0qlbhb24SkHocARS50ZCPBcecLVpeT9Oy6ybL5UmkvbSM3z2YsPZsfJboIKiZgr n6rcEUX2CcqedOKzPpmKlnt5vEZ8OI1uN8Lu50rjGRTJhDMxl5Q/O0Jb29bfUd0V++3g yTig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3x3WNlnnfuF9cDYsyQ/W3xmEQeb6IjCRTBikfVfgTDs=; b=duEcVeia7OPBia63a0KXe2649u+/0V0U/V5e+2yVgzti8+ysFNEKnb8ufX2VbLAFzH XSpkaJE67RBRWPRhebbIuLXwE7niU78Sw8WPanZWuRWw+FSAS0thXeNdI/jwmAQcN0e0 EFu6CpWNyrLcKu8IBKEEUwTeQxkVP8PjSOjsJkR3TLtWZY+Ywr/LjF2gJV1zO3nBPvzY p+DuhcRlB9uGtqpM0atBXbD2U/1/thbPTX7+w6icOSoo5tpAQvGjt461OYzPulE+FptQ HNO9NwFVpdu5HDpDmwBszQWVs0gkfqR+MCh7XI3Q0x/qstX1inOGtSkG3Mt/QzDPlLFm v3Iw== X-Gm-Message-State: ACrzQf0rWGIPd6cN4lgakbCkK8z4YLB8QCXs7z2hLY4nPt+3JuTnzIZy MrWL5tjtLbbIFnB6FTXowRq9GvhujPYOGg== X-Google-Smtp-Source: AMsMyM70rBX4aEzyt3H9KnabejqJ3mGbnAVqhc2aUpKy/2OkLgmTqdiVoFMKiawM5AxTVhR8u+37PQ== X-Received: by 2002:a63:6b88:0:b0:46a:ff3c:b64a with SMTP id g130-20020a636b88000000b0046aff3cb64amr45022512pgc.196.1667862635016; Mon, 07 Nov 2022 15:10:35 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id s11-20020a170902a50b00b00186b86ed450sm5519551plq.156.2022.11.07.15.10.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:34 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 12/25] bpf: Rewrite kfunc argument handling Date: Tue, 8 Nov 2022 04:39:37 +0530 Message-Id: <20221107230950.7117-13-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=27126; i=memxor@gmail.com; h=from:subject; bh=pGHYSTT01Fnac49XyE9IwGr2Y7kHspWMYQoLPjyVPUE=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+2T4aWOI/gYup8DbmHIlaq+Y8+0K8Og9ebJftN 4PvWfJ6JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8RylQTD/ 9ZK96n5AFU9WmU9K23JC0g/RfNYNznqHhbrcm9Ffpf4jVrLABDBgL1trRcwijfaX9E/xBcGccA/185 oaK1V+Pi8G/1iePSzeYDCS3CXYeGs9PTxZF3l/sh8Q7dlLpENG745/DFAlaqqGbLBFtNgnbDgeXSwh uAkCgqYZhvrfmmKq/FgDZ82E5T1fEffPLjI8NDP/AF+vokjeNiZSSk6bWmqNs4WH2fLmCfVztE8Phw 9KhHGkFlpOIk7zkoLm5f/4jTXK5/Wlj9BChCaUg47aLPd3rmf+jeMqj5LpB0J4o+tNFO412kFjgz98 W7/JXoyXnzuwce6j2Pz4urkC/yQfZXycOmmR2DXbt910yChP0i2pe2fn+U+I61c5HCCP3BEbt+PJxA d415xb23yIX5P+c26jO3y5nj+RnU8pqXbxgjFYE9HhUW6HUdTIFc38dcIUNN/TZa6422Zg5rbOTTs5 SNmq/YvuX0aAQpL1PKZOIVW6wRAyacxcnMHccST8x49rBinoJuOLbiVxOXckYIz7fptAEc3JUKC858 l225zs1zuntK7QTaCQR7ep4Ghzn16sU2vLE7L0pIotRO9cAlUQueg+4nkm5HyY298eDnM2g/ovPP6Z S8IgTbtde/+v1VKXwpzuO1531I6kZGmjpY85cc3OZo+0OfjyL8sDKZCqYXbg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net As we continue to add more features, argument types, kfunc flags, and different extensions to kfuncs, the code to verify the correctness of the kfunc prototype wrt the passed in registers has become ad-hoc and ugly to read. To make life easier, and make a very clear split between different stages of argument processing, move all the code into verifier.c and refactor into easier to read helpers and functions. This also makes sharing code within the verifier easier with kfunc argument processing. This will be more and more useful in later patches as we are now moving to implement very core BPF helpers as kfuncs, to keep them experimental before baking into UAPI. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 31 +- kernel/bpf/btf.c | 16 +- kernel/bpf/verifier.c | 547 +++++++++++++++++- .../bpf/prog_tests/kfunc_dynptr_param.c | 2 +- tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../selftests/bpf/verifier/ref_tracking.c | 4 +- 6 files changed, 569 insertions(+), 33 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index 42d8f3730a8d..d5b26380a60f 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -338,6 +338,16 @@ static inline bool btf_type_is_struct(const struct btf_type *t) return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION; } +static inline bool __btf_type_is_struct(const struct btf_type *t) +{ + return BTF_INFO_KIND(t->info) == BTF_KIND_STRUCT; +} + +static inline bool btf_type_is_array(const struct btf_type *t) +{ + return BTF_INFO_KIND(t->info) == BTF_KIND_ARRAY; +} + static inline u16 btf_type_vlen(const struct btf_type *t) { return BTF_INFO_VLEN(t->info); @@ -439,9 +449,10 @@ static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id) return bsearch(&id, set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func); } -#ifdef CONFIG_BPF_SYSCALL struct bpf_prog; +struct bpf_verifier_log; +#ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id); const char *btf_name_by_offset(const struct btf *btf, u32 offset); struct btf *btf_parse_vmlinux(void); @@ -455,6 +466,12 @@ s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id); int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt, struct module *owner); struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id); +const struct btf_member * +btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, + const struct btf_type *t, enum bpf_prog_type prog_type, + int arg); +bool btf_types_are_same(const struct btf *btf1, u32 id1, + const struct btf *btf2, u32 id2); #else static inline const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id) @@ -490,6 +507,18 @@ static inline struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf { return NULL; } +static inline const struct btf_member * +btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, + const struct btf_type *t, enum bpf_prog_type prog_type, + int arg) +{ + return NULL; +} +static inline bool btf_types_are_same(const struct btf *btf1, u32 id1, + const struct btf *btf2, u32 id2) +{ + return false; +} #endif static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type *t) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 685a6998432b..9d374c76b4a4 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -478,16 +478,6 @@ static bool btf_type_nosize_or_null(const struct btf_type *t) return !t || btf_type_nosize(t); } -static bool __btf_type_is_struct(const struct btf_type *t) -{ - return BTF_INFO_KIND(t->info) == BTF_KIND_STRUCT; -} - -static bool btf_type_is_array(const struct btf_type *t) -{ - return BTF_INFO_KIND(t->info) == BTF_KIND_ARRAY; -} - static bool btf_type_is_datasec(const struct btf_type *t) { return BTF_INFO_KIND(t->info) == BTF_KIND_DATASEC; @@ -5533,7 +5523,7 @@ static u8 bpf_ctx_convert_map[] = { #undef BPF_MAP_TYPE #undef BPF_LINK_TYPE -static const struct btf_member * +const struct btf_member * btf_get_prog_ctx_type(struct bpf_verifier_log *log, const struct btf *btf, const struct btf_type *t, enum bpf_prog_type prog_type, int arg) @@ -6319,8 +6309,8 @@ int btf_struct_access(struct bpf_verifier_log *log, * end up with two different module BTFs, but IDs point to the common type in * vmlinux BTF. */ -static bool btf_types_are_same(const struct btf *btf1, u32 id1, - const struct btf *btf2, u32 id2) +bool btf_types_are_same(const struct btf *btf1, u32 id1, + const struct btf *btf2, u32 id2) { if (id1 != id2) return false; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 281a6a04a0d8..7e50772f9f08 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7850,19 +7850,523 @@ static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, } } +struct bpf_kfunc_call_arg_meta { + /* In parameters */ + struct btf *btf; + u32 func_id; + u32 kfunc_flags; + const struct btf_type *func_proto; + const char *func_name; + /* Out parameters */ + u32 ref_obj_id; + u8 release_regno; + bool r0_rdonly; + u64 r0_size; +}; + +static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_ACQUIRE; +} + +static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_RET_NULL; +} + +static bool is_kfunc_release(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_RELEASE; +} + +static bool is_kfunc_trusted_args(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_TRUSTED_ARGS; +} + +static bool is_kfunc_sleepable(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_SLEEPABLE; +} + +static bool is_kfunc_destructive(struct bpf_kfunc_call_arg_meta *meta) +{ + return meta->kfunc_flags & KF_DESTRUCTIVE; +} + +static bool is_kfunc_arg_kptr_get(struct bpf_kfunc_call_arg_meta *meta, int arg) +{ + return arg == 0 && (meta->kfunc_flags & KF_KPTR_GET); +} + +static bool is_kfunc_arg_mem_size(const struct btf *btf, + const struct btf_param *arg, + const struct bpf_reg_state *reg) +{ + int len, sfx_len = sizeof("__sz") - 1; + const struct btf_type *t; + const char *param_name; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) + return false; + + /* In the future, this can be ported to use BTF tagging */ + param_name = btf_name_by_offset(btf, arg->name_off); + if (str_is_empty(param_name)) + return false; + len = strlen(param_name); + if (len < sfx_len) + return false; + param_name += len - sfx_len; + if (strncmp(param_name, "__sz", sfx_len)) + return false; + + return true; +} + +static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, + const struct btf_param *arg, + const struct bpf_reg_state *reg, + const char *name) +{ + int len, target_len = strlen(name); + const struct btf_type *t; + const char *param_name; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) + return false; + + param_name = btf_name_by_offset(btf, arg->name_off); + if (str_is_empty(param_name)) + return false; + len = strlen(param_name); + if (len != target_len) + return false; + if (strcmp(param_name, name)) + return false; + + return true; +} + +enum { + KF_ARG_DYNPTR_ID, +}; + +BTF_ID_LIST(kf_arg_btf_ids) +BTF_ID(struct, bpf_dynptr_kern) + +static bool is_kfunc_arg_dynptr(const struct btf *btf, + const struct btf_param *arg) +{ + const struct btf_type *t; + u32 res_id; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!t) + return false; + if (!btf_type_is_ptr(t)) + return false; + t = btf_type_skip_modifiers(btf, t->type, &res_id); + if (!t) + return false; + return btf_types_are_same(btf, res_id, btf_vmlinux, kf_arg_btf_ids[KF_ARG_DYNPTR_ID]); +} + +/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ +static bool __btf_type_is_scalar_struct(struct bpf_verifier_env *env, + const struct btf *btf, + const struct btf_type *t, int rec) +{ + const struct btf_type *member_type; + const struct btf_member *member; + u32 i; + + if (!btf_type_is_struct(t)) + return false; + + for_each_member(i, t, member) { + const struct btf_array *array; + + member_type = btf_type_skip_modifiers(btf, member->type, NULL); + if (btf_type_is_struct(member_type)) { + if (rec >= 3) { + verbose(env, "max struct nesting depth exceeded\n"); + return false; + } + if (!__btf_type_is_scalar_struct(env, btf, member_type, rec + 1)) + return false; + continue; + } + if (btf_type_is_array(member_type)) { + array = btf_array(member_type); + if (!array->nelems) + return false; + member_type = btf_type_skip_modifiers(btf, array->type, NULL); + if (!btf_type_is_scalar(member_type)) + return false; + continue; + } + if (!btf_type_is_scalar(member_type)) + return false; + } + return true; +} + + +static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { +#ifdef CONFIG_NET + [PTR_TO_SOCKET] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK], + [PTR_TO_SOCK_COMMON] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK_COMMON], + [PTR_TO_TCP_SOCK] = &btf_sock_ids[BTF_SOCK_TYPE_TCP], +#endif +}; + +enum kfunc_ptr_arg_type { + KF_ARG_PTR_TO_CTX, + KF_ARG_PTR_TO_KPTR_STRONG, /* PTR_TO_KPTR but type specific */ + KF_ARG_PTR_TO_DYNPTR, + KF_ARG_PTR_TO_BTF_ID, /* Also covers reg2btf_ids conversions */ + KF_ARG_PTR_TO_MEM, + KF_ARG_PTR_TO_MEM_SIZE, /* Size derived from next argument, skip it */ +}; + +static enum kfunc_ptr_arg_type +get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, + struct bpf_kfunc_call_arg_meta *meta, + const struct btf_type *t, const struct btf_type *ref_t, + const char *ref_tname, const struct btf_param *args, + int argno, int nargs) +{ + u32 regno = argno + 1; + struct bpf_reg_state *regs = cur_regs(env); + struct bpf_reg_state *reg = ®s[regno]; + bool arg_mem_size = false; + + /* In this function, we verify the kfunc's BTF as per the argument type, + * leaving the rest of the verification with respect to the register + * type to our caller. When a set of conditions hold in the BTF type of + * arguments, we resolve it to a known kfunc_ptr_arg_type. + */ + if (btf_get_prog_ctx_type(&env->log, meta->btf, t, resolve_prog_type(env->prog), argno)) + return KF_ARG_PTR_TO_CTX; + + if (is_kfunc_arg_kptr_get(meta, argno)) { + if (!btf_type_is_ptr(ref_t)) { + verbose(env, "arg#0 BTF type must be a double pointer for kptr_get kfunc\n"); + return -EINVAL; + } + ref_t = btf_type_by_id(meta->btf, ref_t->type); + ref_tname = btf_name_by_offset(meta->btf, ref_t->name_off); + if (!btf_type_is_struct(ref_t)) { + verbose(env, "kernel function %s args#0 pointer type %s %s is not supported\n", + meta->func_name, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + return KF_ARG_PTR_TO_KPTR_STRONG; + } + + if (is_kfunc_arg_dynptr(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_DYNPTR; + + if ((base_type(reg->type) == PTR_TO_BTF_ID || reg2btf_ids[base_type(reg->type)])) { + if (!btf_type_is_struct(ref_t)) { + verbose(env, "kernel function %s args#%d pointer type %s %s is not supported\n", + meta->func_name, argno, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + return KF_ARG_PTR_TO_BTF_ID; + } + + if (argno + 1 < nargs && is_kfunc_arg_mem_size(meta->btf, &args[argno + 1], ®s[regno + 1])) + arg_mem_size = true; + + /* This is the catch all argument type of register types supported by + * check_helper_mem_access. However, we only allow when argument type is + * pointer to scalar, or struct composed (recursively) of scalars. When + * arg_mem_size is true, the pointer can be void *. + */ + if (!btf_type_is_scalar(ref_t) && !__btf_type_is_scalar_struct(env, meta->btf, ref_t, 0) && + (arg_mem_size ? !btf_type_is_void(ref_t) : 1)) { + verbose(env, "arg#%d pointer type %s %s must point to %sscalar, or struct with scalar\n", + argno, btf_type_str(ref_t), ref_tname, arg_mem_size ? "void, " : ""); + return -EINVAL; + } + return arg_mem_size ? KF_ARG_PTR_TO_MEM_SIZE : KF_ARG_PTR_TO_MEM; +} + +static int process_kf_arg_ptr_to_btf_id(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, + const struct btf_type *ref_t, + const char *ref_tname, u32 ref_id, + struct bpf_kfunc_call_arg_meta *meta, + int argno) +{ + const struct btf_type *reg_ref_t; + bool strict_type_match = false; + const struct btf *reg_btf; + const char *reg_ref_tname; + u32 reg_ref_id; + + if (reg->type == PTR_TO_BTF_ID) { + reg_btf = reg->btf; + reg_ref_id = reg->btf_id; + } else { + reg_btf = btf_vmlinux; + reg_ref_id = *reg2btf_ids[base_type(reg->type)]; + } + + if (is_kfunc_trusted_args(meta) || (is_kfunc_release(meta) && reg->ref_obj_id)) + strict_type_match = true; + + reg_ref_t = btf_type_skip_modifiers(reg_btf, reg_ref_id, ®_ref_id); + reg_ref_tname = btf_name_by_offset(reg_btf, reg_ref_t->name_off); + if (!btf_struct_ids_match(&env->log, reg_btf, reg_ref_id, reg->off, meta->btf, ref_id, strict_type_match)) { + verbose(env, "kernel function %s args#%d expected pointer to %s %s but R%d has a pointer to %s %s\n", + meta->func_name, argno, btf_type_str(ref_t), ref_tname, argno + 1, + btf_type_str(reg_ref_t), reg_ref_tname); + return -EINVAL; + } + return 0; +} + +static int process_kf_arg_ptr_to_kptr_strong(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, + const struct btf_type *ref_t, + const char *ref_tname, + struct bpf_kfunc_call_arg_meta *meta, + int argno) +{ + struct btf_field *kptr_field; + + /* check_func_arg_reg_off allows var_off for + * PTR_TO_MAP_VALUE, but we need fixed offset to find + * off_desc. + */ + if (!tnum_is_const(reg->var_off)) { + verbose(env, "arg#0 must have constant offset\n"); + return -EINVAL; + } + + kptr_field = btf_record_find(reg->map_ptr->record, reg->off + reg->var_off.value, BPF_KPTR); + if (!kptr_field || kptr_field->type != BPF_KPTR_REF) { + verbose(env, "arg#0 no referenced kptr at map value offset=%llu\n", + reg->off + reg->var_off.value); + return -EINVAL; + } + + if (!btf_struct_ids_match(&env->log, meta->btf, ref_t->type, 0, kptr_field->kptr.btf, + kptr_field->kptr.btf_id, true)) { + verbose(env, "kernel function %s args#%d expected pointer to %s %s\n", + meta->func_name, argno, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + return 0; +} + +static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta) +{ + const char *func_name = meta->func_name, *ref_tname; + const struct btf *btf = meta->btf; + const struct btf_param *args; + u32 i, nargs; + int ret; + + args = (const struct btf_param *)(meta->func_proto + 1); + nargs = btf_type_vlen(meta->func_proto); + if (nargs > MAX_BPF_FUNC_REG_ARGS) { + verbose(env, "Function %s has %d > %d args\n", func_name, nargs, + MAX_BPF_FUNC_REG_ARGS); + return -EINVAL; + } + + /* Check that BTF function arguments match actual types that the + * verifier sees. + */ + for (i = 0; i < nargs; i++) { + struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[i + 1]; + const struct btf_type *t, *ref_t, *resolve_ret; + enum bpf_arg_type arg_type = ARG_DONTCARE; + u32 regno = i + 1, ref_id, type_size; + bool is_ret_buf_sz = false; + int kf_arg_type; + + t = btf_type_skip_modifiers(btf, args[i].type, NULL); + if (btf_type_is_scalar(t)) { + if (reg->type != SCALAR_VALUE) { + verbose(env, "R%d is not a scalar\n", regno); + return -EINVAL; + } + if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdonly_buf_size")) { + meta->r0_rdonly = true; + is_ret_buf_sz = true; + } else if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdwr_buf_size")) { + is_ret_buf_sz = true; + } + + if (is_ret_buf_sz) { + if (meta->r0_size) { + verbose(env, "2 or more rdonly/rdwr_buf_size parameters for kfunc"); + return -EINVAL; + } + + if (!tnum_is_const(reg->var_off)) { + verbose(env, "R%d is not a const\n", regno); + return -EINVAL; + } + + meta->r0_size = reg->var_off.value; + ret = mark_chain_precision(env, regno); + if (ret) + return ret; + } + continue; + } + + if (!btf_type_is_ptr(t)) { + verbose(env, "Unrecognized arg#%d type %s\n", i, btf_type_str(t)); + return -EINVAL; + } + + if (reg->ref_obj_id) { + if (is_kfunc_release(meta) && meta->ref_obj_id) { + verbose(env, "verifier internal error: more than one arg with ref_obj_id R%d %u %u\n", + regno, reg->ref_obj_id, + meta->ref_obj_id); + return -EFAULT; + } + meta->ref_obj_id = reg->ref_obj_id; + if (is_kfunc_release(meta)) + meta->release_regno = regno; + } + + ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id); + ref_tname = btf_name_by_offset(btf, ref_t->name_off); + + kf_arg_type = get_kfunc_ptr_arg_type(env, meta, t, ref_t, ref_tname, args, i, nargs); + if (kf_arg_type < 0) + return kf_arg_type; + + switch (kf_arg_type) { + case KF_ARG_PTR_TO_BTF_ID: + if (!is_kfunc_trusted_args(meta)) + break; + if (!reg->ref_obj_id) { + verbose(env, "R%d must be referenced\n", regno); + return -EINVAL; + } + fallthrough; + case KF_ARG_PTR_TO_CTX: + /* Trusted arguments have the same offset checks as release arguments */ + arg_type |= OBJ_RELEASE; + break; + case KF_ARG_PTR_TO_KPTR_STRONG: + case KF_ARG_PTR_TO_DYNPTR: + case KF_ARG_PTR_TO_MEM: + case KF_ARG_PTR_TO_MEM_SIZE: + /* Trusted by default */ + break; + default: + WARN_ON_ONCE(1); + return -EFAULT; + } + + if (is_kfunc_release(meta) && reg->ref_obj_id) + arg_type |= OBJ_RELEASE; + ret = check_func_arg_reg_off(env, reg, regno, arg_type); + if (ret < 0) + return ret; + + switch (kf_arg_type) { + case KF_ARG_PTR_TO_CTX: + if (reg->type != PTR_TO_CTX) { + verbose(env, "arg#%d expected pointer to ctx, but got %s\n", i, btf_type_str(t)); + return -EINVAL; + } + break; + case KF_ARG_PTR_TO_KPTR_STRONG: + if (reg->type != PTR_TO_MAP_VALUE) { + verbose(env, "arg#0 expected pointer to map value\n"); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_kptr_strong(env, reg, ref_t, ref_tname, meta, i); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_DYNPTR: + if (reg->type != PTR_TO_STACK) { + verbose(env, "arg#%d expected pointer to stack\n", i); + return -EINVAL; + } + + if (!is_dynptr_reg_valid_init(env, reg)) { + verbose(env, "arg#%d pointer type %s %s must be valid and initialized\n", + i, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + + if (!is_dynptr_type_expected(env, reg, ARG_PTR_TO_DYNPTR | DYNPTR_TYPE_LOCAL)) { + verbose(env, "arg#%d pointer type %s %s points to unsupported dynamic pointer type\n", + i, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + break; + case KF_ARG_PTR_TO_BTF_ID: + /* Only base_type is checked, further checks are done here */ + if (reg->type != PTR_TO_BTF_ID && + (!reg2btf_ids[base_type(reg->type)] || type_flag(reg->type))) { + verbose(env, "arg#%d expected pointer to btf or socket\n", i); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_btf_id(env, reg, ref_t, ref_tname, ref_id, meta, i); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_MEM: + resolve_ret = btf_resolve_size(btf, ref_t, &type_size); + if (IS_ERR(resolve_ret)) { + verbose(env, "arg#%d reference type('%s %s') size cannot be determined: %ld\n", + i, btf_type_str(ref_t), ref_tname, PTR_ERR(resolve_ret)); + return -EINVAL; + } + ret = check_mem_reg(env, reg, regno, type_size); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_MEM_SIZE: + ret = check_kfunc_mem_size_reg(env, ®s[regno + 1], regno + 1); + if (ret < 0) { + verbose(env, "arg#%d arg#%d memory, len pair leads to invalid memory access\n", i, i + 1); + return ret; + } + /* Skip next '__sz' argument */ + i++; + break; + } + } + + if (is_kfunc_release(meta) && !meta->release_regno) { + verbose(env, "release kernel function %s expects refcounted PTR_TO_BTF_ID\n", + func_name); + return -EINVAL; + } + + return 0; +} + static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { const struct btf_type *t, *func, *func_proto, *ptr_type; struct bpf_reg_state *regs = cur_regs(env); - struct bpf_kfunc_arg_meta meta = { 0 }; const char *func_name, *ptr_type_name; + struct bpf_kfunc_call_arg_meta meta; u32 i, nargs, func_id, ptr_type_id; int err, insn_idx = *insn_idx_p; const struct btf_param *args; struct btf *desc_btf; u32 *kfunc_flags; - bool acq; /* skip for now, but return error when we find this in fixup_kfunc_call */ if (!insn->imm) @@ -7883,24 +8387,34 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, func_name); return -EACCES; } - if (*kfunc_flags & KF_DESTRUCTIVE && !capable(CAP_SYS_BOOT)) { - verbose(env, "destructive kfunc calls require CAP_SYS_BOOT capabilities\n"); + + /* Prepare kfunc call metadata */ + memset(&meta, 0, sizeof(meta)); + meta.btf = desc_btf; + meta.func_id = func_id; + meta.kfunc_flags = *kfunc_flags; + meta.func_proto = func_proto; + meta.func_name = func_name; + + if (is_kfunc_destructive(&meta) && !capable(CAP_SYS_BOOT)) { + verbose(env, "destructive kfunc calls require CAP_SYS_BOOT capability\n"); return -EACCES; } - acq = *kfunc_flags & KF_ACQUIRE; - - meta.flags = *kfunc_flags; + if (is_kfunc_sleepable(&meta) && !env->prog->aux->sleepable) { + verbose(env, "program must be sleepable to call sleepable kfunc %s\n", func_name); + return -EACCES; + } /* Check the arguments */ - err = btf_check_kfunc_arg_match(env, desc_btf, func_id, regs, &meta); + err = check_kfunc_args(env, &meta); if (err < 0) return err; /* In case of release function, we get register number of refcounted - * PTR_TO_BTF_ID back from btf_check_kfunc_arg_match, do the release now + * PTR_TO_BTF_ID in bpf_kfunc_arg_meta, do the release now. */ - if (err) { - err = release_reference(env, regs[err].ref_obj_id); + if (meta.release_regno) { + err = release_reference(env, regs[meta.release_regno].ref_obj_id); if (err) { verbose(env, "kfunc %s#%d reference has not been acquired before\n", func_name, func_id); @@ -7914,7 +8428,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, /* Check return type */ t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); - if (acq && !btf_type_is_struct_ptr(desc_btf, t)) { + if (is_kfunc_acquire(&meta) && !btf_type_is_struct_ptr(meta.btf, t)) { verbose(env, "acquire kernel function does not return PTR_TO_BTF_ID\n"); return -EINVAL; } @@ -7953,20 +8467,23 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; } - if (*kfunc_flags & KF_RET_NULL) { + if (is_kfunc_ret_null(&meta)) { regs[BPF_REG_0].type |= PTR_MAYBE_NULL; /* For mark_ptr_or_null_reg, see 93c230e3f5bd6 */ regs[BPF_REG_0].id = ++env->id_gen; } mark_btf_func_reg_size(env, BPF_REG_0, sizeof(void *)); - if (acq) { + if (is_kfunc_acquire(&meta)) { int id = acquire_reference_state(env, insn_idx); if (id < 0) return id; - regs[BPF_REG_0].id = id; + if (is_kfunc_ret_null(&meta)) + regs[BPF_REG_0].id = id; regs[BPF_REG_0].ref_obj_id = id; } + if (reg_may_point_to_spin_lock(®s[BPF_REG_0]) && !regs[BPF_REG_0].id) + regs[BPF_REG_0].id = ++env->id_gen; } /* else { add_kfunc_call() ensures it is btf_type_is_void(t) } */ nargs = btf_type_vlen(func_proto); diff --git a/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c b/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c index c210657d4d0a..55d641c1f126 100644 --- a/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c +++ b/tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c @@ -22,7 +22,7 @@ static struct { "arg#0 pointer type STRUCT bpf_dynptr_kern points to unsupported dynamic pointer type", 0}, {"not_valid_dynptr", "arg#0 pointer type STRUCT bpf_dynptr_kern must be valid and initialized", 0}, - {"not_ptr_to_stack", "arg#0 pointer type STRUCT bpf_dynptr_kern not to stack", 0}, + {"not_ptr_to_stack", "arg#0 expected pointer to stack", 0}, {"dynptr_data_null", NULL, -EBADMSG}, }; diff --git a/tools/testing/selftests/bpf/verifier/calls.c b/tools/testing/selftests/bpf/verifier/calls.c index e1a937277b54..86d6fef2e3b4 100644 --- a/tools/testing/selftests/bpf/verifier/calls.c +++ b/tools/testing/selftests/bpf/verifier/calls.c @@ -109,7 +109,7 @@ }, .prog_type = BPF_PROG_TYPE_SCHED_CLS, .result = REJECT, - .errstr = "arg#0 pointer type STRUCT prog_test_ref_kfunc must point", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_kfunc_call_test_acquire", 3 }, { "bpf_kfunc_call_test_release", 5 }, diff --git a/tools/testing/selftests/bpf/verifier/ref_tracking.c b/tools/testing/selftests/bpf/verifier/ref_tracking.c index f18ce867271f..4784471b0b7f 100644 --- a/tools/testing/selftests/bpf/verifier/ref_tracking.c +++ b/tools/testing/selftests/bpf/verifier/ref_tracking.c @@ -142,7 +142,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_user_key", 2 }, { "bpf_key_put", 4 }, @@ -163,7 +163,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_system_key", 1 }, { "bpf_key_put", 3 }, From patchwork Mon Nov 7 23:09:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035456 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 320D7C433FE for ; Mon, 7 Nov 2022 23:10:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233023AbiKGXKv (ORCPT ); Mon, 7 Nov 2022 18:10:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232940AbiKGXKq (ORCPT ); Mon, 7 Nov 2022 18:10:46 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BE8128E2F for ; Mon, 7 Nov 2022 15:10:39 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id q9so12202790pfg.5 for ; Mon, 07 Nov 2022 15:10:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sj1F8KyJD4UmBUrUzxGAba3VXwEIIrjQo5qALbzhx30=; b=ea5pc/vNUsXGU52oZ/1/eGC1VmKaHnjXJsMv8pnGvK+XjkR/fVDlp3OkNWZiI0EWhl b4cTeQuLev5eO4V464q0iG9fGxBPwHhf/g/CnMsT/DVS+GdTBTPjuLekOuFAtSn4KVh7 JhbUXeHy9QLt1tcCADrGH0+mfzluJJvt2xAz07ejo5q/N3+MqKayTvhwKzn2tc/3J27T GjmD2LrbKOomVaHxpTC3QAZUaP0nEQe36ftHTlI4WhKetlY2I0fbYB2VIfegpjZW7uTA yytRL4jlwxT5jmSAglu6U+1uQjGXokznznnXOV4y+7EtFBu/asdonvWl7bh4bKOdLpBj Eluw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sj1F8KyJD4UmBUrUzxGAba3VXwEIIrjQo5qALbzhx30=; b=suHZ51UZuaAgj9M/9iXBtXEbWjxZlYfTXy0Hn5NqHDr64zWmabtwW5tsUFI7x6UOLS HFx+UtVMdq1Qbzw9Vjm1XvWcuONhE6APQJye8UTzT7pieP7gdlOgZGWeUTMJ1MuBCyLv aP5lk4iK8m1/QsTY4LMebc8ZmVDkOlTtBvOIWQC9GE3GUBAAvNx18zGzpIrRTJMzsYIH EekykV+yKZ6zVP5kF/aqb/V6E9Kc0L90j70uLARtJY7qjO69KbJ2cuRfmED3cAhkdQVm iKUEtck7Bz71ewg8Gh1DYpym5PDwdgeWX4+aeJ0sDShOubs0tJhvaiUDfWhjy1DOJv/h 6zHw== X-Gm-Message-State: ACrzQf3lDTHWkl3ROryscPJN+UzOyrp06Dxd9L73uKwrR+eQaQ4XcCtu HTEaOT6lGmtoJ/4ERUDwalPAxiruuwiqHw== X-Google-Smtp-Source: AMsMyM7wIDWDu787qYLLXh3nSBcORyahoqQYAISk3gs94YaWMAtlg+kFQIpQTXhGZMzXB0pKC0w1eQ== X-Received: by 2002:a05:6a02:119:b0:463:cdc8:a3ff with SMTP id bg25-20020a056a02011900b00463cdc8a3ffmr905483pgb.498.1667862638116; Mon, 07 Nov 2022 15:10:38 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id e12-20020a170902d38c00b0017305e99f00sm5465295pld.107.2022.11.07.15.10.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:37 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 13/25] bpf: Drop kfunc bits from btf_check_func_arg_match Date: Tue, 8 Nov 2022 04:39:38 +0530 Message-Id: <20221107230950.7117-14-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=18717; i=memxor@gmail.com; h=from:subject; bh=+cjgPgWlMLoxtEiHZYDej+a43g6gHtOan4NvZLbANvQ=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+2Hd2TM0mQXg6BoO/831xa4yy0vpWxezJJhWSc 65lbWxSJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtgAKCRBM4MiGSL8RynwhD/ wNaCN/Arng88oErrOoW0bzpTwaw/sWBSSHoHnHZk794QkFzLL3TP7lwMHaB0R1pyrLdLN9SFs3ZQz9 4eo4xQfBumrA0HKUbpueu3Z8LI/GRc9odkD671A4wSAImCpMCM0hDjKUgcDELkF5RKy1OIvmwHxT9c qfTt0CkcKGacMSDNIXXRterSmEQRSbd2EAuMlUdYXr6zWsOyQZm6nH2VsjMAuW6k6mBFFM0CyFd3aT fWEk55F4DuA5caA9OwJ9ciLdAP78I08070b/Yp42a98wZB9U82ahZPROMOxY/8NWtaGgbo91csZEt2 F5FxpUpV0MJU3amqMNYlGspS4RE4cXgQ6kwZIR+kRnSQA6+i24blM2lF632hXrJnRmMMokMb7T65WC 8hH43ZYdiEw3srbqtVGmuQLg6P+F/cHNb7feXnqg3SHQe4p4OIWa8ST520f3FzNtqSVqWRW+diZ1OK 1mbqqXQO3gb8FkZkcXmR11daRrqUVJxRT7UgqusRmgqqY/+7SUgl7dfWwr68SytSlccU81cY6WLGi9 QEWFBc7rKPfMg+lGqL/btjehuKgpRtBqQfbfFng3DNF2aGoDrffdeqIYZoCY+Fc5IYqoaTnYXwDWrr pAL8jAQYNDiO9itlmXaL1rLe5gh1SnHPzIFd0QObHaYdxLa+QQx3FxaXrsBQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Remove all kfunc related bits now from btf_check_func_arg_match, as users have been converted away to refactored kfunc argument handling. This is split into a separate commit to aid review, in order to compare what has been preserved from the removed bits easily instead of mixing removed hunks with previous patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 11 -- include/linux/bpf_verifier.h | 2 - kernel/bpf/btf.c | 364 +---------------------------------- kernel/bpf/verifier.c | 4 +- 4 files changed, 10 insertions(+), 371 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 88ac4c12bab0..7ccd90fd19ba 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2108,22 +2108,11 @@ int btf_distill_func_proto(struct bpf_verifier_log *log, const char *func_name, struct btf_func_model *m); -struct bpf_kfunc_arg_meta { - u64 r0_size; - bool r0_rdonly; - int ref_obj_id; - u32 flags; -}; - struct bpf_reg_state; int btf_check_subprog_arg_match(struct bpf_verifier_env *env, int subprog, struct bpf_reg_state *regs); int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, struct bpf_reg_state *regs); -int btf_check_kfunc_arg_match(struct bpf_verifier_env *env, - const struct btf *btf, u32 func_id, - struct bpf_reg_state *regs, - struct bpf_kfunc_arg_meta *meta); int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog, struct bpf_reg_state *reg); int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *prog, diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 70cccac62a15..f3a601d33fb3 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -592,8 +592,6 @@ int check_ptr_off_reg(struct bpf_verifier_env *env, int check_func_arg_reg_off(struct bpf_verifier_env *env, const struct bpf_reg_state *reg, int regno, enum bpf_arg_type arg_type); -int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, - u32 regno); int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, u32 regno, u32 mem_size); bool is_dynptr_reg_valid_init(struct bpf_verifier_env *env, diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 9d374c76b4a4..bfa142d61571 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6592,122 +6592,19 @@ int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *pr return btf_check_func_type_match(log, btf1, t1, btf2, t2); } -static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { -#ifdef CONFIG_NET - [PTR_TO_SOCKET] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK], - [PTR_TO_SOCK_COMMON] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK_COMMON], - [PTR_TO_TCP_SOCK] = &btf_sock_ids[BTF_SOCK_TYPE_TCP], -#endif -}; - -/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ -static bool __btf_type_is_scalar_struct(struct bpf_verifier_log *log, - const struct btf *btf, - const struct btf_type *t, int rec) -{ - const struct btf_type *member_type; - const struct btf_member *member; - u32 i; - - if (!btf_type_is_struct(t)) - return false; - - for_each_member(i, t, member) { - const struct btf_array *array; - - member_type = btf_type_skip_modifiers(btf, member->type, NULL); - if (btf_type_is_struct(member_type)) { - if (rec >= 3) { - bpf_log(log, "max struct nesting depth exceeded\n"); - return false; - } - if (!__btf_type_is_scalar_struct(log, btf, member_type, rec + 1)) - return false; - continue; - } - if (btf_type_is_array(member_type)) { - array = btf_type_array(member_type); - if (!array->nelems) - return false; - member_type = btf_type_skip_modifiers(btf, array->type, NULL); - if (!btf_type_is_scalar(member_type)) - return false; - continue; - } - if (!btf_type_is_scalar(member_type)) - return false; - } - return true; -} - -static bool is_kfunc_arg_mem_size(const struct btf *btf, - const struct btf_param *arg, - const struct bpf_reg_state *reg) -{ - int len, sfx_len = sizeof("__sz") - 1; - const struct btf_type *t; - const char *param_name; - - t = btf_type_skip_modifiers(btf, arg->type, NULL); - if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) - return false; - - /* In the future, this can be ported to use BTF tagging */ - param_name = btf_name_by_offset(btf, arg->name_off); - if (str_is_empty(param_name)) - return false; - len = strlen(param_name); - if (len < sfx_len) - return false; - param_name += len - sfx_len; - if (strncmp(param_name, "__sz", sfx_len)) - return false; - - return true; -} - -static bool btf_is_kfunc_arg_mem_size(const struct btf *btf, - const struct btf_param *arg, - const struct bpf_reg_state *reg, - const char *name) -{ - int len, target_len = strlen(name); - const struct btf_type *t; - const char *param_name; - - t = btf_type_skip_modifiers(btf, arg->type, NULL); - if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) - return false; - - param_name = btf_name_by_offset(btf, arg->name_off); - if (str_is_empty(param_name)) - return false; - len = strlen(param_name); - if (len != target_len) - return false; - if (strcmp(param_name, name)) - return false; - - return true; -} - static int btf_check_func_arg_match(struct bpf_verifier_env *env, const struct btf *btf, u32 func_id, struct bpf_reg_state *regs, bool ptr_to_mem_ok, - struct bpf_kfunc_arg_meta *kfunc_meta, bool processing_call) { enum bpf_prog_type prog_type = resolve_prog_type(env->prog); - bool rel = false, kptr_get = false, trusted_args = false; - bool sleepable = false; struct bpf_verifier_log *log = &env->log; - u32 i, nargs, ref_id, ref_obj_id = 0; - bool is_kfunc = btf_is_kernel(btf); const char *func_name, *ref_tname; const struct btf_type *t, *ref_t; const struct btf_param *args; - int ref_regno = 0, ret; + u32 i, nargs, ref_id; + int ret; t = btf_type_by_id(btf, func_id); if (!t || !btf_type_is_func(t)) { @@ -6733,14 +6630,6 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, return -EINVAL; } - if (is_kfunc && kfunc_meta) { - /* Only kfunc can be release func */ - rel = kfunc_meta->flags & KF_RELEASE; - kptr_get = kfunc_meta->flags & KF_KPTR_GET; - trusted_args = kfunc_meta->flags & KF_TRUSTED_ARGS; - sleepable = kfunc_meta->flags & KF_SLEEPABLE; - } - /* check that BTF function arguments match actual types that the * verifier sees. */ @@ -6748,42 +6637,9 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, enum bpf_arg_type arg_type = ARG_DONTCARE; u32 regno = i + 1; struct bpf_reg_state *reg = ®s[regno]; - bool obj_ptr = false; t = btf_type_skip_modifiers(btf, args[i].type, NULL); if (btf_type_is_scalar(t)) { - if (is_kfunc && kfunc_meta) { - bool is_buf_size = false; - - /* check for any const scalar parameter of name "rdonly_buf_size" - * or "rdwr_buf_size" - */ - if (btf_is_kfunc_arg_mem_size(btf, &args[i], reg, - "rdonly_buf_size")) { - kfunc_meta->r0_rdonly = true; - is_buf_size = true; - } else if (btf_is_kfunc_arg_mem_size(btf, &args[i], reg, - "rdwr_buf_size")) - is_buf_size = true; - - if (is_buf_size) { - if (kfunc_meta->r0_size) { - bpf_log(log, "2 or more rdonly/rdwr_buf_size parameters for kfunc"); - return -EINVAL; - } - - if (!tnum_is_const(reg->var_off)) { - bpf_log(log, "R%d is not a const\n", regno); - return -EINVAL; - } - - kfunc_meta->r0_size = reg->var_off.value; - ret = mark_chain_precision(env, regno); - if (ret) - return ret; - } - } - if (reg->type == SCALAR_VALUE) continue; bpf_log(log, "R%d is not a scalar\n", regno); @@ -6796,88 +6652,14 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, return -EINVAL; } - /* These register types have special constraints wrt ref_obj_id - * and offset checks. The rest of trusted args don't. - */ - obj_ptr = reg->type == PTR_TO_CTX || reg->type == PTR_TO_BTF_ID || - reg2btf_ids[base_type(reg->type)]; - - /* Check if argument must be a referenced pointer, args + i has - * been verified to be a pointer (after skipping modifiers). - * PTR_TO_CTX is ok without having non-zero ref_obj_id. - */ - if (is_kfunc && trusted_args && (obj_ptr && reg->type != PTR_TO_CTX) && !reg->ref_obj_id) { - bpf_log(log, "R%d must be referenced\n", regno); - return -EINVAL; - } - ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id); ref_tname = btf_name_by_offset(btf, ref_t->name_off); - /* Trusted args have the same offset checks as release arguments */ - if ((trusted_args && obj_ptr) || (rel && reg->ref_obj_id)) - arg_type |= OBJ_RELEASE; ret = check_func_arg_reg_off(env, reg, regno, arg_type); if (ret < 0) return ret; - if (is_kfunc && reg->ref_obj_id) { - /* Ensure only one argument is referenced PTR_TO_BTF_ID */ - if (ref_obj_id) { - bpf_log(log, "verifier internal error: more than one arg with ref_obj_id R%d %u %u\n", - regno, reg->ref_obj_id, ref_obj_id); - return -EFAULT; - } - ref_regno = regno; - ref_obj_id = reg->ref_obj_id; - } - - /* kptr_get is only true for kfunc */ - if (i == 0 && kptr_get) { - struct btf_field *kptr_field; - - if (reg->type != PTR_TO_MAP_VALUE) { - bpf_log(log, "arg#0 expected pointer to map value\n"); - return -EINVAL; - } - - /* check_func_arg_reg_off allows var_off for - * PTR_TO_MAP_VALUE, but we need fixed offset to find - * off_desc. - */ - if (!tnum_is_const(reg->var_off)) { - bpf_log(log, "arg#0 must have constant offset\n"); - return -EINVAL; - } - - kptr_field = btf_record_find(reg->map_ptr->record, reg->off + reg->var_off.value, BPF_KPTR); - if (!kptr_field || kptr_field->type != BPF_KPTR_REF) { - bpf_log(log, "arg#0 no referenced kptr at map value offset=%llu\n", - reg->off + reg->var_off.value); - return -EINVAL; - } - - if (!btf_type_is_ptr(ref_t)) { - bpf_log(log, "arg#0 BTF type must be a double pointer\n"); - return -EINVAL; - } - - ref_t = btf_type_skip_modifiers(btf, ref_t->type, &ref_id); - ref_tname = btf_name_by_offset(btf, ref_t->name_off); - - if (!btf_type_is_struct(ref_t)) { - bpf_log(log, "kernel function %s args#%d pointer type %s %s is not supported\n", - func_name, i, btf_type_str(ref_t), ref_tname); - return -EINVAL; - } - if (!btf_struct_ids_match(log, btf, ref_id, 0, kptr_field->kptr.btf, - kptr_field->kptr.btf_id, true)) { - bpf_log(log, "kernel function %s args#%d expected pointer to %s %s\n", - func_name, i, btf_type_str(ref_t), ref_tname); - return -EINVAL; - } - /* rest of the arguments can be anything, like normal kfunc */ - } else if (btf_get_prog_ctx_type(log, btf, t, prog_type, i)) { + if (btf_get_prog_ctx_type(log, btf, t, prog_type, i)) { /* If function expects ctx type in BTF check that caller * is passing PTR_TO_CTX. */ @@ -6887,109 +6669,10 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, i, btf_type_str(t)); return -EINVAL; } - } else if (is_kfunc && (reg->type == PTR_TO_BTF_ID || - (reg2btf_ids[base_type(reg->type)] && !type_flag(reg->type)))) { - const struct btf_type *reg_ref_t; - const struct btf *reg_btf; - const char *reg_ref_tname; - u32 reg_ref_id; - - if (!btf_type_is_struct(ref_t)) { - bpf_log(log, "kernel function %s args#%d pointer type %s %s is not supported\n", - func_name, i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - if (reg->type == PTR_TO_BTF_ID) { - reg_btf = reg->btf; - reg_ref_id = reg->btf_id; - } else { - reg_btf = btf_vmlinux; - reg_ref_id = *reg2btf_ids[base_type(reg->type)]; - } - - reg_ref_t = btf_type_skip_modifiers(reg_btf, reg_ref_id, - ®_ref_id); - reg_ref_tname = btf_name_by_offset(reg_btf, - reg_ref_t->name_off); - if (!btf_struct_ids_match(log, reg_btf, reg_ref_id, - reg->off, btf, ref_id, - trusted_args || (rel && reg->ref_obj_id))) { - bpf_log(log, "kernel function %s args#%d expected pointer to %s %s but R%d has a pointer to %s %s\n", - func_name, i, - btf_type_str(ref_t), ref_tname, - regno, btf_type_str(reg_ref_t), - reg_ref_tname); - return -EINVAL; - } } else if (ptr_to_mem_ok && processing_call) { const struct btf_type *resolve_ret; u32 type_size; - if (is_kfunc) { - bool arg_mem_size = i + 1 < nargs && is_kfunc_arg_mem_size(btf, &args[i + 1], ®s[regno + 1]); - bool arg_dynptr = btf_type_is_struct(ref_t) && - !strcmp(ref_tname, - stringify_struct(bpf_dynptr_kern)); - - /* Permit pointer to mem, but only when argument - * type is pointer to scalar, or struct composed - * (recursively) of scalars. - * When arg_mem_size is true, the pointer can be - * void *. - * Also permit initialized local dynamic pointers. - */ - if (!btf_type_is_scalar(ref_t) && - !__btf_type_is_scalar_struct(log, btf, ref_t, 0) && - !arg_dynptr && - (arg_mem_size ? !btf_type_is_void(ref_t) : 1)) { - bpf_log(log, - "arg#%d pointer type %s %s must point to %sscalar, or struct with scalar\n", - i, btf_type_str(ref_t), ref_tname, arg_mem_size ? "void, " : ""); - return -EINVAL; - } - - if (arg_dynptr) { - if (reg->type != PTR_TO_STACK) { - bpf_log(log, "arg#%d pointer type %s %s not to stack\n", - i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - if (!is_dynptr_reg_valid_init(env, reg)) { - bpf_log(log, - "arg#%d pointer type %s %s must be valid and initialized\n", - i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - if (!is_dynptr_type_expected(env, reg, - ARG_PTR_TO_DYNPTR | DYNPTR_TYPE_LOCAL)) { - bpf_log(log, - "arg#%d pointer type %s %s points to unsupported dynamic pointer type\n", - i, btf_type_str(ref_t), - ref_tname); - return -EINVAL; - } - - continue; - } - - /* Check for mem, len pair */ - if (arg_mem_size) { - if (check_kfunc_mem_size_reg(env, ®s[regno + 1], regno + 1)) { - bpf_log(log, "arg#%d arg#%d memory, len pair leads to invalid memory access\n", - i, i + 1); - return -EINVAL; - } - i++; - continue; - } - } - resolve_ret = btf_resolve_size(btf, ref_t, &type_size); if (IS_ERR(resolve_ret)) { bpf_log(log, @@ -7002,36 +6685,13 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, if (check_mem_reg(env, reg, regno, type_size)) return -EINVAL; } else { - bpf_log(log, "reg type unsupported for arg#%d %sfunction %s#%d\n", i, - is_kfunc ? "kernel " : "", func_name, func_id); + bpf_log(log, "reg type unsupported for arg#%d function %s#%d\n", i, + func_name, func_id); return -EINVAL; } } - /* Either both are set, or neither */ - WARN_ON_ONCE((ref_obj_id && !ref_regno) || (!ref_obj_id && ref_regno)); - /* We already made sure ref_obj_id is set only for one argument. We do - * allow (!rel && ref_obj_id), so that passing such referenced - * PTR_TO_BTF_ID to other kfuncs works. Note that rel is only true when - * is_kfunc is true. - */ - if (rel && !ref_obj_id) { - bpf_log(log, "release kernel function %s expects refcounted PTR_TO_BTF_ID\n", - func_name); - return -EINVAL; - } - - if (sleepable && !env->prog->aux->sleepable) { - bpf_log(log, "kernel function %s is sleepable but the program is not\n", - func_name); - return -EINVAL; - } - - if (kfunc_meta && ref_obj_id) - kfunc_meta->ref_obj_id = ref_obj_id; - - /* returns argument register number > 0 in case of reference release kfunc */ - return rel ? ref_regno : 0; + return 0; } /* Compare BTF of a function declaration with given bpf_reg_state. @@ -7061,7 +6721,7 @@ int btf_check_subprog_arg_match(struct bpf_verifier_env *env, int subprog, return -EINVAL; is_global = prog->aux->func_info_aux[subprog].linkage == BTF_FUNC_GLOBAL; - err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, NULL, false); + err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, false); /* Compiler optimizations can remove arguments from static functions * or mismatched type can be passed into a global function. @@ -7104,7 +6764,7 @@ int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, return -EINVAL; is_global = prog->aux->func_info_aux[subprog].linkage == BTF_FUNC_GLOBAL; - err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, NULL, true); + err = btf_check_func_arg_match(env, btf, btf_id, regs, is_global, true); /* Compiler optimizations can remove arguments from static functions * or mismatched type can be passed into a global function. @@ -7115,14 +6775,6 @@ int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog, return err; } -int btf_check_kfunc_arg_match(struct bpf_verifier_env *env, - const struct btf *btf, u32 func_id, - struct bpf_reg_state *regs, - struct bpf_kfunc_arg_meta *meta) -{ - return btf_check_func_arg_match(env, btf, func_id, regs, true, meta, true); -} - /* Convert BTF of a function into bpf_reg_state if possible * Returns: * EFAULT - there is a verifier bug. Abort verification. diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7e50772f9f08..c315e8448156 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5539,8 +5539,8 @@ int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, return err; } -int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, - u32 regno) +static int check_kfunc_mem_size_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg, + u32 regno) { struct bpf_reg_state *mem_reg = &cur_regs(env)[regno - 1]; bool may_be_null = type_may_be_null(mem_reg->type); From patchwork Mon Nov 7 23:09:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035457 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90E49C433FE for ; Mon, 7 Nov 2022 23:10:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232959AbiKGXKy (ORCPT ); Mon, 7 Nov 2022 18:10:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232448AbiKGXKr (ORCPT ); Mon, 7 Nov 2022 18:10:47 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B28352A970 for ; Mon, 7 Nov 2022 15:10:41 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id 4so12624048pli.0 for ; Mon, 07 Nov 2022 15:10:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WAdcZjcAFhY0+RHLsYzAKGZ84kaa6r7C4S81z37SiLk=; b=Lv2B5qNxlHf4lh+FRw8YmKuMjkCIvIIotxSsb/fMFNUROiMfST2MyBynXdty92boNi 0wwr4b7kieFUWsgcZkWU2yvik7M0mcc7Qb2hbCcie8WRZum/f5dYduYovaKJjexKSJ/Y Yp93v0D4TRKPqX32nSYBqKrXnvgmT9n1WAhSMaQUtvfiDgpnYLVJMO2+R5OLRwTaZBtF XYPPNwFlVyMWRF2846kqAaJQOQXJIN7x1w93MSKr48lNP8bd2BtIek1UoXMksNVs4vKs TussHAKeKtAzpoAeGuxXw55D1rHI+QLtYyR5tRamudzB7DRX6Xlqzj7N2+tLuREmSUF0 GX3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WAdcZjcAFhY0+RHLsYzAKGZ84kaa6r7C4S81z37SiLk=; b=zuUoXLTlqgzfi5Fvh76AotZLqZVE2Y8qnxNRPo+fT1DH+0WjRBq4R8tDFQAepLEWe/ q1Au8KdsV5pPx2YCCiUsu8T6rwCxaGKcZiYf6lPfGjTNPcj5JQZ8gKsWsOfChbN9IXYR M8GoDhDbKYlZifVvCwHYrhNWglNie3F/vTJzXMiD9HkdcCB+v2MYzgR1x8y6mQsk+uQI 55qITW2xUj+6EbNwafREPXNH+Nt0zJzFL4y7bhLk2Abla9rzkveZvEh1uDJb/UEEoYWl PYQd/Vu8dt1rQEQja3Hg+9dEKEP83HM2288+phcl0Cv3HcXqBub5tw3vBJCseutXX1ms W7UA== X-Gm-Message-State: ACrzQf1zaPgeDqNRjthTJC9Sttb03W/mSeHw+9jGdgQSMLHpYB6zTwdU YSSREUgOouqPTioMyVYdqjQgUT0jPkYSAQ== X-Google-Smtp-Source: AMsMyM7VQUL2wU+pvx9+d47tZnCC9Te6CLZriOvwvay7LSlYDtFmEhu/oB3UtVeuwOEd/iGV7r1CiQ== X-Received: by 2002:a17:902:8541:b0:17f:6b27:7a75 with SMTP id d1-20020a170902854100b0017f6b277a75mr52763900plo.65.1667862640930; Mon, 07 Nov 2022 15:10:40 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id f20-20020aa79694000000b0056c2e497b02sm5198063pfk.173.2022.11.07.15.10.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:40 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 14/25] bpf: Support constant scalar arguments for kfuncs Date: Tue, 8 Nov 2022 04:39:39 +0530 Message-Id: <20221107230950.7117-15-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6082; i=memxor@gmail.com; h=from:subject; bh=Yuk9TSTcr8yxRFSmQ6s6zt0wiBKc8WArN47o6Cs77cE=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+3joqUNUeCBthIGF+ZEXd3VFGAv04+p8TedOga tO7NI2OJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8Rys3XD/ 9WjWhL7Q5TRMo9l2F/NZjTglQYHyOjOdw1Wgnc+iHBInwW8STKPUlKu+w4OFoH60G57QTCii2rxV8D YmyJ6uEJ2avz6Wo/CY9jGk1/BEq0xkyJUitK/6ZOIYKIopRVzyDkpn/lIX2YRQI42GgxQr2OUaxYyG GupFiuYyPGiRM2g3zpr8f/Pd5Te0c8oLC/2R+hh78yu9ALP1ESpKNxiW+FvANx1ja5mKCywEm9GFzg e0/bqmAWyawfn3IT98GEjbsKdgZ82z/5JAkqJezLXalpRA2e2HOhXxDL+svrw8LZdBQFj7HaOoqocU PCYDVBKYKxg8hhADWvVx/kqnX8HS025yqRuLW2Pxm69OGamcEF3yfxFRrvjNerAx4Lb2m7XK+iN58b a4EJI7exJqwzbVPbp5fgIdtFEm3Iwsa0imvZoMOF9LXZZLqzBDDbRSAQEwLjgIzfGrNJV0EzDQob2X nO1A34V1DYbnJhssnOKWjaRTlAqqqq2YsJDt3GxoNA1Qldqx1h6KgEPxGxTfIlRXE5+xXMwrbTlV3G PH31WHEJ3wIz7BB2yYLIirfd8yKxE/gwtt1WM8Uqn3M7wx90fe8MdTY1rrrMYgrFfylAiszNRKSl3Z hS3GquAKJ/TCWC9bnFXBWrcp5F/7Sd4autUlmsFiCXFCfdR67tFp5dinMGqw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Allow passing known constant scalars as arguments to kfuncs that do not represent a size parameter. This makes the search pruning optimization of verifier more conservative for such kfunc calls, and each non-distinct argument is considered unequivalent. We will use this support to then expose a global bpf_kptr_alloc function where it takes the local type ID in program BTF, and returns a PTR_TO_BTF_ID to the local type. These will be called local kptrs, and allows programs to allocate their own objects. However, this is still not completely safe, as mark_chain_precision logic is buggy without more work when the constant argument is not a size, but still needs precise marker propagation for pruning checks. Next patch will fix this problem. Signed-off-by: Kumar Kartikeya Dwivedi --- Documentation/bpf/kfuncs.rst | 30 ++++++++++++++++++ kernel/bpf/verifier.c | 59 +++++++++++++++++++++++++++--------- 2 files changed, 75 insertions(+), 14 deletions(-) diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst index 0f858156371d..7608b066a8ba 100644 --- a/Documentation/bpf/kfuncs.rst +++ b/Documentation/bpf/kfuncs.rst @@ -72,6 +72,36 @@ argument as its size. By default, without __sz annotation, the size of the type of the pointer is used. Without __sz annotation, a kfunc cannot accept a void pointer. +2.2.1 __k Annotation +-------------------- + +This annotation is only understood for scalar arguments, where it indicates that +the verifier must check the scalar argument to be a known constant, which does +not indicate a size parameter. This distinction is important, as when the scalar +argument does not represent a size parameter, verifier is more conservative in +state search pruning and does not consider two arguments equivalent for safety +purposes if the already verified value was within range of the new one. + +This assumption holds well for sizes (as memory accessed within smaller bounds +in old verified state will also work for bigger bounds in current to be explored +state), but not for other constant arguments where each carries a distinct +semantic effect. + +An example is given below:: + + void *bpf_obj_new(u32 local_type_id__k, ...) + { + ... + } + +Here, bpf_obj_new uses local_type_id argument to find out the size of that +type ID in program's BTF and return a sized pointer to it. Each type ID will +have a distinct size, hence it is crucial to treat each such call as distinct +when values don't match. + +Hence, whenever a constant scalar argument is accepted by a kfunc which is not a +size parameter, __k suffix must be used. + .. _BPF_kfunc_nodef: 2.3 Using an existing kernel function diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index c315e8448156..7515b31d2c40 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7862,6 +7862,10 @@ struct bpf_kfunc_call_arg_meta { u8 release_regno; bool r0_rdonly; u64 r0_size; + struct { + u64 value; + bool found; + } arg_constant; }; static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) @@ -7899,30 +7903,40 @@ static bool is_kfunc_arg_kptr_get(struct bpf_kfunc_call_arg_meta *meta, int arg) return arg == 0 && (meta->kfunc_flags & KF_KPTR_GET); } -static bool is_kfunc_arg_mem_size(const struct btf *btf, - const struct btf_param *arg, - const struct bpf_reg_state *reg) +static bool __kfunc_param_match_suffix(const struct btf *btf, + const struct btf_param *arg, + const char *suffix) { - int len, sfx_len = sizeof("__sz") - 1; - const struct btf_type *t; + int suffix_len = strlen(suffix), len; const char *param_name; - t = btf_type_skip_modifiers(btf, arg->type, NULL); - if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) - return false; - /* In the future, this can be ported to use BTF tagging */ param_name = btf_name_by_offset(btf, arg->name_off); if (str_is_empty(param_name)) return false; len = strlen(param_name); - if (len < sfx_len) + if (len < suffix_len) return false; - param_name += len - sfx_len; - if (strncmp(param_name, "__sz", sfx_len)) + param_name += len - suffix_len; + return !strncmp(param_name, suffix, suffix_len); +} + +static bool is_kfunc_arg_mem_size(const struct btf *btf, + const struct btf_param *arg, + const struct bpf_reg_state *reg) +{ + const struct btf_type *t; + + t = btf_type_skip_modifiers(btf, arg->type, NULL); + if (!btf_type_is_scalar(t) || reg->type != SCALAR_VALUE) return false; - return true; + return __kfunc_param_match_suffix(btf, arg, "__sz"); +} + +static bool is_kfunc_arg_sfx_constant(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__k"); } static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, @@ -8198,7 +8212,24 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ verbose(env, "R%d is not a scalar\n", regno); return -EINVAL; } - if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdonly_buf_size")) { + if (is_kfunc_arg_sfx_constant(meta->btf, &args[i])) { + /* kfunc is already bpf_capable() only, no need + * to check it here. + */ + if (meta->arg_constant.found) { + verbose(env, "verifier internal error: only one constant argument permitted\n"); + return -EFAULT; + } + if (!tnum_is_const(reg->var_off)) { + verbose(env, "R%d must be a known constant\n", regno); + return -EINVAL; + } + ret = mark_chain_precision(env, regno); + if (ret < 0) + return ret; + meta->arg_constant.found = true; + meta->arg_constant.value = reg->var_off.value; + } else if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdonly_buf_size")) { meta->r0_rdonly = true; is_ret_buf_sz = true; } else if (is_kfunc_arg_ret_buf_size(btf, &args[i], reg, "rdwr_buf_size")) { From patchwork Mon Nov 7 23:09:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035458 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E1F8C433FE for ; Mon, 7 Nov 2022 23:11:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232846AbiKGXLF (ORCPT ); Mon, 7 Nov 2022 18:11:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233017AbiKGXKv (ORCPT ); Mon, 7 Nov 2022 18:10:51 -0500 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B34021255 for ; Mon, 7 Nov 2022 15:10:45 -0800 (PST) Received: by mail-pg1-x543.google.com with SMTP id 78so11834735pgb.13 for ; Mon, 07 Nov 2022 15:10:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NSeAsJCiRcKWXcAZhLXAJihCbepa9Qc8ZsK5curifAA=; b=gubcaWjmzgDH6nGunEyPf7GtmZMW40kD3fIfCaGQvedoSbKLM4GX8GHnPP68Riq+v4 rehAogchqZUpcqaSh/34CIxnH6ZZKazBqTc8IZWy6mkY3ws60AfUu0rWm542YHympUi3 uw0PQIeJ6NcdCw5x44roLXOkXMJ1XV76MDaLNViAYIK5TWKBYn6q4K2C7wpxUOMkXUOG O9+KE0qwrqz5ME+XkZGWZw7SbUVzwmVRO46hHBYqb+vY5M92L9HuKrtDf0aVIyYTNjb6 0p8fgCpjw1Pbo8RQj5gRPVxkeRKSLr9WrXm+UCB2/ncP8xy7x4qSy58XZlAsHiQuQAN3 MbFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NSeAsJCiRcKWXcAZhLXAJihCbepa9Qc8ZsK5curifAA=; b=bx0KJpGHFS1qqJjrTGP5Hr7Z92Q2ERyR9D9YZRzyuwAFNWFpEYsPpvG6bGOKkNdSZ7 PfoncTR1fa5917+Kni9oq46kppb0ctD4yQ5j8GhnEwD4zZLSR1emA5G+DrkkMIoVYX95 W83avS+YC9v35Eu9JK4OGBSdIR9JJbVuyee04Um+kLF0KJY3vc6WzY3EqtAyoT0G527+ 6hfPAhIUo/21J/yEXY1G0+sq3Wx5DmXUFRc9UPUSFlhcZ8rmDVPkjxJmmLg8KkAqWx+M UXi/BJ9NriExza9ofRkvZhvX1gMzroDL9HwI7qtM5znuM5Pbz0CTE/uWIMv3MCmdYf9K xdfw== X-Gm-Message-State: ACrzQf3YmMCVMOcLDJAIAcwnLiAFSdMCSGpjNt8ODODpVBlRDls2UulH KBFAsNswi57PlAziLliWG1gH7gyifBF2UA== X-Google-Smtp-Source: AMsMyM6D0SfWQK1zfc5soZFf0T2BCtR8dGtSXBOtMtoSMshk9bqHAeTF9qJqR4H25wxYxq7arvSFwA== X-Received: by 2002:a63:5c08:0:b0:46e:e211:5694 with SMTP id q8-20020a635c08000000b0046ee2115694mr45366517pgb.441.1667862644648; Mon, 07 Nov 2022 15:10:44 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id a16-20020a17090a6d9000b001f559e00473sm6602804pjk.43.2022.11.07.15.10.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:44 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 15/25] bpf: Teach verifier about non-size constant arguments Date: Tue, 8 Nov 2022 04:39:40 +0530 Message-Id: <20221107230950.7117-16-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=17169; i=memxor@gmail.com; h=from:subject; bh=hqrkxs8kovv9OVcVUqMt1Typw5KaY/iVu+kPRoR7rUU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+3oZ11spkHSgv0AKOqNt3QZGbKZXgOhJ6otP8n RibXJSSJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8RyjSjD/ 400DPhlJx/D3RlPEYg2PTzSsV7hzpLxZVzeT8PBTpjip70nApFS0Q+RLmJprjizJuybmsEyfGwuk8M p/a3c9oI1qizeEAs+AyFQw7JGohhcdbPfj1wE8p7AC7HI6a0zU38mvHUXTcQRZ+joulqereETgNRhK FkzsB5plF8CXt3VaJ4Rw8Kh2CO4gv2BGB6rKLiQ/Q1UfJ6jNQ/LvBvxJBX+yuRwxDe7QD7rrbcBpuB kk7ZT5cFMf2dtwlV2PIlG/Nq2jix01lSq/Z0VUR2/Wec3W883kyRHGFDBNGG2pE08dB9k+Sz/aNHkV hmmChVhe7Vlhm8cfQTLBRCM7U0mZNApzNvHSYRsvYXKPi12VzwHsdBhwSBrI4U9BKaV8r8OOr3dWth KaELKgVu5HOJkK17Dgc/M1DCW4Tjq1dMbwsxXJwoIonMFapwM5GVqp1WgRf8O7LIO185NJYzBt1dZL 4+BezRMPMaD/TWapnalFR+DtciYQ6GU3jU7BbkkiOULwI2ocmBN8KZXcgziYytm0bW+N+ZNTH9vUg0 JEh1r33tkWIJYCaC8wr9H5M8gbegRfRz+6YnVPmYnxFYj0LsV1dH9xvNMNRZSOnv1DqpqHebJHy+FE EGRwnXaShsdCJt23pvj7onmbxHEqCH2l1vRu4lSC0A1GZ9PryBwyQ78ch78g== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, the verifier has support for various arguments that either describe the size of the memory being passed in to a helper, or describe the size of the memory being returned. When a constant is passed in like this, it is assumed for the purposes of precision tracking that if the value in the already explored safe state is within the value in current state, it would fine to prune the search. While this holds well for size arguments, arguments where each value may denote a distinct meaning and needs to be verified separately needs more work. Search can only be pruned if both are equivalent values. In all other cases, it would be incorrect to treat those two precise registers as equivalent if the new value satisfies the old one (i.e. old <= cur). Hence, make the register precision marker tri-state. There are now three values that reg->precise takes: NOT_PRECISE, PRECISE, EXACT. Both PRECISE and EXACT are 'true' values. EXACT affects how regsafe decides whether both registers are equivalent for the purposes of verifier state equivalence. When it sees that old state register has reg->precise == EXACT, unless both are equivalent, it will return false. Otherwise, for PRECISE case it falls back to the default check that is present now (i.e. thinking that we're talking about sizes). This is required as a future patch introduces a BPF memory allocator interface, where we take the program BTF's type ID as an argument. Each distinct type ID may result in the returned pointer obtaining a different size, hence precision tracking is needed, and pruning cannot just happen when the old value is within the current value. It must only happen when the type ID is equal. The type ID will always correspond to prog->aux->btf hence actual type match is not required. Finally, change mark_chain_precision precision argument to EXACT for kfuncs constant non-size scalar arguments (tagged with __k suffix). Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf_verifier.h | 10 +++- kernel/bpf/verifier.c | 111 ++++++++++++++++++++++------------- 2 files changed, 76 insertions(+), 45 deletions(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index f3a601d33fb3..1e246ec2ff37 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -43,6 +43,12 @@ enum bpf_reg_liveness { REG_LIVE_DONE = 0x8, /* liveness won't be updating this register anymore */ }; +enum bpf_reg_precise { + NOT_PRECISE, + PRECISE, + EXACT, +}; + struct bpf_reg_state { /* Ordering of fields matters. See states_equal() */ enum bpf_reg_type type; @@ -180,7 +186,7 @@ struct bpf_reg_state { s32 subreg_def; enum bpf_reg_liveness live; /* if (!precise && SCALAR_VALUE) min/max/tnum don't affect safety */ - bool precise; + enum bpf_reg_precise precise; }; enum bpf_stack_slot_type { @@ -626,8 +632,6 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, struct bpf_attach_target_info *tgt_info); void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab); -int mark_chain_precision(struct bpf_verifier_env *env, int regno); - #define BPF_BASE_TYPE_MASK GENMASK(BPF_BASE_TYPE_BITS - 1, 0) /* extract base type from bpf_{arg, return, reg}_type. */ diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7515b31d2c40..5bfc151711b9 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -864,7 +864,7 @@ static void print_verifier_state(struct bpf_verifier_env *env, print_liveness(env, reg->live); verbose(env, "="); if (t == SCALAR_VALUE && reg->precise) - verbose(env, "P"); + verbose(env, reg->precise == EXACT ? "E" : "P"); if ((t == SCALAR_VALUE || t == PTR_TO_STACK) && tnum_is_const(reg->var_off)) { /* reg->off should be 0 for SCALAR_VALUE */ @@ -961,7 +961,7 @@ static void print_verifier_state(struct bpf_verifier_env *env, t = reg->type; verbose(env, "=%s", t == SCALAR_VALUE ? "" : reg_type_str(env, t)); if (t == SCALAR_VALUE && reg->precise) - verbose(env, "P"); + verbose(env, reg->precise == EXACT ? "E" : "P"); if (t == SCALAR_VALUE && tnum_is_const(reg->var_off)) verbose(env, "%lld", reg->var_off.value + reg->off); } else { @@ -1695,7 +1695,16 @@ static void __mark_reg_unknown(const struct bpf_verifier_env *env, reg->type = SCALAR_VALUE; reg->var_off = tnum_unknown; reg->frameno = 0; - reg->precise = !env->bpf_capable; + /* Helpers requiring EXACT for constant arguments cannot be called from + * programs without CAP_BPF. This is because we don't propagate + * precision markers when CAP_BPF is missing. If we allowed calling such + * heleprs in those programs, the default would have to be EXACT for + * them, which would be too aggresive, or we'd have to propagate it. + * + * Currently, only bpf_obj_new kfunc requires EXACT precision for its + * scalar argument, which is a kfunc (and thus behind CAP_BPF). + */ + reg->precise = !env->bpf_capable ? PRECISE : NOT_PRECISE; __mark_reg_unbounded(reg); } @@ -2750,7 +2759,8 @@ static int backtrack_insn(struct bpf_verifier_env *env, int idx, * For now backtracking falls back into conservative marking. */ static void mark_all_scalars_precise(struct bpf_verifier_env *env, - struct bpf_verifier_state *st) + struct bpf_verifier_state *st, + enum bpf_reg_precise precise) { struct bpf_func_state *func; struct bpf_reg_state *reg; @@ -2769,7 +2779,7 @@ static void mark_all_scalars_precise(struct bpf_verifier_env *env, reg = &func->regs[j]; if (reg->type != SCALAR_VALUE) continue; - reg->precise = true; + reg->precise = precise; } for (j = 0; j < func->allocated_stack / BPF_REG_SIZE; j++) { if (!is_spilled_reg(&func->stack[j])) @@ -2777,7 +2787,7 @@ static void mark_all_scalars_precise(struct bpf_verifier_env *env, reg = &func->stack[j].spilled_ptr; if (reg->type != SCALAR_VALUE) continue; - reg->precise = true; + reg->precise = precise; } } } @@ -2795,7 +2805,7 @@ static void mark_all_scalars_imprecise(struct bpf_verifier_env *env, struct bpf_ reg = &func->regs[j]; if (reg->type != SCALAR_VALUE) continue; - reg->precise = false; + reg->precise = NOT_PRECISE; } for (j = 0; j < func->allocated_stack / BPF_REG_SIZE; j++) { if (!is_spilled_reg(&func->stack[j])) @@ -2803,7 +2813,7 @@ static void mark_all_scalars_imprecise(struct bpf_verifier_env *env, struct bpf_ reg = &func->stack[j].spilled_ptr; if (reg->type != SCALAR_VALUE) continue; - reg->precise = false; + reg->precise = NOT_PRECISE; } } } @@ -2896,7 +2906,7 @@ static void mark_all_scalars_imprecise(struct bpf_verifier_env *env, struct bpf_ * finalized states which help in short circuiting more future states. */ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int regno, - int spi) + int spi, enum bpf_reg_precise precise) { struct bpf_verifier_state *st = env->cur_state; int first_idx = st->first_insn_idx; @@ -2909,8 +2919,11 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int r bool new_marks = false; int i, err; - if (!env->bpf_capable) + if (!env->bpf_capable) { + /* EXACT precision should only be used with CAP_BPF */ + WARN_ON_ONCE(precise == EXACT); return 0; + } /* Do sanity checks against current state of register and/or stack * slot, but don't set precise flag in current state, as precision @@ -2969,7 +2982,7 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int r reg_mask &= ~(1u << i); continue; } - reg->precise = true; + reg->precise = precise; } return 0; } @@ -2988,7 +3001,7 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int r err = backtrack_insn(env, i, ®_mask, &stack_mask); } if (err == -ENOTSUPP) { - mark_all_scalars_precise(env, st); + mark_all_scalars_precise(env, st, precise); return 0; } else if (err) { return err; @@ -3029,7 +3042,7 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int r } if (!reg->precise) new_marks = true; - reg->precise = true; + reg->precise = precise; } bitmap_from_u64(mask, stack_mask); @@ -3048,7 +3061,7 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int r * fp-8 and it's "unallocated" stack space. * In such case fallback to conservative. */ - mark_all_scalars_precise(env, st); + mark_all_scalars_precise(env, st, precise); return 0; } @@ -3063,7 +3076,7 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int r } if (!reg->precise) new_marks = true; - reg->precise = true; + reg->precise = precise; } if (env->log.level & BPF_LOG_LEVEL2) { verbose(env, "parent %s regs=%x stack=%llx marks:", @@ -3083,19 +3096,22 @@ static int __mark_chain_precision(struct bpf_verifier_env *env, int frame, int r return 0; } -int mark_chain_precision(struct bpf_verifier_env *env, int regno) +static int mark_chain_precision(struct bpf_verifier_env *env, int regno, + enum bpf_reg_precise precise) { - return __mark_chain_precision(env, env->cur_state->curframe, regno, -1); + return __mark_chain_precision(env, env->cur_state->curframe, regno, -1, precise); } -static int mark_chain_precision_frame(struct bpf_verifier_env *env, int frame, int regno) +static int mark_chain_precision_frame(struct bpf_verifier_env *env, int frame, int regno, + enum bpf_reg_precise precise) { - return __mark_chain_precision(env, frame, regno, -1); + return __mark_chain_precision(env, frame, regno, -1, precise); } -static int mark_chain_precision_stack_frame(struct bpf_verifier_env *env, int frame, int spi) +static int mark_chain_precision_stack_frame(struct bpf_verifier_env *env, int frame, int spi, + enum bpf_reg_precise precise) { - return __mark_chain_precision(env, frame, -1, spi); + return __mark_chain_precision(env, frame, -1, spi, precise); } static bool is_spillable_regtype(enum bpf_reg_type type) @@ -3230,7 +3246,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env, * Backtrack from here and mark all registers as precise * that contributed into 'reg' being a constant. */ - err = mark_chain_precision(env, value_regno); + err = mark_chain_precision(env, value_regno, PRECISE); if (err) return err; } @@ -3271,7 +3287,7 @@ static int check_stack_write_fixed_off(struct bpf_verifier_env *env, /* when we zero initialize stack slots mark them as such */ if (reg && register_is_null(reg)) { /* backtracking doesn't work for STACK_ZERO yet. */ - err = mark_chain_precision(env, value_regno); + err = mark_chain_precision(env, value_regno, PRECISE); if (err) return err; type = STACK_ZERO; @@ -3387,7 +3403,7 @@ static int check_stack_write_var_off(struct bpf_verifier_env *env, } if (zero_used) { /* backtracking doesn't work for STACK_ZERO yet. */ - err = mark_chain_precision(env, value_regno); + err = mark_chain_precision(env, value_regno, PRECISE); if (err) return err; } @@ -3436,7 +3452,7 @@ static void mark_reg_stack_read(struct bpf_verifier_env *env, * backtracking. Any register that contributed * to const 0 was marked precise before spill. */ - state->regs[dst_regno].precise = true; + state->regs[dst_regno].precise = PRECISE; } else { /* have read misc data from the stack */ mark_reg_unknown(env, state->regs, dst_regno); @@ -5503,7 +5519,7 @@ static int check_mem_size_reg(struct bpf_verifier_env *env, reg->umax_value, zero_size_allowed, meta); if (!err) - err = mark_chain_precision(env, regno); + err = mark_chain_precision(env, regno, PRECISE); return err; } @@ -6311,7 +6327,7 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, return -EACCES; } meta->mem_size = reg->var_off.value; - err = mark_chain_precision(env, regno); + err = mark_chain_precision(env, regno, PRECISE); if (err) return err; break; @@ -7303,7 +7319,7 @@ record_func_key(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta, return 0; } - err = mark_chain_precision(env, BPF_REG_3); + err = mark_chain_precision(env, BPF_REG_3, PRECISE); if (err) return err; if (bpf_map_key_unseen(aux)) @@ -7403,7 +7419,7 @@ static bool loop_flag_is_zero(struct bpf_verifier_env *env) bool reg_is_null = register_is_null(reg); if (reg_is_null) - mark_chain_precision(env, BPF_REG_4); + mark_chain_precision(env, BPF_REG_4, PRECISE); return reg_is_null; } @@ -8224,7 +8240,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ verbose(env, "R%d must be a known constant\n", regno); return -EINVAL; } - ret = mark_chain_precision(env, regno); + ret = mark_chain_precision(env, regno, EXACT); if (ret < 0) return ret; meta->arg_constant.found = true; @@ -8248,7 +8264,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ } meta->r0_size = reg->var_off.value; - ret = mark_chain_precision(env, regno); + ret = mark_chain_precision(env, regno, PRECISE); if (ret) return ret; } @@ -9929,7 +9945,7 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, * This is legal, but we have to reverse our * src/dest handling in computing the range */ - err = mark_chain_precision(env, insn->dst_reg); + err = mark_chain_precision(env, insn->dst_reg, PRECISE); if (err) return err; return adjust_ptr_min_max_vals(env, insn, @@ -9937,14 +9953,16 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, } } else if (ptr_reg) { /* pointer += scalar */ - err = mark_chain_precision(env, insn->src_reg); + err = mark_chain_precision(env, insn->src_reg, PRECISE); if (err) return err; return adjust_ptr_min_max_vals(env, insn, dst_reg, src_reg); } else if (dst_reg->precise) { - /* if dst_reg is precise, src_reg should be precise as well */ - err = mark_chain_precision(env, insn->src_reg); + /* If dst_reg is precise, src_reg should be precise as well. + * Propagate EXACT if it is exact, PRECISE if precise. + */ + err = mark_chain_precision(env, insn->src_reg, dst_reg->precise); if (err) return err; } @@ -10938,10 +10956,10 @@ static int check_cond_jmp_op(struct bpf_verifier_env *env, * above is_branch_taken() special cased the 0 comparison. */ if (!__is_pointer_value(false, dst_reg)) - err = mark_chain_precision(env, insn->dst_reg); + err = mark_chain_precision(env, insn->dst_reg, PRECISE); if (BPF_SRC(insn->code) == BPF_X && !err && !__is_pointer_value(false, src_reg)) - err = mark_chain_precision(env, insn->src_reg); + err = mark_chain_precision(env, insn->src_reg, PRECISE); if (err) return err; } @@ -12262,9 +12280,18 @@ static bool regsafe(struct bpf_verifier_env *env, struct bpf_reg_state *rold, if (rcur->type == SCALAR_VALUE) { if (!rold->precise) return true; - /* new val must satisfy old val knowledge */ - return range_within(rold, rcur) && - tnum_in(rold->var_off, rcur->var_off); + /* For EXACT, we need values to match exactly, so simply + * return false as the memcmp above failed already, + * otherwise current being within the old value + * suffices. + */ + if (rold->precise == EXACT) { + return false; + } else /* PRECISE */ { + /* new val must satisfy old val knowledge */ + return range_within(rold, rcur) && + tnum_in(rold->var_off, rcur->var_off); + } } else { /* We're trying to use a pointer in place of a scalar. * Even if the scalar was unbounded, this could lead to @@ -12595,7 +12622,7 @@ static int propagate_precision(struct bpf_verifier_env *env, continue; if (env->log.level & BPF_LOG_LEVEL2) verbose(env, "frame %d: propagating r%d\n", i, fr); - err = mark_chain_precision_frame(env, fr, i); + err = mark_chain_precision_frame(env, fr, i, state_reg->precise); if (err < 0) return err; } @@ -12610,7 +12637,7 @@ static int propagate_precision(struct bpf_verifier_env *env, if (env->log.level & BPF_LOG_LEVEL2) verbose(env, "frame %d: propagating fp%d\n", (-i - 1) * BPF_REG_SIZE, fr); - err = mark_chain_precision_stack_frame(env, fr, i); + err = mark_chain_precision_stack_frame(env, fr, i, state_reg->precise); if (err < 0) return err; } From patchwork Mon Nov 7 23:09:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035459 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94F82C433FE for ; Mon, 7 Nov 2022 23:11:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232929AbiKGXLO (ORCPT ); Mon, 7 Nov 2022 18:11:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233059AbiKGXLA (ORCPT ); Mon, 7 Nov 2022 18:11:00 -0500 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 018E12A712 for ; Mon, 7 Nov 2022 15:10:59 -0800 (PST) Received: by mail-pg1-x541.google.com with SMTP id v3so11862866pgh.4 for ; Mon, 07 Nov 2022 15:10:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qADuCkr4Gi4q2sb1oTDAuYkeL+jn0mL30LA3Q+LueNk=; b=X9b6FxDYnygNiNaNhuMBaedmzr8tnZ3OndAFwxgOHK0Hbvzmp81bjbEgckAUp4GAnx BLFdTCV8tIOht39inrYBhEUGQTotaLTs6guCqYm6npy1OfaOebhdPPMASbczBRie1UKm YG0Yd6xyuDtkKa95Twkk/d2kzjpJ8H0eGnkw/5QBnaH3i0oJqPq2tkU2ApLX+yEIi3Kq LUU1H0GBPQXZGkIuGNN1NVZfzXwIc8pwDJdJyhfhJHmcst4oapNxhrmq0bVuujhSua7l bgHrTyJ+6ofbNikBFtocgrNECSfLokF/5vBrB696TAwfQbb2h9Mh5lz40XyLRTHNePi1 i8uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qADuCkr4Gi4q2sb1oTDAuYkeL+jn0mL30LA3Q+LueNk=; b=CxpdxHz55Kkck3MVzdyRIuQCXjvAddOfrxyf4sGbvuiBRUOnczWwiZokma/tin5ken ucYsdsmPej6QXqzgsHDgI+MlEMMXnV/F5REQRwnXeEo8kQvFvkBL+HuGG5f3Ta6qYZex FYbK1rRnaA4/UawsoSO0XNGJRue/rOOUSQIacFi9NVrveaKXHIagQt++ZcGOPBLlv0KP PusuemzlaBNKtkmUPuqmCaSESZ3ExKDkBHriE3r8dDFAclvJ+yERmS4mG7WiiDDVa7yf GzrHwNORGXETu71Wtw+7aV7PXL662sxqxOr+D/tHTcubf7+qGF1TUhq7PqR1IDP1ICvJ hcGQ== X-Gm-Message-State: ACrzQf0qb18SBc/7kMnn4trFczQZwOrH0jSEWfBSlW+3YeLGSwnDEY1b +Coj9FtOswFAyxTRV7f6s/iYeRNRqtyqtg== X-Google-Smtp-Source: AMsMyM5zVuPVX+1J6q+2d9+eB/oATDCUuibgtZqEqHhj5t1olOXMYRPtDWrD42cNlCSORhlaQ7lp+w== X-Received: by 2002:a63:66c3:0:b0:470:8fd:7bae with SMTP id a186-20020a6366c3000000b0047008fd7baemr27006521pgc.277.1667862658122; Mon, 07 Nov 2022 15:10:58 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id g5-20020a625205000000b0056bb191f176sm5197503pfb.14.2022.11.07.15.10.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:10:57 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 16/25] bpf: Introduce bpf_obj_new Date: Tue, 8 Nov 2022 04:39:41 +0530 Message-Id: <20221107230950.7117-17-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=15797; i=memxor@gmail.com; h=from:subject; bh=NrvT/wFp9kxNbwjHPoyQZ84lXDmMo0ceWWoivMZtT1g=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+37Bkub57CIsW76TzJ/WU9AJG7G3xwjSdm9ZLL j7VVgQ6JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8RyheiD/ wMYuyyS6xR7+RMudM7dBiGbEcK8eQXNHV2loJvzPCSjxjwKBMW9vVxk7+RrEUj9sYq1OrIXLG1jmhY 102QG34QD0P19zBPvzfgDAHhTqXwyUUyF6GGb006BVnlDslr8cSxXROFvTfCYNPtuciizogYiX41SX Xn0fghr26I2l8A3bTG/jybirgPoB8mRLT23qgSRwEfGP2At/+9TvEo7cGYDeZ0+v9RTfSRcvZaFhfD RX21kHRRVs+6yR7nC7gKPB0C5LXfwhhzzgdPjRJa8C2qqAqOyuyTaAGv6S5IeRrqlXD55wzcg/Trrl DEw6IcEfX3YaKPZXHpMUnVE65cxBDGwnNi8vJ6C3faTsxufeTkK3VjbaFz46KvNZgfkJrVL+rgYkgS izTXmP4Jx6kfUytJ3Op4SPqkAlCv4Hizdymc1d52lyKWx/VdB+TBoxRMkpVPikWRHbd6vETHZNQgkw ziD06CuTrBlCqqNM/zvxoQaBdPtCf+VrMF+Ziof8GRXLiVyE3BqltOAymEJbF2US2lLFOsVWdD0axJ 1rP4k/CaXM8o5aUcp5xlyp7qZnABsZaX+AsluKF/QtA+0yb9xaZaLMtrWt0cwF+dwnvETgNuJiQjeP bVpRazWSwhHWgWFM9iPN9dFlAr5lybBBVJdgxAJVdQHXIzO05bgQVd+dhlGg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Introduce type safe memory allocator bpf_obj_new for BPF programs. The kernel side kfunc is named bpf_obj_new_impl, as passing hidden arguments to kfuncs still requires having them in prototype, unlike BPF helpers which always take 5 arguments and have them checked using bpf_func_proto in verifier, ignoring unset argument types. Introduce __ign suffix to ignore a specific kfunc argument during type checks, then use this to introduce support for passing type metadata to the bpf_obj_new_impl kfunc. The user passes BTF ID of the type it wants to allocates in program BTF, the verifier then rewrites the first argument as the size of this type, after performing some sanity checks (to ensure it exists and it is a struct type). The second argument is also fixed up and passed by the verifier. This is the btf_struct_meta for the type being allocated. It would be needed mostly for the offset array which is required for zero initializing special fields while leaving the rest of storage in unitialized state. It would also be needed in the next patch to perform proper destruction of the object's special fields. Under the hood, bpf_obj_new will call bpf_mem_alloc and bpf_mem_free, using the any context BPF memory allocator introduced recently. To this end, a global instance of the BPF memory allocator is initialized on boot to be used for this purpose. This 'bpf_global_ma' serves all allocations for bpf_obj_new. In the future, bpf_obj_new variants will allow specifying a custom allocator. Note that now that bpf_obj_new can be used to allocate local kptrs that can be linked to BPF linked list (when future linked list helpers are available), we need to also free the elements using bpf_mem_free. However, since the draining of elements is done outside the bpf_spin_lock, we need to do migrate_disable around the call since bpf_list_head_free can be called from map free path where migration is enabled. Otherwise, when called from BPF programs migration is already disabled. A convenience macro is included in the bpf_experimental.h header to hide over the ugly details of the implementation, leading to user code looking similar to a language level extension which allocates and constructs fields of a user type. struct bar { struct bpf_list_node node; }; struct foo { struct bpf_spin_lock lock; struct bpf_list_head head __contains(bar, node); }; void prog(void) { struct foo *f; f = bpf_obj_new(typeof(*f)); if (!f) return; ... } A key piece of this story is still missing, i.e. the free function, which will come in the next patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 21 ++-- include/linux/bpf_verifier.h | 2 + kernel/bpf/core.c | 16 +++ kernel/bpf/helpers.c | 47 ++++++-- kernel/bpf/verifier.c | 107 ++++++++++++++++-- .../testing/selftests/bpf/bpf_experimental.h | 25 ++++ 6 files changed, 195 insertions(+), 23 deletions(-) create mode 100644 tools/testing/selftests/bpf/bpf_experimental.h diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 7ccd90fd19ba..9b40e4c7db92 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -54,6 +54,8 @@ struct cgroup; extern struct idr btf_idr; extern spinlock_t btf_idr_lock; extern struct kobject *btf_kobj; +extern struct bpf_mem_alloc bpf_global_ma; +extern bool bpf_global_ma_set; typedef u64 (*bpf_callback_t)(u64, u64, u64, u64, u64); typedef int (*bpf_iter_init_seq_priv_t)(void *private_data, @@ -333,16 +335,19 @@ static inline bool btf_record_has_field(const struct btf_record *rec, enum btf_f return rec->field_mask & type; } -static inline void check_and_init_map_value(struct bpf_map *map, void *dst) +static inline void bpf_obj_init(const struct btf_field_offs *foffs, void *obj) { - if (!IS_ERR_OR_NULL(map->record)) { - struct btf_field *fields = map->record->fields; - u32 cnt = map->record->cnt; - int i; + int i; - for (i = 0; i < cnt; i++) - memset(dst + fields[i].offset, 0, btf_field_type_size(fields[i].type)); - } + if (!foffs) + return; + for (i = 0; i < foffs->cnt; i++) + memset(obj + foffs->field_off[i], 0, foffs->field_sz[i]); +} + +static inline void check_and_init_map_value(struct bpf_map *map, void *dst) +{ + bpf_obj_init(map->field_offs, dst); } /* memcpy that is used with 8-byte aligned pointers, power-of-8 size and diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 1e246ec2ff37..a009df64ffab 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -428,6 +428,8 @@ struct bpf_insn_aux_data { */ struct bpf_loop_inline_state loop_inline_state; }; + u64 obj_new_size; /* remember the size of type passed to bpf_obj_new to rewrite R1 */ + struct btf_struct_meta *kptr_struct_meta; u64 map_key_state; /* constant (32 bit) key tracking for maps */ int ctx_field_size; /* the ctx field size for load insn, maybe 0 */ u32 seen; /* this insn was processed by the verifier at env->pass_cnt */ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 9c16338bcbe8..2e57fc839a5c 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -60,6 +61,9 @@ #define CTX regs[BPF_REG_CTX] #define IMM insn->imm +struct bpf_mem_alloc bpf_global_ma; +bool bpf_global_ma_set; + /* No hurry in this branch * * Exported for the bpf jit load helper. @@ -2746,6 +2750,18 @@ int __weak bpf_arch_text_invalidate(void *dst, size_t len) return -ENOTSUPP; } +#ifdef CONFIG_BPF_SYSCALL +static int __init bpf_global_ma_init(void) +{ + int ret; + + ret = bpf_mem_alloc_init(&bpf_global_ma, 0, false); + bpf_global_ma_set = !ret; + return ret; +} +late_initcall(bpf_global_ma_init); +#endif + DEFINE_STATIC_KEY_FALSE(bpf_stats_enabled_key); EXPORT_SYMBOL(bpf_stats_enabled_key); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 5bc0b9f0f306..c4f1c22cc44c 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -19,6 +19,7 @@ #include #include #include +#include #include "../../lib/kstrtox.h" @@ -1735,25 +1736,57 @@ void bpf_list_head_free(const struct btf_field *field, void *list_head, obj -= field->list_head.node_offset; head = head->next; - /* TODO: Rework later */ - kfree(obj); + /* The contained type can also have resources, including a + * bpf_list_head which needs to be freed. + */ + bpf_obj_free_fields(field->list_head.value_rec, obj); + /* bpf_mem_free requires migrate_disable(), since we can be + * called from map free path as well apart from BPF program (as + * part of map ops doing bpf_obj_free_fields). + */ + migrate_disable(); + bpf_mem_free(&bpf_global_ma, obj); + migrate_enable(); } } -BTF_SET8_START(tracing_btf_ids) +__diag_push(); +__diag_ignore_all("-Wmissing-prototypes", + "Global functions as their definitions will be in vmlinux BTF"); + +void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign) +{ + struct btf_struct_meta *meta = meta__ign; + u64 size = local_type_id__k; + void *p; + + if (unlikely(!bpf_global_ma_set)) + return NULL; + p = bpf_mem_alloc(&bpf_global_ma, size); + if (!p) + return NULL; + if (meta) + bpf_obj_init(meta->field_offs, p); + return p; +} + +__diag_pop(); + +BTF_SET8_START(generic_btf_ids) #ifdef CONFIG_KEXEC_CORE BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) #endif -BTF_SET8_END(tracing_btf_ids) +BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) +BTF_SET8_END(generic_btf_ids) -static const struct btf_kfunc_id_set tracing_kfunc_set = { +static const struct btf_kfunc_id_set generic_kfunc_set = { .owner = THIS_MODULE, - .set = &tracing_btf_ids, + .set = &generic_btf_ids, }; static int __init kfunc_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &tracing_kfunc_set); + return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set); } late_initcall(kfunc_init); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5bfc151711b9..db658a31d64f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7955,6 +7955,11 @@ static bool is_kfunc_arg_sfx_constant(const struct btf *btf, const struct btf_pa return __kfunc_param_match_suffix(btf, arg, "__k"); } +static bool is_kfunc_arg_sfx_ignore(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__ign"); +} + static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, const struct btf_param *arg, const struct bpf_reg_state *reg, @@ -8223,6 +8228,10 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ int kf_arg_type; t = btf_type_skip_modifiers(btf, args[i].type, NULL); + + if (is_kfunc_arg_sfx_ignore(btf, &args[i])) + continue; + if (btf_type_is_scalar(t)) { if (reg->type != SCALAR_VALUE) { verbose(env, "R%d is not a scalar\n", regno); @@ -8402,6 +8411,17 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return 0; } +enum special_kfunc_type { + KF_bpf_obj_new_impl, +}; + +BTF_SET_START(special_kfunc_set) +BTF_ID(func, bpf_obj_new_impl) +BTF_SET_END(special_kfunc_set) + +BTF_ID_LIST(special_kfunc_list) +BTF_ID(func, bpf_obj_new_impl) + static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { @@ -8476,17 +8496,64 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); if (is_kfunc_acquire(&meta) && !btf_type_is_struct_ptr(meta.btf, t)) { - verbose(env, "acquire kernel function does not return PTR_TO_BTF_ID\n"); - return -EINVAL; + /* Only exception is bpf_obj_new_impl */ + if (meta.btf != btf_vmlinux || meta.func_id != special_kfunc_list[KF_bpf_obj_new_impl]) { + verbose(env, "acquire kernel function does not return PTR_TO_BTF_ID\n"); + return -EINVAL; + } } if (btf_type_is_scalar(t)) { mark_reg_unknown(env, regs, BPF_REG_0); mark_btf_func_reg_size(env, BPF_REG_0, t->size); } else if (btf_type_is_ptr(t)) { - ptr_type = btf_type_skip_modifiers(desc_btf, t->type, - &ptr_type_id); - if (!btf_type_is_struct(ptr_type)) { + ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); + + if (meta.btf == btf_vmlinux && btf_id_set_contains(&special_kfunc_set, meta.func_id)) { + if (!btf_type_is_void(ptr_type)) { + verbose(env, "kernel function %s must have void * return type\n", + meta.func_name); + return -EINVAL; + } + if (meta.func_id == special_kfunc_list[KF_bpf_obj_new_impl]) { + const struct btf_type *ret_t; + struct btf *ret_btf; + u32 ret_btf_id; + + if (((u64)(u32)meta.arg_constant.value) != meta.arg_constant.value) { + verbose(env, "local type ID argument must be in range [0, U32_MAX]\n"); + return -EINVAL; + } + + ret_btf = env->prog->aux->btf; + ret_btf_id = meta.arg_constant.value; + + /* This may be NULL due to user not supplying a BTF */ + if (!ret_btf) { + verbose(env, "bpf_obj_new requires prog BTF\n"); + return -EINVAL; + } + + ret_t = btf_type_by_id(ret_btf, ret_btf_id); + if (!ret_t || !__btf_type_is_struct(ret_t)) { + verbose(env, "bpf_obj_new type ID argument must be of a struct\n"); + return -EINVAL; + } + + mark_reg_known_zero(env, regs, BPF_REG_0); + regs[BPF_REG_0].type = PTR_TO_BTF_ID | MEM_ALLOC; + regs[BPF_REG_0].btf = ret_btf; + regs[BPF_REG_0].btf_id = ret_btf_id; + + env->insn_aux_data[insn_idx].obj_new_size = ret_t->size; + env->insn_aux_data[insn_idx].kptr_struct_meta = + btf_find_struct_meta(ret_btf, ret_btf_id); + } else { + verbose(env, "kernel function %s unhandled dynamic return type\n", + meta.func_name); + return -EFAULT; + } + } else if (!__btf_type_is_struct(ptr_type)) { if (!meta.r0_size) { ptr_type_name = btf_name_by_offset(desc_btf, ptr_type->name_off); @@ -8514,6 +8581,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; } + if (is_kfunc_ret_null(&meta)) { regs[BPF_REG_0].type |= PTR_MAYBE_NULL; /* For mark_ptr_or_null_reg, see 93c230e3f5bd6 */ @@ -14662,8 +14730,8 @@ static int fixup_call_args(struct bpf_verifier_env *env) return err; } -static int fixup_kfunc_call(struct bpf_verifier_env *env, - struct bpf_insn *insn) +static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, + struct bpf_insn *insn_buf, int insn_idx, int *cnt) { const struct bpf_kfunc_desc *desc; @@ -14682,8 +14750,21 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, return -EFAULT; } + *cnt = 0; insn->imm = desc->imm; + if (insn->off) + return 0; + if (desc->func_id == special_kfunc_list[KF_bpf_obj_new_impl]) { + struct btf_struct_meta *kptr_struct_meta = env->insn_aux_data[insn_idx].kptr_struct_meta; + struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) }; + u64 obj_new_size = env->insn_aux_data[insn_idx].obj_new_size; + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_1, obj_new_size); + insn_buf[1] = addr[0]; + insn_buf[2] = addr[1]; + insn_buf[3] = *insn; + *cnt = 4; + } return 0; } @@ -14825,9 +14906,19 @@ static int do_misc_fixups(struct bpf_verifier_env *env) if (insn->src_reg == BPF_PSEUDO_CALL) continue; if (insn->src_reg == BPF_PSEUDO_KFUNC_CALL) { - ret = fixup_kfunc_call(env, insn); + ret = fixup_kfunc_call(env, insn, insn_buf, i + delta, &cnt); if (ret) return ret; + if (cnt == 0) + continue; + + new_prog = bpf_patch_insn_data(env, i + delta, insn_buf, cnt); + if (!new_prog) + return -ENOMEM; + + delta += cnt - 1; + env->prog = prog = new_prog; + insn = new_prog->insnsi + i + delta; continue; } diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h new file mode 100644 index 000000000000..3588fe89890c --- /dev/null +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -0,0 +1,25 @@ +#ifndef __BPF_EXPERIMENTAL__ +#define __BPF_EXPERIMENTAL__ + +#include +#include +#include +#include + +/* Description + * Allocates a local kptr of type represented by 'local_type_id' in program + * BTF. User may use the bpf_core_type_id_local macro to pass the type ID + * of a struct in program BTF. + * + * The 'local_type_id' parameter must be a known constant. + * The 'meta' parameter is a hidden argument that is ignored. + * Returns + * A local kptr corresponding to passed in 'local_type_id', or NULL on + * failure. + */ +extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym; + +/* Convenience macro to wrap over bpf_obj_new_impl */ +#define bpf_obj_new(type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) + +#endif From patchwork Mon Nov 7 23:09:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035461 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2FDDC4321E for ; Mon, 7 Nov 2022 23:11:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232815AbiKGXLP (ORCPT ); Mon, 7 Nov 2022 18:11:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232798AbiKGXLD (ORCPT ); Mon, 7 Nov 2022 18:11:03 -0500 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0414F2A24C for ; Mon, 7 Nov 2022 15:11:02 -0800 (PST) Received: by mail-pg1-x543.google.com with SMTP id h193so11835450pgc.10 for ; Mon, 07 Nov 2022 15:11:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ien0ATfWvVeirqicjop85pwTRZjU8Lsiw9fs6Z86qKM=; b=i79zvBDmE8aZ9mFWbNFbmogII/xjx0VoXm5rcYEtFZ7XIB2B1477+zkJFYeLNXyauw oa0rzTHUNbfpP8f3nSAhcjD+5ZfBDTZDmDaf8oRJczYx55YgMzLqGnOvdAmRAhm6D3Xr zNnYWHLm4XCMKX1jAa4NtgTz2FPx8R8Fy21YRXcPBlvveqlb+LtJc1kTWRl8T79C1t3A iL14jTHd8PkPHC5itaSrxUWRT8KGomV7fy3OdVrREICKPAL52Noa7KvHkge83RjCUaLd gFbVxr/nUhEMiT3ae2lKdCJLz0Il4h5iiqLdVgZZshLsZDLpo/4RKTx0Nui7CI4Y2/6l zJwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ien0ATfWvVeirqicjop85pwTRZjU8Lsiw9fs6Z86qKM=; b=MKdbgpfLbUCyr8/HOdilOOrwNSDFpVKPJ2FuJcby5Bv/ZjBKKBmTmLhRHZAdkhoWyC NST0zGRUYWeZ0wimWG+g8G7P7Xcb+DExu874G2XSe58/lKc3mreiURdyZRSWyzT9gYQl pFRHu16B5AgK807IU66hIxxxrCH/+932HdWQgjICjvEIAe1s9zPqAQfBmCC190H/9zYh 7ew6P4WBMUkXIWr7GjpmhSf2AATQFlbjSPMrUG0Ax0Jmt9aW5Xb5J4czTi1TAhtUv53d YlzrftDCFfN84lF6aHnBZb6xW/oHJ/G8UF2WEziJ3VqgLsW2N+E6u6palZxjFEH1tJA+ C7Ow== X-Gm-Message-State: ACrzQf3XyV/YfXANWl1sBPPcqZY4CPawgMHZOdng16osTOIG3alJpWTI dauicmgvZctTtIzSAOk8wnEh885uIqmTCQ== X-Google-Smtp-Source: AMsMyM4pIKCJ1e6MmO9m9UA+r2J81qNj9TRgdXydBY82FpKHW7Rb6wUqVvD+8BZtelcr6DC32hyR5w== X-Received: by 2002:a62:15cf:0:b0:56e:664f:707c with SMTP id 198-20020a6215cf000000b0056e664f707cmr23845177pfv.39.1667862661210; Mon, 07 Nov 2022 15:11:01 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id e13-20020a17090301cd00b001782398648dsm5550989plh.8.2022.11.07.15.11.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:00 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 17/25] bpf: Introduce bpf_obj_drop Date: Tue, 8 Nov 2022 04:39:42 +0530 Message-Id: <20221107230950.7117-18-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7739; i=memxor@gmail.com; h=from:subject; bh=lXVavpEa6FtVbiHBSnYpRTtxRtwUPmP/uwgjuUq7FsY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+32XBuhN3yOg/T8OgNtXhMIP+nKGx8i+cOSL1n V3CFhJCJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8RyvuAD/ 9UubC1M0El7BWVDSVMjNTtc5ETz8E/cDG+FCQW10L9RaAGfir5x+XqYgsudmmHfYg6B0OF//ruTBSC aim/Vxi1lg+iIZvKN0GI12kgep2SMbzOzWV6E2alXxYxORJ3pTnUrY2EoQ5XOFUNzISZKpKBMS6upN 7xwf6QcPr44nW9+O2ebMt7PKwUwFnWiS3eR9psKEyIQ6yQFBt38oOxdKFMLlKmJ4rN4y/Dvn7jq0uU AivoDLOcsFDM8s3T7WcFuPwLAEcUFk4wQ/QWAmM6urDqN7jJVFC6od3C9LLkJsAd2rTNN+KYn0Jmb0 ZeD0LdpbKN8qAXVEOAl0bkrFl13IWKM3Jce4TRimVIlWFlXqP1ip7KFqoQ0SIUsxQuhHGA6CyiCQ5t d19XUFJHUj6GO4pTqbiCS20dOmCmEWx4eSbrwpc+HW0tGhu9J9yuzOKtBX2pulalVMWgQp5NjZYZYx W6BJntX1yOIUiBVle4Et+jO0RHITNvLjaFUnHmQnEANpvBhetjJwPurM/z5ISzxa2ebiOn97FcBDcE NE+TS2L50aJov8a4+oMxdcpsFqQ7rGEEd5G5wAzcyh4+zMaD9KyLCwV9+6j6XEXfXbCbbEQnjflKBH KO81IiKzLEguXKBv6FNRmuNxjAXFjfUHPXNd7jwU8cWWkL1piPS/QZ0I/hOQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Introduce bpf_obj_drop, which is the kfunc used to free local kptrs allocated using bpf_obj_new. Similar to bpf_obj_new, it implicitly destructs the fields part of the local kptr automatically without user intervention. Just like the previous patch, btf_struct_meta that is needed to free up the special fields is passed as a hidden argument to the kfunc. For the user, a convenience macro hides over the kernel side kfunc which is named bpf_obj_drop_impl. Continuing the previous example: void prog(void) { struct foo *f; f = bpf_obj_new(typeof(*f)); if (!f) return; bpf_obj_drop(f); } Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/helpers.c | 11 ++++ kernel/bpf/verifier.c | 66 +++++++++++++++---- .../testing/selftests/bpf/bpf_experimental.h | 13 ++++ 3 files changed, 79 insertions(+), 11 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index c4f1c22cc44c..02045f6a6ad5 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1770,6 +1770,16 @@ void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign) return p; } +void bpf_obj_drop_impl(void *p__lkptr, void *meta__ign) +{ + struct btf_struct_meta *meta = meta__ign; + void *p = p__lkptr; + + if (meta) + bpf_obj_free_fields(meta->record, p); + bpf_mem_free(&bpf_global_ma, p); +} + __diag_pop(); BTF_SET8_START(generic_btf_ids) @@ -1777,6 +1787,7 @@ BTF_SET8_START(generic_btf_ids) BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) #endif BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE) BTF_SET8_END(generic_btf_ids) static const struct btf_kfunc_id_set generic_kfunc_set = { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index db658a31d64f..7c5d9e933d97 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7882,6 +7882,10 @@ struct bpf_kfunc_call_arg_meta { u64 value; bool found; } arg_constant; + struct { + struct btf *btf; + u32 btf_id; + } arg_obj_drop; }; static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) @@ -7960,6 +7964,11 @@ static bool is_kfunc_arg_sfx_ignore(const struct btf *btf, const struct btf_para return __kfunc_param_match_suffix(btf, arg, "__ign"); } +static bool is_kfunc_arg_local_kptr(const struct btf *btf, const struct btf_param *arg) +{ + return __kfunc_param_match_suffix(btf, arg, "__lkptr"); +} + static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, const struct btf_param *arg, const struct bpf_reg_state *reg, @@ -8060,6 +8069,7 @@ static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_CTX, + KF_ARG_PTR_TO_LOCAL_BTF_ID, /* Local kptr */ KF_ARG_PTR_TO_KPTR_STRONG, /* PTR_TO_KPTR but type specific */ KF_ARG_PTR_TO_DYNPTR, KF_ARG_PTR_TO_BTF_ID, /* Also covers reg2btf_ids conversions */ @@ -8067,6 +8077,20 @@ enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_MEM_SIZE, /* Size derived from next argument, skip it */ }; +enum special_kfunc_type { + KF_bpf_obj_new_impl, + KF_bpf_obj_drop_impl, +}; + +BTF_SET_START(special_kfunc_set) +BTF_ID(func, bpf_obj_new_impl) +BTF_ID(func, bpf_obj_drop_impl) +BTF_SET_END(special_kfunc_set) + +BTF_ID_LIST(special_kfunc_list) +BTF_ID(func, bpf_obj_new_impl) +BTF_ID(func, bpf_obj_drop_impl) + static enum kfunc_ptr_arg_type get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, @@ -8087,6 +8111,9 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (btf_get_prog_ctx_type(&env->log, meta->btf, t, resolve_prog_type(env->prog), argno)) return KF_ARG_PTR_TO_CTX; + if (is_kfunc_arg_local_kptr(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_LOCAL_BTF_ID; + if (is_kfunc_arg_kptr_get(meta, argno)) { if (!btf_type_is_ptr(ref_t)) { verbose(env, "arg#0 BTF type must be a double pointer for kptr_get kfunc\n"); @@ -8305,6 +8332,7 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return kf_arg_type; switch (kf_arg_type) { + case KF_ARG_PTR_TO_LOCAL_BTF_ID: case KF_ARG_PTR_TO_BTF_ID: if (!is_kfunc_trusted_args(meta)) break; @@ -8341,6 +8369,21 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return -EINVAL; } break; + case KF_ARG_PTR_TO_LOCAL_BTF_ID: + if (reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { + verbose(env, "arg#%d expected pointer to local kptr\n", i); + return -EINVAL; + } + if (!reg->ref_obj_id) { + verbose(env, "local kptr must be referenced\n"); + return -EINVAL; + } + if (meta->btf == btf_vmlinux && + meta->func_id == special_kfunc_list[KF_bpf_obj_drop_impl]) { + meta->arg_obj_drop.btf = reg->btf; + meta->arg_obj_drop.btf_id = reg->btf_id; + } + break; case KF_ARG_PTR_TO_KPTR_STRONG: if (reg->type != PTR_TO_MAP_VALUE) { verbose(env, "arg#0 expected pointer to map value\n"); @@ -8411,17 +8454,6 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return 0; } -enum special_kfunc_type { - KF_bpf_obj_new_impl, -}; - -BTF_SET_START(special_kfunc_set) -BTF_ID(func, bpf_obj_new_impl) -BTF_SET_END(special_kfunc_set) - -BTF_ID_LIST(special_kfunc_list) -BTF_ID(func, bpf_obj_new_impl) - static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, int *insn_idx_p) { @@ -8548,6 +8580,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, env->insn_aux_data[insn_idx].obj_new_size = ret_t->size; env->insn_aux_data[insn_idx].kptr_struct_meta = btf_find_struct_meta(ret_btf, ret_btf_id); + } else if (meta.func_id == special_kfunc_list[KF_bpf_obj_drop_impl]) { + env->insn_aux_data[insn_idx].kptr_struct_meta = + btf_find_struct_meta(meta.arg_obj_drop.btf, + meta.arg_obj_drop.btf_id); } else { verbose(env, "kernel function %s unhandled dynamic return type\n", meta.func_name); @@ -14764,6 +14800,14 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, insn_buf[2] = addr[1]; insn_buf[3] = *insn; *cnt = 4; + } else if (desc->func_id == special_kfunc_list[KF_bpf_obj_drop_impl]) { + struct btf_struct_meta *kptr_struct_meta = env->insn_aux_data[insn_idx].kptr_struct_meta; + struct bpf_insn addr[2] = { BPF_LD_IMM64(BPF_REG_2, (long)kptr_struct_meta) }; + + insn_buf[0] = addr[0]; + insn_buf[1] = addr[1]; + insn_buf[2] = *insn; + *cnt = 3; } return 0; } diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 3588fe89890c..d145c93ea663 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -22,4 +22,17 @@ extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym; /* Convenience macro to wrap over bpf_obj_new_impl */ #define bpf_obj_new(type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) +/* Description + * Free a local kptr. All fields of local kptr that require destruction + * will be destructed before the storage is freed. + * + * The 'meta' parameter is a hidden argument that is ignored. + * Returns + * Void. + */ +extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; + +/* Convenience macro to wrap over bpf_obj_drop_impl */ +#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL) + #endif From patchwork Mon Nov 7 23:09:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035460 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8ACD3C43219 for ; Mon, 7 Nov 2022 23:11:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232798AbiKGXLR (ORCPT ); Mon, 7 Nov 2022 18:11:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232847AbiKGXLF (ORCPT ); Mon, 7 Nov 2022 18:11:05 -0500 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB4C329834 for ; Mon, 7 Nov 2022 15:11:04 -0800 (PST) Received: by mail-pl1-x644.google.com with SMTP id c2so12555466plz.11 for ; Mon, 07 Nov 2022 15:11:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=34JwRE/hItzOhlEzOozXW/4nElB65477jrFZyMohsaU=; b=Xxrm63FYus2cZLN9p/ZD7ikxYlwGg6u27LrjsfKXivlwhTUHuUcWd5QmR6ruAxLKtO qXUGQ7Yg6GEjP6WwUqPL3ekqk1wr1H9diyJ/gi+t/D1Ij5WyklVhLLvlPqLNGNuuPqTb 1SBcRemsigzpcH8GZ4Kmp0GCuvb3yT9TFVc035Kb+Ew2SoytQDir7S7ylTRHmP0vuRvj vrgPAeVj203FesMu+lFFY9eT2hYOXGTWyeA+il1rEq3Q3dI24w/UZyDdb/tuTm/Pjipf 4XTqxTyuYB77IqD9T56qFDgmGpfttMhi1j3PTFAXlwnuQ3A/MK/Z4YCsKsJwcBQoge0v oA3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=34JwRE/hItzOhlEzOozXW/4nElB65477jrFZyMohsaU=; b=Kv8rNYhL19IflC/xIDfOxfYs/8LJNnm2jBTJVcrUTiW30V75wRWznQrcD8CszPApTs HPTh+5hisWFVK7wfSsxzQXd3uEF6LW7Bytffhwg/gGUQKSREI2Bw2daNZqQHy+9+I5IR bLC8VJMlkoq7E6x2HLYfGKviSDohkJPSOj0uej4uQoCugtfaI57z7v/7gxxPntYHsiu2 /ltYtdW+9V1ST17lL5GiSjDB/50Wyqt6zFhk/31+An/xXkVe6eC6UKEoNguf7lT2GZLM ZcmaGC44K4cu6gz5YBTyetqqgj+4Dd43GZAsiKAD+SnBC/zwa3GFi2VA8aA0ONiIjtIV GTxw== X-Gm-Message-State: ACrzQf2vsGpNO/DLra+6gp5LyX5eO5ZANhu2jyJGB36ci/c+STeSwC2a rH+LnNQGTZvzhGSLr3OcBYUlPtsCRQh6mg== X-Google-Smtp-Source: AMsMyM48wkVKEjFe+laY2xwOAMymTRTsED0lKsFbJYgvmFrFeKTGHEyOQH2fVe/c6pfVVdXMkrLiKw== X-Received: by 2002:a17:90b:394:b0:212:19d7:9072 with SMTP id ga20-20020a17090b039400b0021219d79072mr70377425pjb.69.1667862664083; Mon, 07 Nov 2022 15:11:04 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id f14-20020a170902ce8e00b00180033438a0sm5493753plg.106.2022.11.07.15.11.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:03 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 18/25] bpf: Permit NULL checking pointer with non-zero fixed offset Date: Tue, 8 Nov 2022 04:39:43 +0530 Message-Id: <20221107230950.7117-19-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2205; i=memxor@gmail.com; h=from:subject; bh=KwYI5sn0hhQ9lV7wTPXq3b80fTzA1hv0z+Cf7VRfx4k=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+36QVCRIN31DsW7OAwJfdhPmPIqX5SO97Vojcj ZTXzVpOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8Ryjb9D/ 9TKdoU97GUhHbCVaVsVJaaaXnHrHXz0MqeuvA0XpXIXasPLAhlqsPHBNX7sFx3d21FHxE2IiELL9Kp xbgqRp99OXI29D2Z5USCj4yZGcxKkaEQ9HVACgmue/aH1N7HDRUdvhUXRMxgQ18GnBneWbWlm9XC1y iBGtB4Mh0d+80vdoO5R4pkcWtJU2k1wg4gNmF0niXAVtMsajvHWRsRMgqjIYKcn8nQSqROxfCGgkhz P0WCerCMyEuA96bn3z1QJxhfVA1KmnMeg+mYFk8O2pi/hJxM15X8xgflJS1DGCPLGrqi2blQfHARzY AXCTAtpbJhswHGW8JVuLt3frvtjWJ1HlAjxfhq3OSTY/ntu4UxrWZ734QObbHX2ntmvy2cjU6QmWJB HW+Z9dQq17PqHOSk4CZPim7G4A10NnWZWTFBeLySH0Lg8ycg2OCNLiD+I83volQZtu31HJxGRDmDxn Tqs9n7Hmp7FnpZnPJumIOJtykxZzFtch7X4AnoetwQkQSy3EXaRIW0y5Oqd48rx2/dXJ4w+s/l+eIe 9uLnw1p4EOAF1H4hlj8X4M5IQr3EmY3eMoLcZ/bInxQ4vqfy6klq+89hdSrV2JP69MyLoZoowD8Zl/ f05+O4lob5ftYU8TKG6yfmwAc1i3nCtwdDZgiv5PMOi+8Tw2ORhGHxKyTvdQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Pointer increment on seeing PTR_MAYBE_NULL is already protected against, hence make an exception for local kptrs while still keeping the warning for other unintended cases that might creep in. bpf_list_pop_{front,_back} helpers planned to be introduced in next commit will return a local kptr with incremented offset pointing to bpf_list_node field. The user is supposed to then obtain the pointer to the entry using container_of after NULL checking it. The current restrictions trigger a warning when doing the NULL checking. Revisiting the reason, it is meant as an assertion which seems to actually work and catch the bad case. Hence, under no other circumstances can reg->off be non-zero for a register that has the PTR_MAYBE_NULL type flag set. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/verifier.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7c5d9e933d97..abcb23a4c6fc 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10809,15 +10809,20 @@ static void mark_ptr_or_null_reg(struct bpf_func_state *state, { if (type_may_be_null(reg->type) && reg->id == id && !WARN_ON_ONCE(!reg->id)) { - if (WARN_ON_ONCE(reg->smin_value || reg->smax_value || - !tnum_equals_const(reg->var_off, 0) || - reg->off)) { + if (reg->smin_value || reg->smax_value || !tnum_equals_const(reg->var_off, 0) || reg->off) { /* Old offset (both fixed and variable parts) should * have been known-zero, because we don't allow pointer * arithmetic on pointers that might be NULL. If we * see this happening, don't convert the register. + * + * But in some cases, some helpers that return local + * kptrs advance offset for the returned pointer. + * In those cases, it is fine to expect to see reg->off. */ - return; + if (WARN_ON_ONCE(reg->type != (PTR_TO_BTF_ID | MEM_ALLOC | PTR_MAYBE_NULL))) + return; + if (WARN_ON_ONCE(reg->smin_value || reg->smax_value || !tnum_equals_const(reg->var_off, 0))) + return; } if (is_null) { reg->type = SCALAR_VALUE; From patchwork Mon Nov 7 23:09:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035463 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DF6BC433FE for ; Mon, 7 Nov 2022 23:11:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232540AbiKGXLT (ORCPT ); Mon, 7 Nov 2022 18:11:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232927AbiKGXLJ (ORCPT ); Mon, 7 Nov 2022 18:11:09 -0500 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F35F2018E for ; Mon, 7 Nov 2022 15:11:08 -0800 (PST) Received: by mail-pf1-x444.google.com with SMTP id y13so12198002pfp.7 for ; Mon, 07 Nov 2022 15:11:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8ITQNdhUhnSfBbvMIXpsOt/p8d9papIohbLxaNonPtE=; b=DO0Xu7kOd4RHfOgOV8V5pyRSIjDydtKmy0b5c0bMYmblYN7EOWNVH4F5LcsIzfdIv3 Tc9mVh97CE4uVFzGybCi080qGdtzPso/C41tHo39w8tFeRHwOddv5KSH88ilaTbgLEIb Fm5+4eG6yXb4RwHm1sEN7NiGqKOkkGIyfwDx7df3+g3YPX1txr27TXz+hHUFN6y3dZ01 poryoWnK3y2E0/gLTr7MND7YAskIeYHvF5OG2hlM6TfQgFOO7RoPmKe3BCklk+9gEowm 0aGn8MGWAdDtlU/G60Q8cDPIbIKecnLYHSCKPgiu9khXk9d/inHKyXPEaLYfmk18IiOD E9NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8ITQNdhUhnSfBbvMIXpsOt/p8d9papIohbLxaNonPtE=; b=RdjhIN1yicrDCRjw7ZTeCQiQk5RiA6Wn1UJSg71fKlGJfbZNN5CWjcN5wl+U5QQbiZ sfblveX2snM8YBtdJLoyBLcspyu+8a47F3aMtDZjhoGvM4hu4ymKvg+mxQ3bzxUbEGA2 zfWzo/7CuANSfuun8usG54GMr5moQsudQ3NZnkLvS2VxmuCXiIgGQm5HMWs1CmHAcR+M KlUg6AB+D0YSKPm5M2oxKfqN3BDcdVHmncMtLB8l/EWEGS7NiFLDnw9/YQVQJH6D9EVH v/C2p03c3XH3SGEhJh+pkoOQQzyNcGXEUKbiDKq01+qcouovqrXQY33DuHcEdSWn/li0 rNdw== X-Gm-Message-State: ACrzQf2Er3KsOB7PoqidFLKscRDZ+tcot+TViLJVvEg5dj0zVk/Djf3K umD/vZ3kxHFZK1QjgJhvCTFVfKUuiB6bNg== X-Google-Smtp-Source: AMsMyM4bIxwWSskPZlC3Sfc+QACbI1NUVpMzScSo4qkYj0JS3vENgwL9h95hQi+8exjM0Ub7N/0RlA== X-Received: by 2002:a05:6a00:190a:b0:56c:123e:3e61 with SMTP id y10-20020a056a00190a00b0056c123e3e61mr52718785pfi.47.1667862667477; Mon, 07 Nov 2022 15:11:07 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id d67-20020a621d46000000b00562a237179esm5023408pfd.131.2022.11.07.15.11.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:07 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 19/25] bpf: Introduce single ownership BPF linked list API Date: Tue, 8 Nov 2022 04:39:44 +0530 Message-Id: <20221107230950.7117-20-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=20800; i=memxor@gmail.com; h=from:subject; bh=0MYi1Wqh3El18Xn8pz5nK7tHNjmWQ8exAupN4LA1hLI=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+3v++S9UlNwNsrlQ0tnE75adm8nfPSpCGU7hWh 3RPp8PWJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8RykQlEA CCTn8HoQvLceookEKTGpe6BuhEFxE6G71RK1aOhNsrJyhXSa3HToFWrtNwfT25ixq9EF1ZN+w1LRpy SfZwdxLoBKM9FYTJw9vgDOJiIfh0RjqVO9BZbTRj91EEFXgiBZdL0PN9msADpxuaSCSxu4lg5TS74T N+Ysq3v7DfPhqaG53591PCw0TnFQHY38AesoiRcXJzEUYtMEPoo6V6FNzwFuC6iiPLKMimyY+J5R3b QHZ3/FOBE/sstxqPhAhYSMW3jmXjjiXD82D9cwFP7vksU1TBawljVtZGceSRVNpJlIpR6HTCsKCjmm IL8cQ4OpBOodziVdJQoF/aoJXrZbU97EF/5wY3pKXOhcIEfclbf7LvYvL8Poz1QQa9Uj0MEQAZg5NX ABQb6G13foHYDGePj4e7D7+B03Ao57MkEM3kyGC3XiONV7xrplAlcmefj8aTHUedTGnrF7sMHX64NT Rbyrf5KOxi4L6mBP9FzWSzcuCy7YfIdDOjv9HGjRod9DDyfDmx3PlJERYPIBwRUboOb+DfKC65ZPaT V2aEl9vY4oKuZ/AR9EKgk9bQmKmDsQ0d70/qTgXnU6DZIPbBDJo/Cd0HMkTp8M8nCQZZU7EZtAS0N8 d5xIKO0fkiQwLQaWlc0ZtXJwrzqoxcWLO5y9YbWUSo3Qr3j8AaPI0milYlqw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add a linked list API for use in BPF programs, where it expects protection from the bpf_spin_lock in the same allocation as the bpf_list_head. For now, only one bpf_spin_lock can be present hence that is assumed to be the one protecting the bpf_list_head. The following functions are added to kick things off: // Add node to beginning of list void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node); // Add node to end of list void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node); // Remove node at beginning of list and return it struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head); // Remove node at end of list and return it struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head); The lock protecting the bpf_list_head needs to be taken for all operations. The verifier ensures that the lock that needs to be taken is always held, and only the correct lock is taken for these operations. These checks are made statically by relying on the reg->id preserved for registers pointing into regions having both bpf_spin_lock and the objects protected by it. The comment over check_reg_allocation_locked in this change describes the logic in detail. Note that bpf_list_push_front and bpf_list_push_back are meant to consume the object containing the node in the 1st argument, however that specific mechanism is intended to not release the ref_obj_id directly until the bpf_spin_unlock is called. In this commit, nothing is done, but the next commit will be introducing logic to handle this case, so it has been left as is for now. bpf_list_pop_front and bpf_list_pop_back delete the first or last item of the list respectively, and return pointer to the element at the list_node offset. The user can then use container_of style macro to get the actual entry type. The verifier however statically knows the actual type, so the safety properties are still preserved. With these additions, programs can now manage their own linked lists and store their objects in them. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/helpers.c | 55 +++- kernel/bpf/verifier.c | 292 +++++++++++++++++- .../testing/selftests/bpf/bpf_experimental.h | 28 ++ 3 files changed, 361 insertions(+), 14 deletions(-) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 02045f6a6ad5..461e0c3174c6 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1780,6 +1780,50 @@ void bpf_obj_drop_impl(void *p__lkptr, void *meta__ign) bpf_mem_free(&bpf_global_ma, p); } +static void __bpf_list_add(struct bpf_list_node *node, struct bpf_list_head *head, bool tail) +{ + struct list_head *n = (void *)node, *h = (void *)head; + + if (unlikely(!h->next)) + INIT_LIST_HEAD(h); + if (unlikely(!n->next)) + INIT_LIST_HEAD(n); + tail ? list_add_tail(n, h) : list_add(n, h); +} + +void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node) +{ + return __bpf_list_add(node, head, false); +} + +void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node) +{ + return __bpf_list_add(node, head, true); +} + +static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head, bool tail) +{ + struct list_head *n, *h = (void *)head; + + if (unlikely(!h->next)) + INIT_LIST_HEAD(h); + if (list_empty(h)) + return NULL; + n = tail ? h->prev : h->next; + list_del_init(n); + return (struct bpf_list_node *)n; +} + +struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head) +{ + return __bpf_list_del(head, false); +} + +struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) +{ + return __bpf_list_del(head, true); +} + __diag_pop(); BTF_SET8_START(generic_btf_ids) @@ -1788,6 +1832,10 @@ BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE) #endif BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_list_push_front) +BTF_ID_FLAGS(func, bpf_list_push_back) +BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL) BTF_SET8_END(generic_btf_ids) static const struct btf_kfunc_id_set generic_kfunc_set = { @@ -1797,7 +1845,12 @@ static const struct btf_kfunc_id_set generic_kfunc_set = { static int __init kfunc_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set); + int ret; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &generic_kfunc_set); + if (ret) + return ret; + return register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &generic_kfunc_set); } late_initcall(kfunc_init); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index abcb23a4c6fc..5b87ef859046 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7886,6 +7886,9 @@ struct bpf_kfunc_call_arg_meta { struct btf *btf; u32 btf_id; } arg_obj_drop; + struct { + struct btf_field *field; + } arg_list_head; }; static bool is_kfunc_acquire(struct bpf_kfunc_call_arg_meta *meta) @@ -7996,13 +7999,17 @@ static bool is_kfunc_arg_ret_buf_size(const struct btf *btf, enum { KF_ARG_DYNPTR_ID, + KF_ARG_LIST_HEAD_ID, + KF_ARG_LIST_NODE_ID, }; BTF_ID_LIST(kf_arg_btf_ids) BTF_ID(struct, bpf_dynptr_kern) +BTF_ID(struct, bpf_list_head) +BTF_ID(struct, bpf_list_node) -static bool is_kfunc_arg_dynptr(const struct btf *btf, - const struct btf_param *arg) +static bool __is_kfunc_ptr_arg_type(const struct btf *btf, + const struct btf_param *arg, int type) { const struct btf_type *t; u32 res_id; @@ -8015,7 +8022,22 @@ static bool is_kfunc_arg_dynptr(const struct btf *btf, t = btf_type_skip_modifiers(btf, t->type, &res_id); if (!t) return false; - return btf_types_are_same(btf, res_id, btf_vmlinux, kf_arg_btf_ids[KF_ARG_DYNPTR_ID]); + return btf_types_are_same(btf, res_id, btf_vmlinux, kf_arg_btf_ids[type]); +} + +static bool is_kfunc_arg_dynptr(const struct btf *btf, const struct btf_param *arg) +{ + return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_DYNPTR_ID); +} + +static bool is_kfunc_arg_list_head(const struct btf *btf, const struct btf_param *arg) +{ + return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_LIST_HEAD_ID); +} + +static bool is_kfunc_arg_list_node(const struct btf *btf, const struct btf_param *arg) +{ + return __is_kfunc_ptr_arg_type(btf, arg, KF_ARG_LIST_NODE_ID); } /* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ @@ -8072,6 +8094,8 @@ enum kfunc_ptr_arg_type { KF_ARG_PTR_TO_LOCAL_BTF_ID, /* Local kptr */ KF_ARG_PTR_TO_KPTR_STRONG, /* PTR_TO_KPTR but type specific */ KF_ARG_PTR_TO_DYNPTR, + KF_ARG_PTR_TO_LIST_HEAD, + KF_ARG_PTR_TO_LIST_NODE, KF_ARG_PTR_TO_BTF_ID, /* Also covers reg2btf_ids conversions */ KF_ARG_PTR_TO_MEM, KF_ARG_PTR_TO_MEM_SIZE, /* Size derived from next argument, skip it */ @@ -8080,16 +8104,28 @@ enum kfunc_ptr_arg_type { enum special_kfunc_type { KF_bpf_obj_new_impl, KF_bpf_obj_drop_impl, + KF_bpf_list_push_front, + KF_bpf_list_push_back, + KF_bpf_list_pop_front, + KF_bpf_list_pop_back, }; BTF_SET_START(special_kfunc_set) BTF_ID(func, bpf_obj_new_impl) BTF_ID(func, bpf_obj_drop_impl) +BTF_ID(func, bpf_list_push_front) +BTF_ID(func, bpf_list_push_back) +BTF_ID(func, bpf_list_pop_front) +BTF_ID(func, bpf_list_pop_back) BTF_SET_END(special_kfunc_set) BTF_ID_LIST(special_kfunc_list) BTF_ID(func, bpf_obj_new_impl) BTF_ID(func, bpf_obj_drop_impl) +BTF_ID(func, bpf_list_push_front) +BTF_ID(func, bpf_list_push_back) +BTF_ID(func, bpf_list_pop_front) +BTF_ID(func, bpf_list_pop_back) static enum kfunc_ptr_arg_type get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, @@ -8132,6 +8168,12 @@ get_kfunc_ptr_arg_type(struct bpf_verifier_env *env, if (is_kfunc_arg_dynptr(meta->btf, &args[argno])) return KF_ARG_PTR_TO_DYNPTR; + if (is_kfunc_arg_list_head(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_LIST_HEAD; + + if (is_kfunc_arg_list_node(meta->btf, &args[argno])) + return KF_ARG_PTR_TO_LIST_NODE; + if ((base_type(reg->type) == PTR_TO_BTF_ID || reg2btf_ids[base_type(reg->type)])) { if (!btf_type_is_struct(ref_t)) { verbose(env, "kernel function %s args#%d pointer type %s %s is not supported\n", @@ -8227,6 +8269,194 @@ static int process_kf_arg_ptr_to_kptr_strong(struct bpf_verifier_env *env, return 0; } +/* Implementation details: + * + * Each register points to some region of memory, which we define as an + * allocation. Each allocation may embed a bpf_spin_lock which protects any + * special BPF objects (bpf_list_head, bpf_rb_root, etc.) part of the same + * allocation. The lock and the data it protects are colocated in the same + * memory region. + * + * Hence, everytime a register holds a pointer value pointing to such + * allocation, the verifier preserves a unique reg->id for it. + * + * The verifier remembers the lock 'ptr' and the lock 'id' whenever + * bpf_spin_lock is called. + * + * To enable this, lock state in the verifier captures two values: + * active_lock.ptr = Register's type specific pointer + * active_lock.id = A unique ID for each register pointer value + * + * Currently, PTR_TO_MAP_VALUE and PTR_TO_BTF_ID | MEM_ALLOC are the two + * supported register types. + * + * The active_lock.ptr in case of map values is the reg->map_ptr, and in case of + * local kptrs is the reg->btf pointer. + * + * The active_lock.id is non-unique for maps supporting direct_value_addr, as we + * can establish the provenance of the map value statically for each distinct + * lookup into such maps. They always contain a single map value hence unique + * IDs for each pseudo load pessimizes the algorithm and rejects valid programs. + * + * So, in case of global variables, they use array maps with max_entries = 1, + * hence their active_lock.ptr becomes map_ptr and id = 0 (since they all point + * into the same map value as max_entries is 1, as described above). + * + * In case of inner map lookups, the inner map pointer has same map_ptr as the + * outer map pointer (in verifier context), but each lookup into an inner map + * assigns a fresh reg->id to the lookup, so while lookups into distinct inner + * maps from the same outer map share the same map_ptr as active_lock.ptr, they + * will get different reg->id assigned to each lookup, hence different + * active_lock.id. + * + * In case of local kptrs, active_lock.ptr is the reg->btf, and the reg->id is a + * unique ID preserved after the NULL pointer check on the local kptr after its + * allocation using bpf_obj_new. Each allocation receives a new reg->id. + */ +static int check_reg_allocation_locked(struct bpf_verifier_env *env, struct bpf_reg_state *reg) +{ + void *ptr; + u32 id; + + switch ((int)reg->type) { + case PTR_TO_MAP_VALUE: + ptr = reg->map_ptr; + break; + case PTR_TO_BTF_ID | MEM_ALLOC: + ptr = reg->btf; + break; + default: + verbose(env, "verifier internal error: unknown reg type for lock check\n"); + return -EFAULT; + } + id = reg->id; + + if (!env->cur_state->active_lock.ptr) + return -EINVAL; + if (env->cur_state->active_lock.ptr != ptr || + env->cur_state->active_lock.id != id) { + verbose(env, "held lock and object are not in the same allocation\n"); + return -EINVAL; + } + return 0; +} + +static bool is_bpf_list_api_kfunc(u32 btf_id) +{ + return btf_id == special_kfunc_list[KF_bpf_list_push_front] || + btf_id == special_kfunc_list[KF_bpf_list_push_back] || + btf_id == special_kfunc_list[KF_bpf_list_pop_front] || + btf_id == special_kfunc_list[KF_bpf_list_pop_back]; +} + +static int process_kf_arg_ptr_to_list_head(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, u32 regno, + struct bpf_kfunc_call_arg_meta *meta) +{ + struct btf_record *rec = NULL; + struct btf_field *field; + u32 list_head_off; + + if (meta->btf != btf_vmlinux || !is_bpf_list_api_kfunc(meta->func_id)) { + verbose(env, "verifier internal error: bpf_list_head argument for unknown kfunc\n"); + return -EFAULT; + } + + if (reg->type == PTR_TO_MAP_VALUE) { + rec = reg->map_ptr->record; + } else /* PTR_TO_BTF_ID | MEM_ALLOC */ { + struct btf_struct_meta *meta; + + meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (!meta) { + verbose(env, "bpf_list_head not found for local kptr\n"); + return -EINVAL; + } + rec = meta->record; + } + + if (!tnum_is_const(reg->var_off)) { + verbose(env, + "R%d doesn't have constant offset. bpf_list_head has to be at the constant offset\n", + regno); + return -EINVAL; + } + + list_head_off = reg->off + reg->var_off.value; + field = btf_record_find(rec, list_head_off, BPF_LIST_HEAD); + if (!field) { + verbose(env, "bpf_list_head not found at offset=%u\n", list_head_off); + return -EINVAL; + } + + /* All functions require bpf_list_head to be protected using a bpf_spin_lock */ + if (check_reg_allocation_locked(env, reg)) { + verbose(env, "bpf_spin_lock at off=%d must be held for bpf_list_head\n", + rec->spin_lock_off); + return -EINVAL; + } + + if (meta->arg_list_head.field) { + verbose(env, "verifier internal error: repeating bpf_list_head arg\n"); + return -EFAULT; + } + meta->arg_list_head.field = field; + return 0; +} + +static int process_kf_arg_ptr_to_list_node(struct bpf_verifier_env *env, + struct bpf_reg_state *reg, u32 regno, + struct bpf_kfunc_call_arg_meta *meta) +{ + struct btf_struct_meta *struct_meta; + struct btf_field *field; + struct btf_record *rec; + u32 list_node_off; + + if (meta->btf != btf_vmlinux || + (meta->func_id != special_kfunc_list[KF_bpf_list_push_front] && + meta->func_id != special_kfunc_list[KF_bpf_list_push_back])) { + verbose(env, "verifier internal error: bpf_list_head argument for unknown kfunc\n"); + return -EFAULT; + } + + if (!tnum_is_const(reg->var_off)) { + verbose(env, + "R%d doesn't have constant offset. bpf_list_head has to be at the constant offset\n", + regno); + return -EINVAL; + } + + struct_meta = btf_find_struct_meta(reg->btf, reg->btf_id); + if (!struct_meta) { + verbose(env, "bpf_list_node not found for local kptr\n"); + return -EINVAL; + } + rec = struct_meta->record; + + list_node_off = reg->off + reg->var_off.value; + field = btf_record_find(rec, list_node_off, BPF_LIST_NODE); + if (!field || field->offset != list_node_off) { + verbose(env, "bpf_list_node not found at offset=%u\n", list_node_off); + return -EINVAL; + } + + field = meta->arg_list_head.field; + + if (!btf_struct_ids_match(&env->log, reg->btf, reg->btf_id, 0, field->list_head.btf, + field->list_head.value_btf_id, true)) { + verbose(env, "bpf_list_head value type does not match arg#1\n"); + return -EINVAL; + } + + if (list_node_off != field->list_head.node_offset) { + verbose(env, "arg#1 offset must be for bpf_list_node at off=%d\n", + field->list_head.node_offset); + return -EINVAL; + } + return 0; +} + static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta) { const char *func_name = meta->func_name, *ref_tname; @@ -8347,6 +8577,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ break; case KF_ARG_PTR_TO_KPTR_STRONG: case KF_ARG_PTR_TO_DYNPTR: + case KF_ARG_PTR_TO_LIST_HEAD: + case KF_ARG_PTR_TO_LIST_NODE: case KF_ARG_PTR_TO_MEM: case KF_ARG_PTR_TO_MEM_SIZE: /* Trusted by default */ @@ -8411,6 +8643,33 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ return -EINVAL; } break; + case KF_ARG_PTR_TO_LIST_HEAD: + if (reg->type != PTR_TO_MAP_VALUE && + reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { + verbose(env, "arg#%d expected pointer to map value or local kptr\n", i); + return -EINVAL; + } + if (reg->type == (PTR_TO_BTF_ID | MEM_ALLOC) && !reg->ref_obj_id) { + verbose(env, "local kptr must be referenced\n"); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_list_head(env, reg, regno, meta); + if (ret < 0) + return ret; + break; + case KF_ARG_PTR_TO_LIST_NODE: + if (reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { + verbose(env, "arg#%d expected pointer to local kptr\n", i); + return -EINVAL; + } + if (!reg->ref_obj_id) { + verbose(env, "local kptr must be referenced\n"); + return -EINVAL; + } + ret = process_kf_arg_ptr_to_list_node(env, reg, regno, meta); + if (ret < 0) + return ret; + break; case KF_ARG_PTR_TO_BTF_ID: /* Only base_type is checked, further checks are done here */ if (reg->type != PTR_TO_BTF_ID && @@ -8542,11 +8801,6 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); if (meta.btf == btf_vmlinux && btf_id_set_contains(&special_kfunc_set, meta.func_id)) { - if (!btf_type_is_void(ptr_type)) { - verbose(env, "kernel function %s must have void * return type\n", - meta.func_name); - return -EINVAL; - } if (meta.func_id == special_kfunc_list[KF_bpf_obj_new_impl]) { const struct btf_type *ret_t; struct btf *ret_btf; @@ -8584,6 +8838,15 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, env->insn_aux_data[insn_idx].kptr_struct_meta = btf_find_struct_meta(meta.arg_obj_drop.btf, meta.arg_obj_drop.btf_id); + } else if (meta.func_id == special_kfunc_list[KF_bpf_list_pop_front] || + meta.func_id == special_kfunc_list[KF_bpf_list_pop_back]) { + struct btf_field *field = meta.arg_list_head.field; + + mark_reg_known_zero(env, regs, BPF_REG_0); + regs[BPF_REG_0].type = PTR_TO_BTF_ID | MEM_ALLOC; + regs[BPF_REG_0].btf = field->list_head.btf; + regs[BPF_REG_0].btf_id = field->list_head.value_btf_id; + regs[BPF_REG_0].off = field->list_head.node_offset; } else { verbose(env, "kernel function %s unhandled dynamic return type\n", meta.func_name); @@ -13262,11 +13525,14 @@ static int do_check(struct bpf_verifier_env *env) return -EINVAL; } - if (env->cur_state->active_lock.ptr && - (insn->src_reg == BPF_PSEUDO_CALL || - insn->imm != BPF_FUNC_spin_unlock)) { - verbose(env, "function calls are not allowed while holding a lock\n"); - return -EINVAL; + if (env->cur_state->active_lock.ptr) { + if ((insn->src_reg == BPF_REG_0 && insn->imm != BPF_FUNC_spin_unlock) || + (insn->src_reg == BPF_PSEUDO_CALL) || + (insn->src_reg == BPF_PSEUDO_KFUNC_CALL && + (insn->off != 0 || !is_bpf_list_api_kfunc(insn->imm)))) { + verbose(env, "function calls are not allowed while holding a lock\n"); + return -EINVAL; + } } if (insn->src_reg == BPF_PSEUDO_CALL) err = check_func_call(env, insn, &env->insn_idx); diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index d145c93ea663..828dd868b658 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -35,4 +35,32 @@ extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; /* Convenience macro to wrap over bpf_obj_drop_impl */ #define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL) +/* Description + * Add a new entry to the beginning of the BPF linked list. + * Returns + * Void. + */ +extern void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node) __ksym; + +/* Description + * Add a new entry to the end of the BPF linked list. + * Returns + * Void. + */ +extern void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node) __ksym; + +/* Description + * Remove the entry at the beginning of the BPF linked list. + * Returns + * Pointer to bpf_list_node of deleted entry, or NULL if list is empty. + */ +extern struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head) __ksym; + +/* Description + * Remove the entry at the end of the BPF linked list. + * Returns + * Pointer to bpf_list_node of deleted entry, or NULL if list is empty. + */ +extern struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head) __ksym; + #endif From patchwork Mon Nov 7 23:09:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035462 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01081C4332F for ; Mon, 7 Nov 2022 23:11:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232564AbiKGXLT (ORCPT ); Mon, 7 Nov 2022 18:11:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232662AbiKGXLO (ORCPT ); Mon, 7 Nov 2022 18:11:14 -0500 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3644224BDD for ; Mon, 7 Nov 2022 15:11:12 -0800 (PST) Received: by mail-pf1-x443.google.com with SMTP id v28so12183519pfi.12 for ; Mon, 07 Nov 2022 15:11:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WErGwUy0Dg6GQhKNp6eaQMti1iGRFV8I7VW8cRQQLXU=; b=UmjRD0FWSD5uzytfRHaearCqGlfthg6eDpbwPQQno9OMnfYEzsbawD7fOCgaG2c8Gr 3/qYyJH7vwyKCwWXLQq9Jap1Bt3Gx0jT8jV7LxHacisqCfGLH89YH2BIcugvcJn2tulj d71/wlwYv6XZun4Rfy8cnNNTOmgvj0lUzOM2HDYb8jSIFCeMzfP8e+4/W7TmI32aMqMt kAiQTju1OvDATgYs8jwcHbm4Kps0uBTjKptN+K4Em+q/Okb7Or9Gk/kkV3Q8C5iJ2Tz+ jLtu8j2yJyS+5p+eXWRS5+2V/Gz32wAKUgJoTx3lCJ7kmx0lt7f0Qcp2GaMzeRz4R8K0 rstA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WErGwUy0Dg6GQhKNp6eaQMti1iGRFV8I7VW8cRQQLXU=; b=Zuzx9T6UNVY+9c1pTpsbChCCVfsJ6HiRRYsE0xsbFiu+SV47u1fI1tRcsUMK0rA9Rr gxkdGUWl+lBL/r2K+apkZdkLDannskvj/n6y2yQCRCbU+C+PXTboaDmmq9Wyc1hIM9jy /ql1juDEnee+ppWXvhlGWl/OrtSHs5fGDE1ZnTYv/xRqbDr0r7LhNDjCRPrT6KztU4rH Lneud/RFkKXTZMv6sxkmMGon2v2IgsV35bGNdbV+1IlqmGiGbO0nkLTTvG3KqW4UXN38 xMnA61VJ/hC+xhLSeK4qSe768xox2OicPWU/c/QOt8J5EezLfsFeWZyu2ItNwBNk8b+l Gefg== X-Gm-Message-State: ACrzQf26b11uMU0vjEAcr+GRbIad8FLv+q1kzzp+jeX1mfjamLymhHu1 Lr81GYVq1Ywq8GwPlD5k4NKeT6hQcUclOQ== X-Google-Smtp-Source: AMsMyM7K2skyDF+xdYBQaD6t/dpPQAypkN4MSOeLobuxKY1q1iQNDsflyZ+rA/Eob8fmQbbYLVQGWQ== X-Received: by 2002:a63:4182:0:b0:46f:1263:1f6 with SMTP id o124-20020a634182000000b0046f126301f6mr45269279pga.611.1667862671459; Mon, 07 Nov 2022 15:11:11 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id d1-20020a63f241000000b0043ae1797e2bsm4586931pgk.63.2022.11.07.15.11.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:11 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 20/25] bpf: Add 'release on unlock' logic for bpf_list_push_{front,back} Date: Tue, 8 Nov 2022 04:39:45 +0530 Message-Id: <20221107230950.7117-21-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5140; i=memxor@gmail.com; h=from:subject; bh=788X7f/Hodk3UU/0udiU3xZj5ZUkK6rnpr5rcLgSNkQ=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+3pPqRc6i+i8PmjSYiDV/gwTdI1rWSvcVKUxtq a1eXHLKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8RyjF1D/ 47NO5OeZXfqxSKyoPAJoV9/Ehjr7KxP8u7cSwg7sc7WeRf49b8c3NlIw0dvZ/SGilG/nUe3UECKquG XBcL/q/gLIIzuOqA6Z/GuLR7zER+k6Odyb4SilBrDrL75VpxGaHf0nKsaK9zXItukoKpIvf2BP/SJS l61CFM+y9mlZ4IEkv1E8579sSTXL3kDEKTiJ5xYKLmJXX/Ab/m0zF0AHPdOqDi/haBDnLB8cv0wOX3 G0lr3UNPf0vqXL/MQPgNTtKZ/TurMZzHQxQEGof2CBnDFHGjre6o7ruY49Ze08Je9BiM1a1yDeXu96 L9KVLwl2T5TLWCJH7iZFr1ClEC5ZpG6z3EiskOJznFScrv6SELSqnY+sTA3LcY/m/3+ztY+nJ5zAwl GhRc1gW2sxyXjibSWFUE1EyFPRPJykpJ7Sthmx0voa4uziyBXEW0kC+n4vBuMZX/ISozpHE+CURk8W NHaXtq38271OOKIpakunZpl4eRIBMtQJCnnW0WIDzA4rDLgQkjJsPPynyWjM4TDzO41GMu6Z/MEHtl kTWEKnaQj+4ujN+m5ujKCnbPL0OmBBQxLQXJuDlWdf9xB/0elBEub6yOXhkb8DUxKprOO3EMxzyClb A0kmapYZws93drQGasu/9Fl6hRnjcXAjE0F4PCrVFQXx9N7+JqvT66zKw7TQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit implements the delayed release logic for bpf_list_push_front and bpf_list_push_back. Once a node has been added to the list, it's pointer changes to PTR_UNTRUSTED. However, it is only released once the lock protecting the list is unlocked. For such local kptrs with PTR_UNTRUSTED set but an active ref_obj_id, it is still permitted to read them as long as the lock is held. Writing to them is not allowed. This allows having read access to push items we no longer own until we release the lock guarding the list, allowing a little more flexibility when working with these APIs. Note that enabling write support has fairly tricky interactions with what happens inside the critical section. Just as an example, currently, bpf_obj_drop is not permitted, but if it were, being able to write to the PTR_UNTRUSTED pointer while the object gets released back to the memory allocator would violate safety properties we wish to guarantee (i.e. not crashing the kernel). The memory could be reused for a different type in the BPF program or even in the kernel as it gets eventually kfree'd. Not enabling bpf_obj_drop inside the critical section would appear to prevent all of the above, but that is more of an artifical limitation right now. Since the write support is tangled with how we handle potential aliasing of nodes inside the critical section that may or may not be part of the list anymore, it has been deferred to a future patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf_verifier.h | 5 ++++ kernel/bpf/verifier.c | 48 +++++++++++++++++++++++++++++++++++- 2 files changed, 52 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index a009df64ffab..c3b202559b87 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -229,6 +229,11 @@ struct bpf_reference_state { * exiting a callback function. */ int callback_ref; + /* Mark the reference state to release the registers sharing the same id + * on bpf_spin_unlock (for nodes that we will lose ownership to but are + * safe to access inside the critical section). + */ + bool release_on_unlock; }; /* state of the program: diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5b87ef859046..1900af72df94 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -5659,7 +5659,9 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, cur->active_lock.ptr = btf; cur->active_lock.id = reg->id; } else { + struct bpf_func_state *fstate = cur_func(env); void *ptr; + int i; if (map) ptr = map; @@ -5677,6 +5679,16 @@ static int process_spin_lock(struct bpf_verifier_env *env, int regno, } cur->active_lock.ptr = NULL; cur->active_lock.id = 0; + + for (i = 0; i < fstate->acquired_refs; i++) { + /* WARN because this reference state cannot be freed + * before this point, as bpf_spin_lock CS does not + * allow functions that release the local kptr + * immediately. + */ + if (fstate->refs[i].release_on_unlock) + WARN_ON_ONCE(release_reference(env, fstate->refs[i].id)); + } } return 0; } @@ -8269,6 +8281,39 @@ static int process_kf_arg_ptr_to_kptr_strong(struct bpf_verifier_env *env, return 0; } +static int ref_set_release_on_unlock(struct bpf_verifier_env *env, u32 ref_obj_id) +{ + struct bpf_func_state *state = cur_func(env); + struct bpf_reg_state *reg; + int i; + + /* bpf_spin_lock only allows calling list_push and list_pop, no BPF + * subprogs, no global functions. This means that the references would + * not be released inside the critical section but they may be added to + * the reference state, and the acquired_refs are never copied out for a + * different frame as BPF to BPF calls don't work in bpf_spin_lock + * critical sections. + */ + if (!ref_obj_id) { + verbose(env, "verifier internal error: ref_obj_id is zero for release_on_unlock\n"); + return -EFAULT; + } + for (i = 0; i < state->acquired_refs; i++) { + if (state->refs[i].id == ref_obj_id) { + WARN_ON_ONCE(state->refs[i].release_on_unlock); + state->refs[i].release_on_unlock = true; + /* Now mark everyone sharing same ref_obj_id as untrusted */ + bpf_for_each_reg_in_vstate(env->cur_state, state, reg, ({ + if (reg->ref_obj_id == ref_obj_id) + reg->type |= PTR_UNTRUSTED; + })); + return 0; + } + } + verbose(env, "verifier internal error: ref state missing for ref_obj_id\n"); + return -EFAULT; +} + /* Implementation details: * * Each register points to some region of memory, which we define as an @@ -8454,7 +8499,8 @@ static int process_kf_arg_ptr_to_list_node(struct bpf_verifier_env *env, field->list_head.node_offset); return -EINVAL; } - return 0; + /* Set arg#1 for expiration after unlock */ + return ref_set_release_on_unlock(env, reg->ref_obj_id); } static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta) From patchwork Mon Nov 7 23:09:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035464 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5DD7C43217 for ; Mon, 7 Nov 2022 23:11:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232862AbiKGXLU (ORCPT ); Mon, 7 Nov 2022 18:11:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232932AbiKGXLQ (ORCPT ); Mon, 7 Nov 2022 18:11:16 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F5AC252A2 for ; Mon, 7 Nov 2022 15:11:15 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id l2so12547049pld.13 for ; Mon, 07 Nov 2022 15:11:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MvClsRh3v5gFbaU1jzvofNyI0Z9RpJBT7Vd2BU8lRCc=; b=leCXJnBsTqt2zPsVDMw7+SsiUzBtyKJv4xu9hGO4K0y1e2vUd2bp8KS1HYonLa9rzm +Vx36ZgYw5ajMG2NGGKC8xxQqe4+daI9VmZ12dPnZubUL1H5M2eFQjy5lzgahMA6EAQ3 FlHLhVszKkoF8MGybJwEjmmaKNAvK1T3Tkiu5MZV0hSj1MZlvkF7sEjGq43xv4JjKqul NvH28LwBb9iI6xNQs/lJ9nrmJu3Juz0OWzdkybERZOao8aSbkKrjH+v4UTPAvueokJbj I5j2c2vbDQpZgX8CQekhG0Dpa1Wiq/wSV+uJC9rkZY/EENE2qdiLSMnaV33MDU4ig59x okcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MvClsRh3v5gFbaU1jzvofNyI0Z9RpJBT7Vd2BU8lRCc=; b=u8uNcV58IivWu7pPelapNRwQMwi+KgnTqTDi3A7B7k4fbSbWSK4S1ZufJI/fOfe8q6 Y/HmFWjq+thIzyu3m2WAFEu17MwluqZuP1OS9ClmOL+KCDR9kBwWxxcgRKBn5OXIYMWP lxccRjyWo4Q6Xam+SbtaBIgeeIM5d+GW4vthG2+XUekXPGlUVSFDlcxJZl4F3F5TuOWV H20lcHgaLyid+NgFJuTRF4c2hAbHgWB3by/E/WnKIreZ+MvmDbexLsHkCmYMyG1CXx5w lZmaBD7twahT9IgDKtrrEMzgQnPOFXEGMyEQBJqCWG2Pga9M3vuWUJ5bzoec3h3A8zvh iEIg== X-Gm-Message-State: ACrzQf1MwGhoTTSdjsl/jpRWhH46TCpnkOpYwPBSKfexGb8yov61jMIz xcoDsPj63gZXsokY2DXxKIt0V1gNEnvgWQ== X-Google-Smtp-Source: AMsMyM6qxJYObRu/o6OaNewQ/JHvvsP0Hr1jXcDUP+8+un2UB01d3JS2KIK7epiV3o1HhgTUau3qGA== X-Received: by 2002:a17:902:9b8e:b0:187:30ec:67dd with SMTP id y14-20020a1709029b8e00b0018730ec67ddmr38613316plp.79.1667862674923; Mon, 07 Nov 2022 15:11:14 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id w7-20020a634747000000b0042b5095b7b4sm4662062pgk.5.2022.11.07.15.11.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:14 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 21/25] selftests/bpf: Add __contains macro to bpf_experimental.h Date: Tue, 8 Nov 2022 04:39:46 +0530 Message-Id: <20221107230950.7117-22-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=963; i=memxor@gmail.com; h=from:subject; bh=3SBEBkFd6yk9WI5S8fottjaJAS2IxuA3Y+yaTULKiTw=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+3FvEaxj1+aO2Gm8TsAmFakFfU+XZWjWzSjQ6/ 9REQUBCJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8RyvrPD/ 9jCmikgaHEMhk1kIml3UXMAoLS2f2cxwCn2YlpS1dVAaNX/LoDOrxJSzyAyPigooOUugtTBNw4bIKl zMYsS+4KTTGsHS1Aiufsdw71V18ZhNV5blhDpToZpAzLyTVZY+FPUVxLDK5Y1RMO9BgcD63PjwRdBZ YDI/wSdt3teb9mkj+JRzEeU+nh11mQqFuKRCDcHs9tEap+rcNvYE/R7pi0MGr4AvWv+wee3rReGn3j WMW1VpnM//MJW0XLM429iCx4XTFMPVpaLycXIaUlWEMLaRqctfW5YoG36IHHQMww4QW49l4zgmXQpz JKDj5bF4tmlKzUX9uCV6Q7gytnRiJD6BMhJuCY1ssLyMq8zYJLV6tYFuwXiNgzIPy3i51f6n7viDIc sPdEANvQ7P+zipHnPk2eqlQvl+JtoEAXOztBgODPKwudpvDI4TzcowZQKQ3ovrxQVDAc5XZjXK/M79 u8k7kKkF7zdgMH5IQAxzXo+BRDi0uYyP1q5+7PjbjX9hk49+TCMTy7W5j5yIQfVoIsXoztQD2mrfw9 3nTcrB8TrqUL/gZEHGps5mXZ15h+EGTFfH5AzhXXjfaL2SegqGuG2SdscIUYw51A3QOkFWWu5kA+ky rFxCvc28LVKJd8F8zz6frRpte9bfWSz8N5b9lVgbmTgxzfUSqJtHoqNgu0LA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add user facing __contains macro which provides a convenient wrapper over the verbose kernel specific BTF declaration tag required to annotate BPF list head structs in user types. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/testing/selftests/bpf/bpf_experimental.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 828dd868b658..c5103587b6df 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -6,6 +6,8 @@ #include #include +#define __contains(name, node) __attribute__((btf_decl_tag("contains:" #name ":" #node))) + /* Description * Allocates a local kptr of type represented by 'local_type_id' in program * BTF. User may use the bpf_core_type_id_local macro to pass the type ID From patchwork Mon Nov 7 23:09:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035465 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53998C43219 for ; Mon, 7 Nov 2022 23:11:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232913AbiKGXLV (ORCPT ); Mon, 7 Nov 2022 18:11:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47774 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232448AbiKGXLS (ORCPT ); Mon, 7 Nov 2022 18:11:18 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4826C21255 for ; Mon, 7 Nov 2022 15:11:18 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id 130so12200860pfu.8 for ; Mon, 07 Nov 2022 15:11:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a/pYzeiT/HbBhrq855JyBemRpl0D3d/RiN0J/SST5Cs=; b=gsgVwgZGORhaP2GIqXpP9ojPnOGkBoR18Em6l18F4OUnOKtCHtsBJ1nd0cKjn5I/Jf vaTI1QI5fgmC8wg4xaNR5xmXvs1MMCI1oCVpDOpeKf+8u3WEhiFoIVqd2mfQHzkxE+4F V5qvemXTcLEmxZcqRjmICCyXQfILJKjWWnV250qED/k+OYRE6YUCqUx6h4i5orHkg5pL a0dvAi6cpXsrQer9p7/uM87DOyW88dCRD0FuAWw0utmT6XQFU2aqDP8c41nDF/bzlVjU m0ZglP1AenPd5EwoW4JlOs0H++52k+iQGAIVIF80vwEf8vFOkIgVaPaYS+TH9datp4q3 vQJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a/pYzeiT/HbBhrq855JyBemRpl0D3d/RiN0J/SST5Cs=; b=17xMURfXDaUWSi5O3VTBP8vtlaOmHr4hBpLKgsfSKSFUzIHTNAT0emjb1tWPNjo+pI Qc/nDA1k8WippOPzb6R1lFA8imBL8Wf1AhK83n0B5Afmqhb2UhliujtJLUlkcxiswpDN R+FXg6cWTsCkpw71hkFeA9g1/9vb4QjHDIG54OgMaxgZ5JTQhySe92urNZvTVVFJZ/sc R2Mdd2yUZAJ7knF+tmJIeh3y6apzA+Ag+VcvKKdyJXSUMrfybOjWgigwaq4RTOU6JSYE 4iRdnLvM8m9IyqbY4/hNYv/JqH4FtoYAiLjes9Q+2XJpjprWB+kCSqzEgDrmIhXXBWur avxw== X-Gm-Message-State: ACrzQf2Lwbz8TJ0NrxlhD1dOT5ocbqFzPIWvkQ36msjYWbUigYAcnOyZ Oq8K1S8RFRyV908UYS1TxnM1AyR2ZPwMqA== X-Google-Smtp-Source: AMsMyM6my81H3VwTa4pVBumepcvrG4hgXqbq8xxwhMFKZp9YTJAIQOnQovoga3LEbxtRb2Ih7B2nxw== X-Received: by 2002:a63:24d:0:b0:452:87c1:9781 with SMTP id 74-20020a63024d000000b0045287c19781mr45686073pgc.512.1667862677595; Mon, 07 Nov 2022 15:11:17 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id f186-20020a6238c3000000b0056e5bce5b7asm4977209pfa.201.2022.11.07.15.11.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:17 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 22/25] selftests/bpf: Update spinlock selftest Date: Tue, 8 Nov 2022 04:39:47 +0530 Message-Id: <20221107230950.7117-23-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4216; i=memxor@gmail.com; h=from:subject; bh=gzlzDi/pDR4JSxvoLhdPAoukG/FRGz4EvJTMKaO3GFo=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+3/eQd65yff16YKsQ6wIDB40eW3Biv3CaU6wh+ IAIsz6KJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPtwAKCRBM4MiGSL8RyiyGEA Cw1USzboLjyl0Ki+MGUAUlgVZID/SPjkRloC/HW/hpnJ1w8lea9KGPnlBP2/gQOuNOF069i6+SjE+i 19UGvz1iu1T1UbvAlfH2zqnWk7sCdtVRHcujnDV9MbYkSw+XLO8xNKh/PS4opmqY3RyNRMe6J7T5N9 RBRKctnUPwWNmW3dUgmPi2WjdVfsjfR5NJF0f8LvopJ6xTJqeSAEgWWt6WuYElnjPxIUddXKEOo8WQ X+TxLRCoSlMgP8mFwTlEZrYwSD4q5P9K/LpuXxoXAlKguWNIumWiwCj/sX3Fud9xUskl96LElTaQD2 iFoA8CnLZqJHBAyBw27+qfwUk89PVPzD4WBzTWv3AtE2MJPkojX3poliLwZGEBh4O2EFvXTgb7RH3k lnLeQa5XgY/Z7A8OvRAtMFOPeRuB9f3jCGAioMUfpb9qVpl/o10iJ/ACjNPaKE0O9RC8RONdMZ1vGF ufHSuz+8Z4LzfEuz7MfUxz/QLIp+dw2HwKYyemRCsLisgQ6dU3iQVqWpyTmUqFnVpI3RDAb06mnsfp Dvp0XhsXlLgscd8J5Qgjotc/h3zqDJAuvkhMJgEQNMX1mhRGTxGX1cEsAjYRiuwZxxi4DXFqo8IItw EWdNn99tzIpeR6r3LIrCnucAUxCcPpo1uI7Tfd5a/aHpCsO2NYVQnSCPYDKQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Make updates in preparation for adding more test cases to this selftest: - Convert from CHECK_ to ASSERT macros. - Use BPF skeleton - Fix typo sping -> spin - Rename spinlock.c -> spin_lock.c Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/spin_lock.c | 46 +++++++++++++++++++ .../selftests/bpf/prog_tests/spinlock.c | 45 ------------------ .../selftests/bpf/progs/test_spin_lock.c | 4 +- 3 files changed, 48 insertions(+), 47 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/spin_lock.c delete mode 100644 tools/testing/selftests/bpf/prog_tests/spinlock.c diff --git a/tools/testing/selftests/bpf/prog_tests/spin_lock.c b/tools/testing/selftests/bpf/prog_tests/spin_lock.c new file mode 100644 index 000000000000..1f720002fe56 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/spin_lock.c @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +#include "test_spin_lock.skel.h" + +static void *spin_lock_thread(void *arg) +{ + int err, prog_fd = *(u32 *) arg; + LIBBPF_OPTS(bpf_test_run_opts, topts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .repeat = 10000, + ); + + err = bpf_prog_test_run_opts(prog_fd, &topts); + ASSERT_OK(err, "test_run"); + ASSERT_OK(topts.retval, "test_run retval"); + pthread_exit(arg); +} + +void test_spinlock(void) +{ + struct test_spin_lock *skel; + pthread_t thread_id[4]; + int prog_fd, i; + void *ret; + + skel = test_spin_lock__open_and_load(); + if (!ASSERT_OK_PTR(skel, "test_spin_lock__open_and_load")) + return; + prog_fd = bpf_program__fd(skel->progs.bpf_spin_lock_test); + for (i = 0; i < 4; i++) + if (!ASSERT_OK(pthread_create(&thread_id[i], NULL, + &spin_lock_thread, &prog_fd), "pthread_create")) + goto end; + + for (i = 0; i < 4; i++) { + if (!ASSERT_OK(pthread_join(thread_id[i], &ret), "pthread_join")) + goto end; + if (!ASSERT_EQ(ret, &prog_fd, "ret == prog_fd")) + goto end; + } +end: + test_spin_lock__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/prog_tests/spinlock.c b/tools/testing/selftests/bpf/prog_tests/spinlock.c deleted file mode 100644 index 15eb1372d771..000000000000 --- a/tools/testing/selftests/bpf/prog_tests/spinlock.c +++ /dev/null @@ -1,45 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -#include -#include - -static void *spin_lock_thread(void *arg) -{ - int err, prog_fd = *(u32 *) arg; - LIBBPF_OPTS(bpf_test_run_opts, topts, - .data_in = &pkt_v4, - .data_size_in = sizeof(pkt_v4), - .repeat = 10000, - ); - - err = bpf_prog_test_run_opts(prog_fd, &topts); - ASSERT_OK(err, "test_run"); - ASSERT_OK(topts.retval, "test_run retval"); - pthread_exit(arg); -} - -void test_spinlock(void) -{ - const char *file = "./test_spin_lock.bpf.o"; - pthread_t thread_id[4]; - struct bpf_object *obj = NULL; - int prog_fd; - int err = 0, i; - void *ret; - - err = bpf_prog_test_load(file, BPF_PROG_TYPE_CGROUP_SKB, &obj, &prog_fd); - if (CHECK_FAIL(err)) { - printf("test_spin_lock:bpf_prog_test_load errno %d\n", errno); - goto close_prog; - } - for (i = 0; i < 4; i++) - if (CHECK_FAIL(pthread_create(&thread_id[i], NULL, - &spin_lock_thread, &prog_fd))) - goto close_prog; - - for (i = 0; i < 4; i++) - if (CHECK_FAIL(pthread_join(thread_id[i], &ret) || - ret != (void *)&prog_fd)) - goto close_prog; -close_prog: - bpf_object__close(obj); -} diff --git a/tools/testing/selftests/bpf/progs/test_spin_lock.c b/tools/testing/selftests/bpf/progs/test_spin_lock.c index 7e88309d3229..5bd10409285b 100644 --- a/tools/testing/selftests/bpf/progs/test_spin_lock.c +++ b/tools/testing/selftests/bpf/progs/test_spin_lock.c @@ -45,8 +45,8 @@ struct { #define CREDIT_PER_NS(delta, rate) (((delta) * rate) >> 20) -SEC("tc") -int bpf_sping_lock_test(struct __sk_buff *skb) +SEC("cgroup_skb/ingress") +int bpf_spin_lock_test(struct __sk_buff *skb) { volatile int credit = 0, max_credit = 100, pkt_len = 64; struct hmap_elem zero = {}, *val; From patchwork Mon Nov 7 23:09:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035466 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8760C433FE for ; Mon, 7 Nov 2022 23:11:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232882AbiKGXLZ (ORCPT ); Mon, 7 Nov 2022 18:11:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232880AbiKGXLW (ORCPT ); Mon, 7 Nov 2022 18:11:22 -0500 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5D702018E for ; Mon, 7 Nov 2022 15:11:21 -0800 (PST) Received: by mail-pj1-x1041.google.com with SMTP id b11so12137436pjp.2 for ; Mon, 07 Nov 2022 15:11:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VIQCRURQA1zUo2qbQsX+DwVwv5vTUxhfroRc5uMhdlc=; b=nzCbu2WGr3sRsoYtnAa72cHCHWp2eS0JtvgedIhuXu55f2ehAbt7ZL1Vn96YdVLq06 89QAQB/cPjDI/77vcLHGfHSu5UBWG39xZArpFU/W76aZgmPYhjNfa+Y3u8CdSEem4HUa WKMNp3J/5RQeBe5ipVlwCep3Ac5hAngWLMJechFI5JT6KPzG4AJyeWYUjFa6/FUXfHm3 pzNT48TATOFiJrsrH2AhWSRWzjlVnIG2FShboslZrHmWmg4TawuPx08EbL+IqLXTDkJ2 3QiE/uueGnw9ft1W4hl0pifAH3+KSw/Dp99TDl+CqE+5mzi8Al26uve3+UNhWn+q8DgQ WGbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VIQCRURQA1zUo2qbQsX+DwVwv5vTUxhfroRc5uMhdlc=; b=pMEuVmmsEPNHVOLdI5YB4LLg9q93D4jmTDokEnXCGrYSiIuYJPR2WSzaeqkE1hkaAD hORFh9M50RXHt7hOV9Wgmle997QSbe23oar5YBceAd7MwCZkbxTHu6ubrMMCn1miYfGc qaGK7dIq43EK8DiikckBxLvujzFsqH1Kr5CyjIiYzZOvZHLKui41JU3Ytau+mWKxXMBL f/oiLIStKmY9feDnHf0JbInW9GsHT4br8JPt2Ia4V1WYYawEZLMNSAPfDrFMeaz0nvL5 vHvF3xJFZLVocd2O2WCY2ojbnwH6LK3TFd5k4hkjGeg83mypPT8MzT4pebrXvFByKr6t yDFA== X-Gm-Message-State: ACrzQf1PFXtP3LBG9Sf5IeIqMX1nTnRoN7R7e9uq7vqvq8CGg1HdZnVh 8etM67lj+jrIYMzipXWLL5ClDdjK9ZSmdw== X-Google-Smtp-Source: AMsMyM6d1uOeKUw4El6up2sGaiwCKOx/iMjGg0446aiQy/iVFSTgEp8FbR0UkBXUCGKMxi3rZ53vWQ== X-Received: by 2002:a17:90a:70c2:b0:214:132a:2b90 with SMTP id a2-20020a17090a70c200b00214132a2b90mr843454pjm.195.1667862680848; Mon, 07 Nov 2022 15:11:20 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id j63-20020a625542000000b005627868e27esm4976061pfb.127.2022.11.07.15.11.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:20 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 23/25] selftests/bpf: Add failure test cases for spin lock pairing Date: Tue, 8 Nov 2022 04:39:48 +0530 Message-Id: <20221107230950.7117-24-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10926; i=memxor@gmail.com; h=from:subject; bh=k+3ZSMEGNPAv+SyMd8Vm+yW877Fe7P+gsH1G641amws=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+4b1Z9fAX0VtDGxVkg4/+HY+waaHNmZ5KuyC1V bHbXgJ+JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPuAAKCRBM4MiGSL8RyvUID/ wKgU/MJ8HU7bns4MaoeaALejNpdKPoJds56oA7uo27Ku99BdQCThdWbRn/Yz0tyzZLb6O38+WW4Y+L 5gPEY0LqZu6bfS/t3H1f7PXR6Q3rBtPQJHrapSvo//EhPI3zJWRO0GOlwEG8CJB04feU3ypfFC2Ejr M8HukN9/6glD2ex2iTHrqYg66kC9F+cnqb4PHWVVlvfayrN3GydcRwJOZyJbw4FPPu825G766ugroD QpEqOO1aSF8EmBbmdzqai8+3i/H100XPH/XRER1SH0FVA15hyUmMOYLdH6YUzY+4EFUnBgVrbNw2SJ y8gL8dmD/J36KCDj8lNgLsXuxFqBQLSUBflzvkUOoET0oXkO5CT2p/KtVfeH0IUmlHNM1CXtinsl2c Vf/W8ceZsncvSMDxJ7YSgq66RzuCua9JKVBneEU6P35zU5b2h76+5eB6AqGkZ8948k+VoVrTB9sNyz ygLFWMwNTY97+KEvL+9iuTzbQ5V0/zuS+u48kLonb0bZjYimeMD593qxfzJFkM2YF9TcJx7DXTqDXV eEWlipGC0YpCkPrqPaVvR8kBSGpjnpWJCAd3C23/Lh/5dUHQwb1FvgwnyrS8AdgAcdLp1PdLhCZ2cM gj9/b7DCOQfYIyTgUfAw51hxtQbg+acepdn4+KBXmdFkw9Dz4ooax8e9dzvg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net First, ensure that whenever a bpf_spin_lock is present in an allocation, the reg->id is preserved. This won't be true for global variables however, since they have a single map value per map, hence the verifier harcodes it to 0 (so that multiple pseudo ldimm64 insns can yield the same lock object per map at a given offset). Next, add test cases for all possible combinations (kptr, global, map value, inner map value). Since we lifted restriction on locking in inner maps, also add test cases for them. Currently, each lookup into an inner map gets a fresh reg->id, so even if the reg->map_ptr is same, they will be treated as separate allocations and the incorrect unlock pairing will be rejected. Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/spin_lock.c | 89 +++++++- .../selftests/bpf/progs/test_spin_lock_fail.c | 204 ++++++++++++++++++ 2 files changed, 292 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/test_spin_lock_fail.c diff --git a/tools/testing/selftests/bpf/prog_tests/spin_lock.c b/tools/testing/selftests/bpf/prog_tests/spin_lock.c index 1f720002fe56..5df959c78fee 100644 --- a/tools/testing/selftests/bpf/prog_tests/spin_lock.c +++ b/tools/testing/selftests/bpf/prog_tests/spin_lock.c @@ -3,6 +3,79 @@ #include #include "test_spin_lock.skel.h" +#include "test_spin_lock_fail.skel.h" + +static char log_buf[1024 * 1024]; + +static struct { + const char *prog_name; + const char *err_msg; +} spin_lock_fail_tests[] = { + { "lock_id_kptr_preserve", + "5: (bf) r1 = r0 ; R0_w=ptr_foo(id=2,ref_obj_id=2,off=0,imm=0) " + "R1_w=ptr_foo(id=2,ref_obj_id=2,off=0,imm=0) refs=2\n6: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=ptr_ expected=percpu_ptr_" }, + { "lock_id_global_zero", + "; R1_w=map_value(off=0,ks=4,vs=4,imm=0)\n2: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=map_value expected=percpu_ptr_" }, + { "lock_id_mapval_preserve", + "8: (bf) r1 = r0 ; R0_w=map_value(id=1,off=0,ks=4,vs=8,imm=0) " + "R1_w=map_value(id=1,off=0,ks=4,vs=8,imm=0)\n9: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=map_value expected=percpu_ptr_" }, + { "lock_id_innermapval_preserve", + "13: (bf) r1 = r0 ; R0=map_value(id=2,off=0,ks=4,vs=8,imm=0) " + "R1_w=map_value(id=2,off=0,ks=4,vs=8,imm=0)\n14: (85) call bpf_this_cpu_ptr#154\n" + "R1 type=map_value expected=percpu_ptr_" }, + { "lock_id_mismatch_kptr_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_kptr_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_kptr_mapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_kptr_innermapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_mapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_global_innermapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_mapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_mapval_innermapval", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_innermapval1", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_innermapval2", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_kptr", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_global", "bpf_spin_unlock of different lock" }, + { "lock_id_mismatch_innermapval_mapval", "bpf_spin_unlock of different lock" }, +}; + +static void test_spin_lock_fail_prog(const char *prog_name, const char *err_msg) +{ + LIBBPF_OPTS(bpf_object_open_opts, opts, .kernel_log_buf = log_buf, + .kernel_log_size = sizeof(log_buf), + .kernel_log_level = 1); + struct test_spin_lock_fail *skel; + struct bpf_program *prog; + int ret; + + skel = test_spin_lock_fail__open_opts(&opts); + if (!ASSERT_OK_PTR(skel, "test_spin_lock_fail__open_opts")) + return; + + prog = bpf_object__find_program_by_name(skel->obj, prog_name); + if (!ASSERT_OK_PTR(prog, "bpf_object__find_program_by_name")) + goto end; + + bpf_program__set_autoload(prog, true); + + ret = test_spin_lock_fail__load(skel); + if (!ASSERT_ERR(ret, "test_spin_lock_fail__load must fail")) + goto end; + + if (!ASSERT_OK_PTR(strstr(log_buf, err_msg), "expected error message")) { + fprintf(stderr, "Expected: %s\n", err_msg); + fprintf(stderr, "Verifier: %s\n", log_buf); + } + +end: + test_spin_lock_fail__destroy(skel); +} static void *spin_lock_thread(void *arg) { @@ -19,7 +92,7 @@ static void *spin_lock_thread(void *arg) pthread_exit(arg); } -void test_spinlock(void) +void test_spin_lock_success(void) { struct test_spin_lock *skel; pthread_t thread_id[4]; @@ -44,3 +117,17 @@ void test_spinlock(void) end: test_spin_lock__destroy(skel); } + +void test_spin_lock(void) +{ + int i; + + test_spin_lock_success(); + + for (i = 0; i < ARRAY_SIZE(spin_lock_fail_tests); i++) { + if (!test__start_subtest(spin_lock_fail_tests[i].prog_name)) + continue; + test_spin_lock_fail_prog(spin_lock_fail_tests[i].prog_name, + spin_lock_fail_tests[i].err_msg); + } +} diff --git a/tools/testing/selftests/bpf/progs/test_spin_lock_fail.c b/tools/testing/selftests/bpf/progs/test_spin_lock_fail.c new file mode 100644 index 000000000000..86cd183ef6dc --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_spin_lock_fail.c @@ -0,0 +1,204 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include "bpf_experimental.h" + +struct foo { + struct bpf_spin_lock lock; + int data; +}; + +struct array_map { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct foo); + __uint(max_entries, 1); +} array_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); + __uint(max_entries, 1); + __type(key, int); + __type(value, int); + __array(values, struct array_map); +} map_of_maps SEC(".maps") = { + .values = { + [0] = &array_map, + }, +}; + +SEC(".data.A") struct bpf_spin_lock lockA; +SEC(".data.B") struct bpf_spin_lock lockB; + +SEC("?tc") +int lock_id_kptr_preserve(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_this_cpu_ptr(f); + return 0; +} + +SEC("?tc") +int lock_id_global_zero(void *ctx) +{ + bpf_this_cpu_ptr(&lockA); + return 0; +} + +SEC("?tc") +int lock_id_mapval_preserve(void *ctx) +{ + struct foo *f; + int key = 0; + + f = bpf_map_lookup_elem(&array_map, &key); + if (!f) + return 0; + bpf_this_cpu_ptr(f); + return 0; +} + +SEC("?tc") +int lock_id_innermapval_preserve(void *ctx) +{ + struct foo *f; + int key = 0; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f = bpf_map_lookup_elem(map, &key); + if (!f) + return 0; + bpf_this_cpu_ptr(f); + return 0; +} + +#define CHECK(test, A, B) \ + SEC("?tc") \ + int lock_id_mismatch_##test(void *ctx) \ + { \ + struct foo *f1, *f2, *v, *iv; \ + int key = 0; \ + void *map; \ + \ + map = bpf_map_lookup_elem(&map_of_maps, &key); \ + if (!map) \ + return 0; \ + iv = bpf_map_lookup_elem(map, &key); \ + if (!iv) \ + return 0; \ + v = bpf_map_lookup_elem(&array_map, &key); \ + if (!v) \ + return 0; \ + f1 = bpf_obj_new(typeof(*f1)); \ + if (!f1) \ + return 0; \ + f2 = bpf_obj_new(typeof(*f2)); \ + if (!f2) { \ + bpf_obj_drop(f1); \ + return 0; \ + } \ + bpf_spin_lock(A); \ + bpf_spin_unlock(B); \ + return 0; \ + } + +CHECK(kptr_kptr, &f1->lock, &f2->lock); +CHECK(kptr_global, &f1->lock, &lockA); +CHECK(kptr_mapval, &f1->lock, &v->lock); +CHECK(kptr_innermapval, &f1->lock, &iv->lock); + +CHECK(global_global, &lockA, &lockB); +CHECK(global_kptr, &lockA, &f1->lock); +CHECK(global_mapval, &lockA, &v->lock); +CHECK(global_innermapval, &lockA, &iv->lock); + +SEC("?tc") +int lock_id_mismatch_mapval_mapval(void *ctx) +{ + struct foo *f1, *f2; + int key = 0; + + f1 = bpf_map_lookup_elem(&array_map, &key); + if (!f1) + return 0; + f2 = bpf_map_lookup_elem(&array_map, &key); + if (!f2) + return 0; + + bpf_spin_lock(&f1->lock); + f1->data = 42; + bpf_spin_unlock(&f2->lock); + + return 0; +} + +CHECK(mapval_kptr, &v->lock, &f1->lock); +CHECK(mapval_global, &v->lock, &lockB); +CHECK(mapval_innermapval, &v->lock, &iv->lock); + +SEC("?tc") +int lock_id_mismatch_innermapval_innermapval1(void *ctx) +{ + struct foo *f1, *f2; + int key = 0; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f1 = bpf_map_lookup_elem(map, &key); + if (!f1) + return 0; + f2 = bpf_map_lookup_elem(map, &key); + if (!f2) + return 0; + + bpf_spin_lock(&f1->lock); + f1->data = 42; + bpf_spin_unlock(&f2->lock); + + return 0; +} + +SEC("?tc") +int lock_id_mismatch_innermapval_innermapval2(void *ctx) +{ + struct foo *f1, *f2; + int key = 0; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f1 = bpf_map_lookup_elem(map, &key); + if (!f1) + return 0; + map = bpf_map_lookup_elem(&map_of_maps, &key); + if (!map) + return 0; + f2 = bpf_map_lookup_elem(map, &key); + if (!f2) + return 0; + + bpf_spin_lock(&f1->lock); + f1->data = 42; + bpf_spin_unlock(&f2->lock); + + return 0; +} + +CHECK(innermapval_kptr, &iv->lock, &f1->lock); +CHECK(innermapval_global, &iv->lock, &lockA); +CHECK(innermapval_mapval, &iv->lock, &v->lock); + +#undef CHECK + +char _license[] SEC("license") = "GPL"; From patchwork Mon Nov 7 23:09:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035467 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C01F0C4332F for ; Mon, 7 Nov 2022 23:11:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232847AbiKGXL3 (ORCPT ); Mon, 7 Nov 2022 18:11:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232501AbiKGXL1 (ORCPT ); Mon, 7 Nov 2022 18:11:27 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43EE52018E for ; Mon, 7 Nov 2022 15:11:25 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id k5so12120940pjo.5 for ; Mon, 07 Nov 2022 15:11:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+jVn5nGQJuqBXGvoa6OS8PRdM6jjqP3SM7OvRqWqhGE=; b=CvzFzSo8K8Ke4C05ywQ9aSqT/sHutRo9VSMwK0cKuWZmJXEDdtmDN90pGrWF6LmyNA t7YaNvP9dKe2wNKUdEbSRNVSXR80TvVosnydBqysr/iu8f2aHW6HC6dePAIFPi83Ew3P 6zOHVgT+QpSRVN3iSSxGuA9z4c1DB5q+P/v27QTdML0kyt8YNOd58wu8CK5PrQZwaZKo YULYxY+CK1Hv64MphW34VBAUcg6RcKpmapLUYbVnp7c0z3bHm0IH6Lm0179pG5S2BYze V45E1xjbi/isNw5A2AGBSTpHtOqX83bo9AqnVIPX+hWiA6tGvsvRECxy/ITtwdCD63jc PYxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+jVn5nGQJuqBXGvoa6OS8PRdM6jjqP3SM7OvRqWqhGE=; b=RGVZkYbgo3oz5EJS+ngRFQ3UvdT5wrsCwveafQ6Th9yvY7upzIZ06fGoNA8vJyqh3W hlgZ9LN6pQZSfFjH2GD5mnWZv3AcMmbdtW/a2yj/hgnYGpHYcHOFiTfiuAFuZ4RjMW0W Yo4elVf0pSGE9JMOZZmOt2dd4Ke+5FSjaDM5/xO/AZ2nV1W84nxMBaCp87rZWjwvLMur KrTozbk1qm5QLomkhB+QbLDEulLLaykwUZmFrEUWOQt8MFs/1dNNyINt376bZbcDyoiy GAIBa4CrzeVTrHKinMdWUCcPzE3iGCMqnwJVzmKFnckP6hrWEAhvnIyYfZWXpVVfv2bN Ze5A== X-Gm-Message-State: ACrzQf1afsmZkvsPJ4Do9D2b7EkmiqL1Wda065v39120ngbemyeHngo8 E55f2Irz7DQGb9+oFV/nZdaO3217wCJ7dw== X-Google-Smtp-Source: AMsMyM7OUdFuZDOE6dIHEVhh1pNU4QWOilxmi1cMCq+N8Xup26tJgd6Ybfb6Le49G1UM1GeJsln6mw== X-Received: by 2002:a17:902:ef47:b0:179:d18e:4262 with SMTP id e7-20020a170902ef4700b00179d18e4262mr51917125plx.22.1667862683975; Mon, 07 Nov 2022 15:11:23 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id t72-20020a63784b000000b0044ed37dbca8sm4699279pgc.2.2022.11.07.15.11.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:23 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 24/25] selftests/bpf: Add BPF linked list API tests Date: Tue, 8 Nov 2022 04:39:49 +0530 Message-Id: <20221107230950.7117-25-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=35450; i=memxor@gmail.com; h=from:subject; bh=vmY5PL7lxZqEzEaOWO+E2ZAqPruU/vWiP+3ffsyCFfg=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+4FczlFqjTYsyg9D9KUrNp0sN/HWqq9Uc6R0EG pvOWfxmJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPuAAKCRBM4MiGSL8RylPUEA CYGNzjotOYare6u9VN4iXnE5HE8PE8YOF4YiLaJ13jAjMffnqhV8P6H+TYzMxmrTpY5VdpPK0kBY6W 1oi2DYPcV5KdyUiO3/ANtHDtjobjcav0Ibre/ti4PCHufG6f7ltZlkUAwXS9hsTkU3VzJ6GiXsS099 Y8DzvxA0o2vfAso+yAGlL/JK7Cle5QO7bRFCUsngry22mCVa1lFtaglAZwLaSGP6FQ0mARYuC70K7/ aNbk3Wqe0q7JPlLI28UfuyogUdYgTbXNvGBXma42FJC5hKYR8DujNeL8ahczzjTX3pHTfPwo8uMaP+ fWCpIGdW9Fj1fSKjtqvkgvLmfhhioUMzHcGaAaUyKjjZcJNUqccH6FY6RMusKmNZzWBKHULaiqWb/i pzyqddll0zVJpVFuPO/S1AUFmprw0s3n1Usu77QxYTXGgNXZ2tPqFgO+EhHbDaUWhEf/A+CVrh6sFy fexX2BlLExN07IugFxx3a8d7lK6NCjNZ1IgRic7zLLN7TM5BMGjJmyUSdzDm6r97/JkeFf2Mpdi9Ic SKq0cNVp9psUBf8fRgSWF+QfYpjyrCP2LraX15yFmnOR69MSLbtmr1Obn+2nLH1RzMZWSqOEOEE8gA 6PUUcWNjsJjobAaVj+/+Ms8M5uJaauH9LJ5TSC4yAt0813nnILIFSysQhZUg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Include various tests covering the success and failure cases. Also, run the success cases at runtime to verify correctness of linked list manipulation routines, in addition to ensuring successful verification. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/linked_list.c | 253 ++++++++ .../testing/selftests/bpf/progs/linked_list.c | 370 +++++++++++ .../testing/selftests/bpf/progs/linked_list.h | 56 ++ .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++++++++++++ 5 files changed, 1261 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.c diff --git a/tools/testing/selftests/bpf/DENYLIST.s390x b/tools/testing/selftests/bpf/DENYLIST.s390x index be4e3d47ea3e..072243af93b0 100644 --- a/tools/testing/selftests/bpf/DENYLIST.s390x +++ b/tools/testing/selftests/bpf/DENYLIST.s390x @@ -33,6 +33,7 @@ ksyms_module # test_ksyms_module__open_and_load unex ksyms_module_libbpf # JIT does not support calling kernel function (kfunc) ksyms_module_lskel # test_ksyms_module_lskel__open_and_load unexpected error: -9 (?) libbpf_get_fd_by_id_opts # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +linked_list # JIT does not support calling kernel function (kfunc) lookup_key # JIT does not support calling kernel function (kfunc) lru_bug # prog 'printk': failed to auto-attach: -524 map_kptr # failed to open_and_load program: -524 (trampoline) diff --git a/tools/testing/selftests/bpf/prog_tests/linked_list.c b/tools/testing/selftests/bpf/prog_tests/linked_list.c new file mode 100644 index 000000000000..669ef4bb9b87 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/linked_list.c @@ -0,0 +1,253 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +#include "linked_list.skel.h" +#include "linked_list_fail.skel.h" + +static char log_buf[1024 * 1024]; + +static struct { + const char *prog_name; + const char *err_msg; +} linked_list_fail_tests[] = { +#define TEST(test, off) \ + { #test "_missing_lock_push_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_push_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, + TEST(kptr, 32) + TEST(global, 16) + TEST(map, 0) + TEST(inner_map, 0) +#undef TEST +#define TEST(test, op) \ + { #test "_kptr_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=32 must be held for bpf_list_head" }, \ + { #test "_global_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=16 must be held for bpf_list_head" }, \ + { #test "_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, \ + { #test "_inner_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, + TEST(kptr, push_front) + TEST(kptr, push_back) + TEST(kptr, pop_front) + TEST(kptr, pop_back) + TEST(global, push_front) + TEST(global, push_back) + TEST(global, pop_front) + TEST(global, pop_back) + TEST(map, push_front) + TEST(map, push_back) + TEST(map, pop_front) + TEST(map, pop_back) + TEST(inner_map, push_front) + TEST(inner_map, push_back) + TEST(inner_map, pop_front) + TEST(inner_map, pop_back) +#undef TEST + { "map_compat_kprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_kretprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_perf", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp_w", "tracing progs cannot use bpf_list_head yet" }, + { "obj_type_id_oor", "local type ID argument must be in range [0, U32_MAX]" }, + { "obj_new_no_composite", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_new_no_struct", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_drop_non_zero_off", "R1 must have zero offset when passed to release func" }, + { "new_null_ret", "R0 invalid mem access 'ptr_or_null_'" }, + { "obj_new_acq", "Unreleased reference id=" }, + { "use_after_drop", "invalid mem access 'scalar'" }, + { "ptr_walk_scalar", "type=scalar expected=percpu_ptr_" }, + { "direct_read_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_write_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_read_head", "direct access to bpf_list_head is disallowed" }, + { "direct_write_head", "direct access to bpf_list_head is disallowed" }, + { "direct_read_node", "direct access to bpf_list_node is disallowed" }, + { "direct_write_node", "direct access to bpf_list_node is disallowed" }, + { "write_after_push_front", "only read is supported" }, + { "write_after_push_back", "only read is supported" }, + { "use_after_unlock_push_front", "invalid mem access 'scalar'" }, + { "use_after_unlock_push_back", "invalid mem access 'scalar'" }, + { "double_push_front", "arg#1 expected pointer to local kptr" }, + { "double_push_back", "arg#1 expected pointer to local kptr" }, + { "no_node_value_type", "bpf_list_node not found for local kptr\n" }, + { "incorrect_value_type", "bpf_list_head value type does not match arg#1" }, + { "incorrect_node_var_off", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_node_off1", "bpf_list_node not found at offset=1" }, + { "incorrect_node_off2", "arg#1 offset must be for bpf_list_node at off=0" }, + { "no_head_type", "bpf_list_head not found for local kptr" }, + { "incorrect_head_var_off1", "R1 doesn't have constant offset" }, + { "incorrect_head_var_off2", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_head_off1", "bpf_list_head not found at offset=17" }, + { "incorrect_head_off2", "bpf_list_head not found at offset=1" }, + { "pop_front_off", + "15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " + "R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, + { "pop_back_off", + "15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " + "R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, +}; + +static void test_linked_list_fail_prog(const char *prog_name, const char *err_msg) +{ + LIBBPF_OPTS(bpf_object_open_opts, opts, .kernel_log_buf = log_buf, + .kernel_log_size = sizeof(log_buf), + .kernel_log_level = 1); + struct linked_list_fail *skel; + struct bpf_program *prog; + int ret; + + skel = linked_list_fail__open_opts(&opts); + if (!ASSERT_OK_PTR(skel, "linked_list_fail__open_opts")) + return; + + prog = bpf_object__find_program_by_name(skel->obj, prog_name); + if (!ASSERT_OK_PTR(prog, "bpf_object__find_program_by_name")) + goto end; + + bpf_program__set_autoload(prog, true); + + ret = linked_list_fail__load(skel); + if (!ASSERT_ERR(ret, "linked_list_fail__load must fail")) + goto end; + + if (!ASSERT_OK_PTR(strstr(log_buf, err_msg), "expected error message")) { + fprintf(stderr, "Expected: %s\n", err_msg); + fprintf(stderr, "Verifier: %s\n", log_buf); + } + +end: + linked_list_fail__destroy(skel); +} + +static void clear_fields(struct bpf_map *map) +{ + char buf[24]; + int key = 0; + + memset(buf, 0xff, sizeof(buf)); + ASSERT_OK(bpf_map__update_elem(map, &key, sizeof(key), buf, sizeof(buf), 0), "check_and_free_fields"); +} + +enum { + TEST_ALL, + PUSH_POP, + PUSH_POP_MULT, + LIST_IN_LIST, +}; + +static void test_linked_list_success(int mode, bool leave_in_map) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .repeat = 1, + ); + struct linked_list *skel; + int ret; + + skel = linked_list__open_and_load(); + if (!ASSERT_OK_PTR(skel, "linked_list__open_and_load")) + return; + + if (mode == LIST_IN_LIST) + goto lil; + if (mode == PUSH_POP_MULT) + goto ppm; + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.map_list_push_pop), &opts); + ASSERT_OK(ret, "map_list_push_pop"); + ASSERT_OK(opts.retval, "map_list_push_pop retval"); + if (!leave_in_map) + clear_fields(skel->maps.array_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.inner_map_list_push_pop), &opts); + ASSERT_OK(ret, "inner_map_list_push_pop"); + ASSERT_OK(opts.retval, "inner_map_list_push_pop retval"); + if (!leave_in_map) + clear_fields(skel->maps.inner_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_push_pop), &opts); + ASSERT_OK(ret, "global_list_push_pop"); + ASSERT_OK(opts.retval, "global_list_push_pop retval"); + if (!leave_in_map) + clear_fields(skel->maps.data_A); + + if (mode == PUSH_POP) + goto end; + +ppm: + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.map_list_push_pop_multiple), &opts); + ASSERT_OK(ret, "map_list_push_pop_multiple"); + ASSERT_OK(opts.retval, "map_list_push_pop_multiple retval"); + if (!leave_in_map) + clear_fields(skel->maps.array_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.inner_map_list_push_pop_multiple), &opts); + ASSERT_OK(ret, "inner_map_list_push_pop_multiple"); + ASSERT_OK(opts.retval, "inner_map_list_push_pop_multiple retval"); + if (!leave_in_map) + clear_fields(skel->maps.inner_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_push_pop_multiple), &opts); + ASSERT_OK(ret, "global_list_push_pop_multiple"); + ASSERT_OK(opts.retval, "global_list_push_pop_multiple retval"); + if (!leave_in_map) + clear_fields(skel->maps.data_A); + + if (mode == PUSH_POP_MULT) + goto end; + +lil: + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.map_list_in_list), &opts); + ASSERT_OK(ret, "map_list_in_list"); + ASSERT_OK(opts.retval, "map_list_in_list retval"); + if (!leave_in_map) + clear_fields(skel->maps.array_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.inner_map_list_in_list), &opts); + ASSERT_OK(ret, "inner_map_list_in_list"); + ASSERT_OK(opts.retval, "inner_map_list_in_list retval"); + if (!leave_in_map) + clear_fields(skel->maps.inner_map); + + ret = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.global_list_in_list), &opts); + ASSERT_OK(ret, "global_list_in_list"); + ASSERT_OK(opts.retval, "global_list_in_list retval"); + if (!leave_in_map) + clear_fields(skel->maps.data_A); +end: + linked_list__destroy(skel); +} + +void test_linked_list(void) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(linked_list_fail_tests); i++) { + if (!test__start_subtest(linked_list_fail_tests[i].prog_name)) + continue; + test_linked_list_fail_prog(linked_list_fail_tests[i].prog_name, + linked_list_fail_tests[i].err_msg); + } + test_linked_list_success(PUSH_POP, false); + test_linked_list_success(PUSH_POP, true); + test_linked_list_success(PUSH_POP_MULT, false); + test_linked_list_success(PUSH_POP_MULT, true); + test_linked_list_success(LIST_IN_LIST, false); + test_linked_list_success(LIST_IN_LIST, true); + test_linked_list_success(TEST_ALL, false); +} diff --git a/tools/testing/selftests/bpf/progs/linked_list.c b/tools/testing/selftests/bpf/progs/linked_list.c new file mode 100644 index 000000000000..2c7b615c6d41 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/linked_list.c @@ -0,0 +1,370 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_experimental.h" + +#ifndef ARRAY_SIZE +#define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) +#endif + +#include "linked_list.h" + +static __always_inline +int list_push_pop(struct bpf_spin_lock *lock, struct bpf_list_head *head, bool leave_in_map) +{ + struct bpf_list_node *n; + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 2; + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + bpf_obj_drop(f); + return 3; + } + + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + bpf_obj_drop(f); + return 4; + } + + + bpf_spin_lock(lock); + f->data = 42; + bpf_list_push_front(head, &f->node); + bpf_spin_unlock(lock); + if (leave_in_map) + return 0; + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (!n) + return 5; + f = container_of(n, struct foo, node); + if (f->data != 42) { + bpf_obj_drop(f); + return 6; + } + + bpf_spin_lock(lock); + f->data = 13; + bpf_list_push_front(head, &f->node); + bpf_spin_unlock(lock); + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (!n) + return 7; + f = container_of(n, struct foo, node); + if (f->data != 13) { + bpf_obj_drop(f); + return 8; + } + bpf_obj_drop(f); + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 9; + } + + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 10; + } + return 0; +} + + +static __always_inline +int list_push_pop_multiple(struct bpf_spin_lock *lock, struct bpf_list_head *head, bool leave_in_map) +{ + struct bpf_list_node *n; + struct foo *f[8], *pf; + int i; + + for (i = 0; i < ARRAY_SIZE(f); i++) { + f[i] = bpf_obj_new(typeof(**f)); + if (!f[i]) + return 2; + f[i]->data = i; + bpf_spin_lock(lock); + bpf_list_push_front(head, &f[i]->node); + bpf_spin_unlock(lock); + } + + for (i = 0; i < ARRAY_SIZE(f); i++) { + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (!n) + return 3; + pf = container_of(n, struct foo, node); + if (pf->data != (ARRAY_SIZE(f) - i - 1)) { + bpf_obj_drop(pf); + return 4; + } + bpf_spin_lock(lock); + bpf_list_push_back(head, &pf->node); + bpf_spin_unlock(lock); + } + + if (leave_in_map) + return 0; + + for (i = 0; i < ARRAY_SIZE(f); i++) { + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (!n) + return 5; + pf = container_of(n, struct foo, node); + if (pf->data != i) { + bpf_obj_drop(pf); + return 6; + } + bpf_obj_drop(pf); + } + bpf_spin_lock(lock); + n = bpf_list_pop_back(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 7; + } + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (n) { + bpf_obj_drop(container_of(n, struct foo, node)); + return 8; + } + return 0; +} + +static __always_inline +int list_in_list(struct bpf_spin_lock *lock, struct bpf_list_head *head, bool leave_in_map) +{ + struct bpf_list_node *n; + struct bar *ba[8], *b; + struct foo *f; + int i; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 2; + for (i = 0; i < ARRAY_SIZE(ba); i++) { + b = bpf_obj_new(typeof(*b)); + if (!b) { + bpf_obj_drop(f); + return 3; + } + b->data = i; + bpf_spin_lock(&f->lock); + bpf_list_push_back(&f->head, &b->node); + bpf_spin_unlock(&f->lock); + } + + bpf_spin_lock(lock); + f->data = 42; + bpf_list_push_front(head, &f->node); + bpf_spin_unlock(lock); + + if (leave_in_map) + return 0; + + bpf_spin_lock(lock); + n = bpf_list_pop_front(head); + bpf_spin_unlock(lock); + if (!n) + return 4; + f = container_of(n, struct foo, node); + if (f->data != 42) { + bpf_obj_drop(f); + return 5; + } + + for (i = 0; i < ARRAY_SIZE(ba); i++) { + bpf_spin_lock(&f->lock); + n = bpf_list_pop_front(&f->head); + bpf_spin_unlock(&f->lock); + if (!n) { + bpf_obj_drop(f); + return 6; + } + b = container_of(n, struct bar, node); + if (b->data != i) { + bpf_obj_drop(f); + bpf_obj_drop(b); + return 7; + } + bpf_obj_drop(b); + } + bpf_spin_lock(&f->lock); + n = bpf_list_pop_front(&f->head); + bpf_spin_unlock(&f->lock); + if (n) { + bpf_obj_drop(f); + bpf_obj_drop(container_of(n, struct bar, node)); + return 8; + } + bpf_obj_drop(f); + return 0; +} + +static __always_inline +int test_list_push_pop(struct bpf_spin_lock *lock, struct bpf_list_head *head) +{ + int ret; + + ret = list_push_pop(lock, head, false); + if (ret) + return ret; + return list_push_pop(lock, head, true); +} + +static __always_inline +int test_list_push_pop_multiple(struct bpf_spin_lock *lock, struct bpf_list_head *head) +{ + int ret; + + ret = list_push_pop_multiple(lock ,head, false); + if (ret) + return ret; + return list_push_pop_multiple(lock, head, true); +} + +static __always_inline +int test_list_in_list(struct bpf_spin_lock *lock, struct bpf_list_head *head) +{ + int ret; + + ret = list_in_list(lock, head, false); + if (ret) + return ret; + return list_in_list(lock, head, true); +} + +SEC("tc") +int map_list_push_pop(void *ctx) +{ + struct map_value *v; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop(&v->lock, &v->head); +} + +SEC("tc") +int inner_map_list_push_pop(void *ctx) +{ + struct map_value *v; + void *map; + + map = bpf_map_lookup_elem(&map_of_maps, &(int){0}); + if (!map) + return 1; + v = bpf_map_lookup_elem(map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop(&v->lock, &v->head); +} + +SEC("tc") +int global_list_push_pop(void *ctx) +{ + return test_list_push_pop(&glock, &ghead); +} + +SEC("tc") +int map_list_push_pop_multiple(void *ctx) +{ + struct map_value *v; + int ret; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop_multiple(&v->lock, &v->head); +} + +SEC("tc") +int inner_map_list_push_pop_multiple(void *ctx) +{ + struct map_value *v; + void *map; + int ret; + + map = bpf_map_lookup_elem(&map_of_maps, &(int){0}); + if (!map) + return 1; + v = bpf_map_lookup_elem(map, &(int){0}); + if (!v) + return 1; + return test_list_push_pop_multiple(&v->lock, &v->head); +} + +SEC("tc") +int global_list_push_pop_multiple(void *ctx) +{ + int ret; + + ret = list_push_pop_multiple(&glock, &ghead, false); + if (ret) + return ret; + return list_push_pop_multiple(&glock, &ghead, true); +} + +SEC("tc") +int map_list_in_list(void *ctx) +{ + struct map_value *v; + int ret; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return 1; + return test_list_in_list(&v->lock, &v->head); +} + +SEC("tc") +int inner_map_list_in_list(void *ctx) +{ + struct map_value *v; + void *map; + int ret; + + map = bpf_map_lookup_elem(&map_of_maps, &(int){0}); + if (!map) + return 1; + v = bpf_map_lookup_elem(map, &(int){0}); + if (!v) + return 1; + return test_list_in_list(&v->lock, &v->head); +} + +SEC("tc") +int global_list_in_list(void *ctx) +{ + return test_list_in_list(&glock, &ghead); +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/linked_list.h b/tools/testing/selftests/bpf/progs/linked_list.h new file mode 100644 index 000000000000..8db80ed64db1 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/linked_list.h @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0 +#ifndef LINKED_LIST_H +#define LINKED_LIST_H + +#include +#include +#include "bpf_experimental.h" + +struct bar { + struct bpf_list_node node; + int data; +}; + +struct foo { + struct bpf_list_node node; + struct bpf_list_head head __contains(bar, node); + struct bpf_spin_lock lock; + int data; + struct bpf_list_node node2; +}; + +struct map_value { + struct bpf_spin_lock lock; + int data; + struct bpf_list_head head __contains(foo, node); +}; + +struct array_map { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct map_value); + __uint(max_entries, 1); +}; + +struct array_map array_map SEC(".maps"); +struct array_map inner_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS); + __uint(max_entries, 1); + __type(key, int); + __type(value, int); + __array(values, struct array_map); +} map_of_maps SEC(".maps") = { + .values = { + [0] = &inner_map, + }, +}; + +#define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8))) + +private(A) struct bpf_spin_lock glock; +private(A) struct bpf_list_head ghead __contains(foo, node); +private(B) struct bpf_spin_lock glock2; + +#endif diff --git a/tools/testing/selftests/bpf/progs/linked_list_fail.c b/tools/testing/selftests/bpf/progs/linked_list_fail.c new file mode 100644 index 000000000000..1d9017240e19 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/linked_list_fail.c @@ -0,0 +1,581 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include "bpf_experimental.h" + +#include "linked_list.h" + +#define INIT \ + struct map_value *v, *v2, *iv, *iv2; \ + struct foo *f, *f1, *f2; \ + struct bar *b; \ + void *map; \ + \ + map = bpf_map_lookup_elem(&map_of_maps, &(int){ 0 }); \ + if (!map) \ + return 0; \ + v = bpf_map_lookup_elem(&array_map, &(int){ 0 }); \ + if (!v) \ + return 0; \ + v2 = bpf_map_lookup_elem(&array_map, &(int){ 0 }); \ + if (!v2) \ + return 0; \ + iv = bpf_map_lookup_elem(map, &(int){ 0 }); \ + if (!iv) \ + return 0; \ + iv2 = bpf_map_lookup_elem(map, &(int){ 0 }); \ + if (!iv2) \ + return 0; \ + f = bpf_obj_new(typeof(*f)); \ + if (!f) \ + return 0; \ + f1 = f; \ + f2 = bpf_obj_new(typeof(*f2)); \ + if (!f2) { \ + bpf_obj_drop(f1); \ + return 0; \ + } \ + b = bpf_obj_new(typeof(*b)); \ + if (!b) { \ + bpf_obj_drop(f2); \ + bpf_obj_drop(f1); \ + return 0; \ + } + +#define CHECK(test, op, hexpr) \ + SEC("?tc") \ + int test##_missing_lock_##op(void *ctx) \ + { \ + INIT; \ + void (*p)(void *) = (void *)&bpf_list_##op; \ + p(hexpr); \ + return 0; \ + } + +CHECK(kptr, push_front, &f->head); +CHECK(kptr, push_back, &f->head); +CHECK(kptr, pop_front, &f->head); +CHECK(kptr, pop_back, &f->head); + +CHECK(global, push_front, &ghead); +CHECK(global, push_back, &ghead); +CHECK(global, pop_front, &ghead); +CHECK(global, pop_back, &ghead); + +CHECK(map, push_front, &v->head); +CHECK(map, push_back, &v->head); +CHECK(map, pop_front, &v->head); +CHECK(map, pop_back, &v->head); + +CHECK(inner_map, push_front, &iv->head); +CHECK(inner_map, push_back, &iv->head); +CHECK(inner_map, pop_front, &iv->head); +CHECK(inner_map, pop_back, &iv->head); + +#undef CHECK + +#define CHECK(test, op, lexpr, hexpr) \ + SEC("?tc") \ + int test##_incorrect_lock_##op(void *ctx) \ + { \ + INIT; \ + void (*p)(void *) = (void *)&bpf_list_##op; \ + bpf_spin_lock(lexpr); \ + p(hexpr); \ + return 0; \ + } + +#define CHECK_OP(op) \ + CHECK(kptr_kptr, op, &f1->lock, &f2->head); \ + CHECK(kptr_global, op, &f1->lock, &ghead); \ + CHECK(kptr_map, op, &f1->lock, &v->head); \ + CHECK(kptr_inner_map, op, &f1->lock, &iv->head); \ + \ + CHECK(global_global, op, &glock2, &ghead); \ + CHECK(global_kptr, op, &glock, &f1->head); \ + CHECK(global_map, op, &glock, &v->head); \ + CHECK(global_inner_map, op, &glock, &iv->head); \ + \ + CHECK(map_map, op, &v->lock, &v2->head); \ + CHECK(map_kptr, op, &v->lock, &f2->head); \ + CHECK(map_global, op, &v->lock, &ghead); \ + CHECK(map_inner_map, op, &v->lock, &iv->head); \ + \ + CHECK(inner_map_inner_map, op, &iv->lock, &iv2->head); \ + CHECK(inner_map_kptr, op, &iv->lock, &f2->head); \ + CHECK(inner_map_global, op, &iv->lock, &ghead); \ + CHECK(inner_map_map, op, &iv->lock, &v->head); + +CHECK_OP(push_front); +CHECK_OP(push_back); +CHECK_OP(pop_front); +CHECK_OP(pop_back); + +#undef CHECK +#undef CHECK_OP +#undef INIT + +SEC("?kprobe/xyz") +int map_compat_kprobe(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?kretprobe/xyz") +int map_compat_kretprobe(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?tracepoint/xyz") +int map_compat_tp(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?perf_event") +int map_compat_perf(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?raw_tp/xyz") +int map_compat_raw_tp(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?raw_tp.w/xyz") +int map_compat_raw_tp_w(void *ctx) +{ + bpf_list_push_front(&ghead, NULL); + return 0; +} + +SEC("?tc") +int obj_type_id_oor(void *ctx) +{ + bpf_obj_new_impl(~0UL, NULL); + return 0; +} + +SEC("?tc") +int obj_new_no_composite(void *ctx) +{ + bpf_obj_new_impl(bpf_core_type_id_local(int), (void *)42); + return 0; +} + +SEC("?tc") +int obj_new_no_struct(void *ctx) +{ + + bpf_obj_new(union { int data; unsigned udata; }); + return 0; +} + +SEC("?tc") +int obj_drop_non_zero_off(void *ctx) +{ + void *f; + + f = bpf_obj_new(struct foo); + if (!f) + return 0; + bpf_obj_drop(f+1); + return 0; +} + +SEC("?tc") +int new_null_ret(void *ctx) +{ + return bpf_obj_new(struct foo)->data; +} + +SEC("?tc") +int obj_new_acq(void *ctx) +{ + bpf_obj_new(struct foo); + return 0; +} + +SEC("?tc") +int use_after_drop(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_obj_drop(f); + return f->data; +} + +SEC("?tc") +int ptr_walk_scalar(void *ctx) +{ + struct test1 { + struct test2 { + struct test2 *next; + } *ptr; + } *p; + + p = bpf_obj_new(typeof(*p)); + if (!p) + return 0; + bpf_this_cpu_ptr(p->ptr); + return 0; +} + +SEC("?tc") +int direct_read_lock(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + return *(int *)&f->lock; +} + +SEC("?tc") +int direct_write_lock(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + *(int *)&f->lock = 0; + return 0; +} + +SEC("?tc") +int direct_read_head(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + return *(int *)&f->head; +} + +SEC("?tc") +int direct_write_head(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + *(int *)&f->head = 0; + return 0; +} + +SEC("?tc") +int direct_read_node(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + return *(int *)&f->node; +} + +SEC("?tc") +int direct_write_node(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + *(int *)&f->node = 0; + return 0; +} + +static __always_inline +int write_after_op(void (*push_op)(void *head, void *node)) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + push_op(&ghead, &f->node); + f->data = 42; + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int write_after_push_front(void *ctx) +{ + return write_after_op((void *)bpf_list_push_front); +} + +SEC("?tc") +int write_after_push_back(void *ctx) +{ + return write_after_op((void *)bpf_list_push_back); +} + +static __always_inline +int use_after_unlock(void (*op)(void *head, void *node)) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + f->data = 42; + op(&ghead, &f->node); + bpf_spin_unlock(&glock); + + return f->data; +} + +SEC("?tc") +int use_after_unlock_push_front(void *ctx) +{ + return use_after_unlock((void *)bpf_list_push_front); +} + +SEC("?tc") +int use_after_unlock_push_back(void *ctx) +{ + return use_after_unlock((void *)bpf_list_push_back); +} + +static __always_inline +int list_double_add(void (*op)(void *head, void *node)) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + op(&ghead, &f->node); + op(&ghead, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int double_push_front(void *ctx) +{ + return list_double_add((void *)bpf_list_push_front); +} + +SEC("?tc") +int double_push_back(void *ctx) +{ + return list_double_add((void *)bpf_list_push_back); +} + +SEC("?tc") +int no_node_value_type(void *ctx) +{ + void *p; + + p = bpf_obj_new(struct { int data; }); + if (!p) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, p); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_value_type(void *ctx) +{ + struct bar *b; + + b = bpf_obj_new(typeof(*b)); + if (!b) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, &b->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_node_var_off(struct __sk_buff *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, (void *)&f->node + ctx->protocol); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_node_off1(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, (void *)&f->node + 1); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_node_off2(void *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(&ghead, &f->node2); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int no_head_type(void *ctx) +{ + void *p; + + p = bpf_obj_new(typeof(struct { int data; })); + if (!p) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front(p, NULL); + bpf_spin_lock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_head_var_off1(struct __sk_buff *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front((void *)&ghead + ctx->protocol, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_head_var_off2(struct __sk_buff *ctx) +{ + struct foo *f; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + bpf_spin_lock(&glock); + bpf_list_push_front((void *)&f->head + ctx->protocol, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +SEC("?tc") +int incorrect_head_off1(void *ctx) +{ + struct foo *f; + struct bar *b; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + b = bpf_obj_new(typeof(*b)); + if (!b) { + bpf_obj_drop(f); + return 0; + } + + bpf_spin_lock(&f->lock); + bpf_list_push_front((void *)&f->head + 1, &b->node); + bpf_spin_unlock(&f->lock); + + return 0; +} + +SEC("?tc") +int incorrect_head_off2(void *ctx) +{ + struct foo *f; + struct bar *b; + + f = bpf_obj_new(typeof(*f)); + if (!f) + return 0; + + bpf_spin_lock(&glock); + bpf_list_push_front((void *)&ghead + 1, &f->node); + bpf_spin_unlock(&glock); + + return 0; +} + +static __always_inline +int pop_ptr_off(void *(*op)(void *head)) +{ + struct { + struct bpf_list_head head __contains(foo, node2); + struct bpf_spin_lock lock; + } *p; + struct bpf_list_node *n; + + p = bpf_obj_new(typeof(*p)); + if (!p) + return 0; + bpf_spin_lock(&p->lock); + n = op(&p->head); + bpf_spin_unlock(&p->lock); + + bpf_this_cpu_ptr(n); + return 0; +} + +SEC("?tc") +int pop_front_off(void *ctx) +{ + return pop_ptr_off((void *)bpf_list_pop_front); +} + +SEC("?tc") +int pop_back_off(void *ctx) +{ + return pop_ptr_off((void *)bpf_list_pop_back); +} + +char _license[] SEC("license") = "GPL"; From patchwork Mon Nov 7 23:09:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 13035468 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D310AC433FE for ; Mon, 7 Nov 2022 23:11:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232910AbiKGXLb (ORCPT ); Mon, 7 Nov 2022 18:11:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232662AbiKGXLa (ORCPT ); Mon, 7 Nov 2022 18:11:30 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6220521255 for ; Mon, 7 Nov 2022 15:11:28 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id l22-20020a17090a3f1600b00212fbbcfb78so16245888pjc.3 for ; Mon, 07 Nov 2022 15:11:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tLVRV+t4+FZTDBLmAANvc6K1pqY5UOLPredtWxM9pUc=; b=WtjBh2ZkMYz0Y4crvDPekLsubj3rnxEKSu5nkzRsC9kZLEbSel0zJR8ADRrOioAP3m JFuEd9BFnwvy2Y8mHhNFK3AfcqD94/fsgn8zeJh0lmyp13QiVzciTkMnO/SZeOZjE6Tf eWtIuMkjx/jgti5T8uOdXzXiT6EFlO23epQpAULnSY29zjsLxBwukpN5Tx8CH1YsEbIC YjcWrLJ8gegyZQuobVVuiUmuZ/i1LzJfErJQOK8XO7E+S2JDEYq4LOLPDcgWFjw5EoRl HqSu+vLcz1p0Ap3sKmTdiHl3lAAyGkhDkxaS/kAU4+Xv7nbVzTmEgz4tweZH0zVKPyXs qjmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tLVRV+t4+FZTDBLmAANvc6K1pqY5UOLPredtWxM9pUc=; b=KmAirjMtyg7hFqY7kZDFRA4Xswrzk6rJH7Ey/tr+qOi6oG5HkGj0tYGA5tvenWCjy/ izP5cBOehPV+Ad2HfrrkgdVIiEQTOeTuf9g3TmeXO6YUxAzNNAMVmSrsmvsnDg9POkyz 31Kn1oQI0e1Wm/fxPOXoZPydH2H/hXZtt9qMqmM8um6HaDrzqYjPOgzdc5kP3lCLxLfw Uv+kJd6G1OgNVdTK+FJk+f64HJRuAcmnv9TL4pYuJ4Rzq8XIVloYW4nw41KAgPbSTwO2 NThW3OcP8Ie3knpWh/dI1rIskUHGAnH/N75y78j69uzMafc57y6no/c44Y/ooGwXOSYc hkww== X-Gm-Message-State: ACrzQf0ZAHX//yfu00R0F6+c0tVEiRNm1ZeE2qceWsy0tI8h4V5GcxCp saWx51lyzkWay+CtW6+XhJeVwLXm5ShU2g== X-Google-Smtp-Source: AMsMyM5G/DTB58qSOzcXCsQSd3jucCqFyfkEqQKtEswCwvg2sRja2jpsG+9J5UHSoqJ0Q26VKqt/Uw== X-Received: by 2002:a17:90a:ad08:b0:212:d5f1:e0c6 with SMTP id r8-20020a17090aad0800b00212d5f1e0c6mr69512916pjq.228.1667862687648; Mon, 07 Nov 2022 15:11:27 -0800 (PST) Received: from localhost ([14.96.13.220]) by smtp.gmail.com with ESMTPSA id l16-20020a170903245000b001885041d7b8sm5471215pls.293.2022.11.07.15.11.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 15:11:27 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , Martin KaFai Lau , Dave Marchevsky , Delyan Kratunov Subject: [PATCH bpf-next v5 25/25] selftests/bpf: Add BTF sanity tests Date: Tue, 8 Nov 2022 04:39:50 +0530 Message-Id: <20221107230950.7117-26-memxor@gmail.com> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221107230950.7117-1-memxor@gmail.com> References: <20221107230950.7117-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10933; i=memxor@gmail.com; h=from:subject; bh=kzEurx3V6kdGvcxcROAKf2a3IYFN9EHUhuRar6z8164=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBjaY+4iLkGwa9CKLBhnNZ44dDjeB0SYikZnv4e52zl 4M+bJaOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCY2mPuAAKCRBM4MiGSL8RymQjD/ 9WnGsx9IKSHz+XfLYKGQWd2MYvCn1vnWlPyK8JJlaWJP7zDy2FICxjvLBc5sWOiuX1RoUlo8myclYv xClIi7uEhL5wRgaNx15th8J4S//CKoDE8ChINLzjzEt/i5EpFN9Olnhe8r3vxh1HUSXH7bp7iMgVkE XXcT8sLKu6Jd5eS/QLU0TFHo8jiasUZlqApo0rBaN1gUAFAC7etPjOTRUGeq/HE/UcHvq5g29bgnl8 sEp4YDcTE/NghKu9xr/uMWB6Ho+c0z03t3RvjX7xvO8lrkwiOLZ9tUWOpVCDLFDWQOZYEnzWlReqhU JuPrUGlwrVv2M1vjgU1eRhUX8gAYrwvQv6Nr73bT9i2/NH8EgW6kdlHNQ3bii28Zuk0pmadBIE+9v9 AIaApFoQ5ru+x9v5a3j0oDSVbUEEdwIP/7FKxchzw6EXtQQkktzNrrfp3RlSMl0cBrqWCK5m1L7Rlb 9YURLUrEfeV8vTB9N5cuPez2PUx3UJMmsDpH+1WehHkw5DEdC2NzdY/Mj/V2S2pppxWPr74iq6xEwL ljyr3pRyOFnfp9sya9GoxyohywlLwzFv5abyfyoEj76dMTzpfwFEkn/Zkc8OaR4IxzOsWxmTHbdok6 0M+thYaaV40Ln9P8v/eCoVYo1k+/ZMdnyki4fHEKdMulPfujUFLrjXDTdVBg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Preparing the metadata for bpf_list_head involves a complicated parsing step and type resolution for the contained value. Ensure that corner cases are tested against and invalid specifications in source are duly rejected. Also include tests for incorrect ownership relationships in the BTF. Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/linked_list.c | 271 ++++++++++++++++++ 1 file changed, 271 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/linked_list.c b/tools/testing/selftests/bpf/prog_tests/linked_list.c index 669ef4bb9b87..40070e2d22f2 100644 --- a/tools/testing/selftests/bpf/prog_tests/linked_list.c +++ b/tools/testing/selftests/bpf/prog_tests/linked_list.c @@ -1,4 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 +#include +#include +#include #include #include @@ -233,6 +236,273 @@ static void test_linked_list_success(int mode, bool leave_in_map) linked_list__destroy(skel); } +/* struct bpf_spin_lock { + * int foo; + * }; + * struct bpf_list_head { + * __u64 :64; + * __u64 :64; + * } __attribute__((aligned(8))); + * struct bpf_list_node { + * __u64 :64; + * __u64 :64; + * } __attribute__((aligned(8))); + */ +static const char btf_str_sec[] = "\0bpf_spin_lock\0bpf_list_head\0bpf_list_node\0foo\0bar\0baz" + "\0contains:foo:foo\0contains:bar:bar\0contains:baz:baz\0bam" + "\0contains:bam:bam"; + +#define INIT_BTF_TILL_4 \ + /* int */ \ + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ \ + /* struct bpf_spin_lock */ /* [2] */ \ + BTF_TYPE_ENC(1, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 4), \ + BTF_MEMBER_ENC(43, 1, 0), \ + /* struct bpf_list_head */ /* [3] */ \ + BTF_TYPE_ENC(15, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 0), 16), \ + /* struct bpf_list_node */ /* [4] */ \ + BTF_TYPE_ENC(29, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 0), 16), + +static void check_btf(u32 *types, u32 types_len, int error) +{ + LIBBPF_OPTS(bpf_btf_load_opts, opts, + .log_buf = log_buf, + .log_size = sizeof(log_buf), + ); + struct btf_header hdr = { + .magic = BTF_MAGIC, + .version = BTF_VERSION, + .hdr_len = sizeof(struct btf_header), + .type_len = types_len, + .str_off = types_len, + .str_len = sizeof(btf_str_sec), + }; + void *ptr, *raw_btf; + int fd, ret; + + raw_btf = malloc(sizeof(hdr) + hdr.type_len + hdr.str_len); + if (!ASSERT_OK_PTR(raw_btf, "malloc(raw_btf)")) + return; + + ptr = raw_btf; + memcpy(ptr, &hdr, sizeof(hdr)); + ptr += sizeof(hdr); + memcpy(ptr, types, hdr.type_len); + ptr += hdr.type_len; + memcpy(ptr, btf_str_sec, hdr.str_len); + ptr += hdr.str_len; + + fd = bpf_btf_load(raw_btf, ptr - raw_btf, &opts); + ret = fd < 0 ? -errno : 0; + if (fd >= 0) + close(fd); + if (error) + ASSERT_LT(fd, 0, "bpf_btf_load"); + else + ASSERT_GE(fd, 0, "bpf_btf_load"); + if (!ASSERT_EQ(ret, error, "-errno == error")) + printf("BTF Log:\n%s\n", log_buf); + free(raw_btf); + return; +} + +#define SPIN_LOCK 2 +#define LIST_HEAD 3 +#define LIST_NODE 4 +#define FOO 43 +#define BAR 47 +#define BAZ 51 +#define BAM 106 +#define CONT_FOO_FOO 55 +#define CONT_BAR_BAR 72 +#define CONT_BAZ_BAZ 89 +#define CONT_BAM_BAM 110 + +static void test_btf(void) +{ + if (test__start_subtest("btf: too many locks")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 24), /* [5] */ + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 0), + BTF_MEMBER_ENC(FOO, SPIN_LOCK, 32), + BTF_MEMBER_ENC(BAR, LIST_HEAD, 64), + }; + check_btf(types, sizeof(types), -E2BIG); + } + if (test__start_subtest("btf: missing lock")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 16), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_DECL_TAG_ENC(CONT_BAZ_BAZ, 5, 0), + BTF_TYPE_ENC(BAZ, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 16), + BTF_MEMBER_ENC(BAZ, LIST_NODE, 0), + }; + check_btf(types, sizeof(types), -EINVAL); + } + if (test__start_subtest("btf: bad offset")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(FOO, LIST_NODE, 0), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 0), + BTF_DECL_TAG_ENC(CONT_FOO_FOO, 5, 0), + }; + check_btf(types, sizeof(types), -EFAULT); + } + if (test__start_subtest("btf: missing contains:")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 24), /* [5] */ + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 0), + BTF_MEMBER_ENC(BAR, LIST_HEAD, 64), + }; + check_btf(types, sizeof(types), -EINVAL); + } + if (test__start_subtest("btf: missing struct")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 24), /* [5] */ + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 0), + BTF_MEMBER_ENC(BAR, LIST_HEAD, 64), + BTF_DECL_TAG_ENC(CONT_BAR_BAR, 5, 1), + }; + check_btf(types, sizeof(types), -ENOENT); + } + if (test__start_subtest("btf: missing node")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 24), /* [5] */ + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 0), + BTF_MEMBER_ENC(BAR, LIST_HEAD, 64), + BTF_DECL_TAG_ENC(CONT_FOO_FOO, 5, 1), + }; + check_btf(types, sizeof(types), -ENOENT); + } + if (test__start_subtest("btf: node incorrect type")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 20), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAR, SPIN_LOCK, 128), + BTF_DECL_TAG_ENC(CONT_BAZ_BAZ, 5, 0), + BTF_TYPE_ENC(BAZ, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 4), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 0), + }; + check_btf(types, sizeof(types), -EINVAL); + } + if (test__start_subtest("btf: multiple bpf_list_node with name foo")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 52), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(FOO, LIST_NODE, 128), + BTF_MEMBER_ENC(FOO, LIST_NODE, 256), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 384), + BTF_DECL_TAG_ENC(CONT_FOO_FOO, 5, 0), + }; + check_btf(types, sizeof(types), -EINVAL); + } + if (test__start_subtest("btf: owning | owned AA cycle")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(FOO, LIST_NODE, 128), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_FOO_FOO, 5, 0), + }; + check_btf(types, sizeof(types), -ELOOP); + } + if (test__start_subtest("btf: owning | owned ABA cycle")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(FOO, LIST_NODE, 128), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_BAR_BAR, 5, 0), /* [6] */ + BTF_TYPE_ENC(BAR, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [7] */ + BTF_MEMBER_ENC(FOO, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAR, LIST_NODE, 128), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_FOO_FOO, 7, 0), + }; + check_btf(types, sizeof(types), -ELOOP); + } + if (test__start_subtest("btf: owning -> owned")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 20), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAR, SPIN_LOCK, 128), + BTF_DECL_TAG_ENC(CONT_BAZ_BAZ, 5, 0), + BTF_TYPE_ENC(BAZ, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 16), + BTF_MEMBER_ENC(BAZ, LIST_NODE, 0), + }; + check_btf(types, sizeof(types), 0); + } + if (test__start_subtest("btf: owning -> owning | owned -> owned")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 20), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 128), + BTF_DECL_TAG_ENC(CONT_BAR_BAR, 5, 0), /* [6] */ + BTF_TYPE_ENC(BAR, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [7] */ + BTF_MEMBER_ENC(FOO, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAR, LIST_NODE, 128), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_BAZ_BAZ, 7, 0), + BTF_TYPE_ENC(BAZ, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 16), + BTF_MEMBER_ENC(BAZ, LIST_NODE, 0), + }; + check_btf(types, sizeof(types), 0); + } + if (test__start_subtest("btf: owning | owned -> owning | owned -> owned")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(FOO, LIST_NODE, 128), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_BAR_BAR, 5, 0), /* [6] */ + BTF_TYPE_ENC(BAR, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [7] */ + BTF_MEMBER_ENC(FOO, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAR, LIST_NODE, 128), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_BAZ_BAZ, 7, 0), + BTF_TYPE_ENC(BAZ, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 16), + BTF_MEMBER_ENC(BAZ, LIST_NODE, 0), + }; + check_btf(types, sizeof(types), -ELOOP); + } + if (test__start_subtest("btf: owning -> owning | owned -> owning | owned -> owned")) { + u32 types[] = { + INIT_BTF_TILL_4 + BTF_TYPE_ENC(FOO, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 20), /* [5] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 128), + BTF_DECL_TAG_ENC(CONT_BAR_BAR, 5, 0), /* [6] */ + BTF_TYPE_ENC(BAR, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [7] */ + BTF_MEMBER_ENC(FOO, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAR, LIST_NODE, 128), + BTF_MEMBER_ENC(BAZ, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_BAZ_BAZ, 7, 0), + BTF_TYPE_ENC(BAZ, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 36), /* [9] */ + BTF_MEMBER_ENC(BAR, LIST_HEAD, 0), + BTF_MEMBER_ENC(BAZ, LIST_NODE, 128), + BTF_MEMBER_ENC(FOO, SPIN_LOCK, 256), + BTF_DECL_TAG_ENC(CONT_BAM_BAM, 9, 0), + BTF_TYPE_ENC(BAM, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 16), + BTF_MEMBER_ENC(BAM, LIST_NODE, 0), + }; + check_btf(types, sizeof(types), -ELOOP); + } +} + void test_linked_list(void) { int i; @@ -243,6 +513,7 @@ void test_linked_list(void) test_linked_list_fail_prog(linked_list_fail_tests[i].prog_name, linked_list_fail_tests[i].err_msg); } + test_btf(); test_linked_list_success(PUSH_POP, false); test_linked_list_success(PUSH_POP, true); test_linked_list_success(PUSH_POP_MULT, false);