From patchwork Tue Aug 13 21:24:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13762587 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yb1-f178.google.com (mail-yb1-f178.google.com [209.85.219.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A289443155 for ; Tue, 13 Aug 2024 21:24:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584274; cv=none; b=uNvO4sXc2jBrPy/axgkVIw9qORI2y6zQSspdO3ooRZma+IVNzexlyCQjMQvZuxtXtEGz8UecfN8kEQj7WTsHsMuJYAQBKbiAAFp6N+lju/xVdhwJXbD5+YsqsOdPafm79SHlBnuk8FgOoTquGH58vw6eMTRtDX6H7iEsLW5Kx+Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584274; c=relaxed/simple; bh=jMdhYVwjomP9CrFL2w+7rS734zB9bEh9rnY1Yd9RqFw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BjVQKaS4X49xO2VgbQB6rD/IgE+JlnHQfl0wyq2x8YJ3hgo0+yv0baxgjUbzvANUldVEcCphRys6XQ0R81Q+ROplbLGa3v4TdTFDT2Ka+pisvz6kfcJAA9ZQYT9rudJfiWeR89sVs4t2DC+dwFEzwmb79ABOnphl9EgEah+7S2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=O/UOcICH; arc=none smtp.client-ip=209.85.219.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="O/UOcICH" Received: by mail-yb1-f178.google.com with SMTP id 3f1490d57ef6-e0e88873825so5859099276.2 for ; Tue, 13 Aug 2024 14:24:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1723584271; x=1724189071; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dWxNuRa0wj8S8v/LOzoSw3WgU9BkUVUiTpq+NL0RSZI=; b=O/UOcICHTYHJwXeFx3AmivdvPYd4oRQvcVFRwoG9to3shU+haJElYgtsFMzup7fSTp VGXyxjNYnAhxcjI835ds8hvbU/ihmw9GZSFMIpURxEkEKVmfcwYUDZ55LoNqMHsp6bJT 0w6rrOGRg8h4CJIpy0evp8fqQ4uQMVzSojwIH17wtZXK/PaLkjOI40mYNYCYrl8DpfMk ebVeKThLSmHsTIsr98/CLtZWW0wBk+88mX7edX4PqQLZXqLgr7uZSiUst+Qi4Ftb1RYg ze2y6Sn+MfUXxVXPSWmKwjQJsNXO18HmffdnPMeIRGMGtQ8RQVOO23tzzg6RJz4OxvOA skdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723584271; x=1724189071; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dWxNuRa0wj8S8v/LOzoSw3WgU9BkUVUiTpq+NL0RSZI=; b=Figm+EbBtOiBdhDeGO296pyJKaR7f1sXlaMDPanLyT4ZmcTwlt4T+PZfoMEfI7PszU 9Fwi9Fr9tp7YyXgXkshMPi3famkGZ06ul25/5OLr1lk1Cak/GsagyfllPGozVrjgC0V1 1f49SelqDmviN0LbFu3915P+cSus2e0p9vQ49aR9vCTEkbHp6K2XjlrJEQko0mgxU6Xr hWPRnY/5Q3MLV3r/nMLzVTnQ2mQz8rsMa8OAc7moEgIQRpmhcVd0gs3RfdzEEVSCjlfA dq/DKGsfor6snTMUK/5QK/glrBqv0x8HpqT/fiEkhLUhkekgmCF4NaF65Yxbug41hdzr V9oA== X-Gm-Message-State: AOJu0YyHvkh4BRFjrd+C3GkI2y9UrRyn3gm6F8TYF0n8ZdY4Ae9VFGko TGMtiiPIkqFYbZgVxDMWy5xQBAtePCi/RQQCvMxABDKFZrmxH9k1EZ+eMzNeS0IP/JeYVfb40cP 1ca4= X-Google-Smtp-Source: AGHT+IFVRgte7lcTqZ4sDEteGXsw9tThPddTYjz9atG4v5/olvXu6Fdby1Ey6rnP5w6gpNbskjMVRA== X-Received: by 2002:a05:6902:1881:b0:e0b:5b37:d0c9 with SMTP id 3f1490d57ef6-e1155ab1330mr1125143276.14.1723584271407; Tue, 13 Aug 2024 14:24:31 -0700 (PDT) Received: from n36-183-057.byted.org ([130.44.212.116]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6bf432fb61dsm21390786d6.52.2024.08.13.14.24.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 14:24:31 -0700 (PDT) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, houtao@huaweicloud.com, sinquersw@gmail.com, davemarchevsky@fb.com, ameryhung@gmail.com, Amery Hung , Hou Tao Subject: [PATCH v4 bpf-next 1/5] bpf: Let callers of btf_parse_kptr() track life cycle of prog btf Date: Tue, 13 Aug 2024 21:24:20 +0000 Message-Id: <20240813212424.2871455-2-amery.hung@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240813212424.2871455-1-amery.hung@bytedance.com> References: <20240813212424.2871455-1-amery.hung@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net btf_parse_kptr() and btf_record_free() do btf_get() and btf_put() respectively when working on btf_record in program and map if there are kptr fields. If the kptr is from program BTF, since both callers has already tracked the life cycle of program BTF, it is safe to remove the btf_get() and btf_put(). This change prevents memory leak of program BTF later when we start searching for kptr fields when building btf_record for program. It can happen when the btf fd is closed. The btf_put() corresponding to the btf_get() in btf_parse_kptr() was supposed to be called by btf_record_free() in btf_free_struct_meta_tab() in btf_free(). However, it will never happen since the invocation of btf_free() depends on the refcount of the btf to become 0 in the first place. Acked-by: Martin KaFai Lau Acked-by: Hou Tao Signed-off-by: Amery Hung --- kernel/bpf/btf.c | 2 +- kernel/bpf/syscall.c | 6 ++++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 95426d5b634e..deacf9d7b276 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3759,6 +3759,7 @@ static int btf_find_field(const struct btf *btf, const struct btf_type *t, return -EINVAL; } +/* Callers have to ensure the life cycle of btf if it is program BTF */ static int btf_parse_kptr(const struct btf *btf, struct btf_field *field, struct btf_field_info *info) { @@ -3787,7 +3788,6 @@ static int btf_parse_kptr(const struct btf *btf, struct btf_field *field, field->kptr.dtor = NULL; id = info->kptr.type_id; kptr_btf = (struct btf *)btf; - btf_get(kptr_btf); goto found_dtor; } if (id < 0) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 869265852d51..4003e1025264 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -550,7 +550,8 @@ void btf_record_free(struct btf_record *rec) case BPF_KPTR_PERCPU: if (rec->fields[i].kptr.module) module_put(rec->fields[i].kptr.module); - btf_put(rec->fields[i].kptr.btf); + if (btf_is_kernel(rec->fields[i].kptr.btf)) + btf_put(rec->fields[i].kptr.btf); break; case BPF_LIST_HEAD: case BPF_LIST_NODE: @@ -596,7 +597,8 @@ struct btf_record *btf_record_dup(const struct btf_record *rec) case BPF_KPTR_UNREF: case BPF_KPTR_REF: case BPF_KPTR_PERCPU: - btf_get(fields[i].kptr.btf); + if (btf_is_kernel(fields[i].kptr.btf)) + btf_get(fields[i].kptr.btf); if (fields[i].kptr.module && !try_module_get(fields[i].kptr.module)) { ret = -ENXIO; goto free; From patchwork Tue Aug 13 21:24:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13762588 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68BD96A8CF for ; Tue, 13 Aug 2024 21:24:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584275; cv=none; b=RBJR7rI9K6+BtG5Wpbk9oxzyvzrOoOYh92A6+g8oQkPJg/LMu1kZycRBjN6Sm0gb3g0GwqdzP2iRBuFbqBPP5RBL+SGiu8RvmVzHGBurhoPo8jHcDCKJcISFMC6xfHuujsCTxSX5JGbKfXreN6duWQD20M3/0wtoCoZ0cvirx+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584275; c=relaxed/simple; bh=QTQzYRe4h83jE6M5+olMf5O80fRE+6prUmVFhyU0eB8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MbjJRn/QQmrOINNEqVOIq7gzUQ4Y8bEDuDHN3EJRhaHCyM1ERPjGMVre6ZyuNg/dnhxYMho35F0Oh6DxSCujpjeZ+V/49//ln/0U1fh6eHLRGB1ZlHyB1sJmwb6khj780lJbot6aeP5RTxebUy82UWBUxUeVstlKr76EzNg9wn8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=jfig55Sn; arc=none smtp.client-ip=209.85.219.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="jfig55Sn" Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-6b7a36f26f3so2770096d6.1 for ; Tue, 13 Aug 2024 14:24:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1723584272; x=1724189072; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LGaWHW0YkJek/nHgtPyy2GmojNTQU/G0zc+vDmOrCsk=; b=jfig55SnGc5YkSQh1NOxzFioTGzWik0sbiFXro2KrSV1Mxu0h3eq8/ELelMmZCAiib VJBd484oa/Vl087Eox9dpLwST/iCLWSlH7/2IDmyQtahZUG0ruWfmSK6XN74vW5vqGAl 5FVA9WJijRKQWJNVCaNCb6m24YBvgBzCL5+0gfpU/MhDOGDuXwF4tISaSkN+70prqOp9 HygdKpAtemUJyiXxpzX+pOkomJwEvPGaSR+5hkI/Qp8q42T7PRdTSRWN0ebfC+gs27Zy ZsLlhAQGkIfZK8xd0MZxK2gbo8Mvr41DfamL1NjnWQHj4LwLSApL9oxoLJ5dIa43LzpQ t2pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723584272; x=1724189072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LGaWHW0YkJek/nHgtPyy2GmojNTQU/G0zc+vDmOrCsk=; b=HnqAzwvZqQSpRvGYPumGUx2iy5HqDTayVAThwkQfiR/W/xj4ZfWAPNCDZBbviX/xaI kYQxTSa0uNqViKnPfrqLUk8BIVYCnMKX1qGVEpZCXl1Gep6eHTTesvgkvqnxM59W+5FQ Q/J5WSkvneeyUMTppguYnS+xb/+KaMiIpr6Vf1yqRiwzrRNfkKMN8WUdbKUX1Z/2GUtG Cliw1o8EU5Erv7JUJI4TOi0lp4JDwWZBAFtwkcmVbj9LgTf8IDB+sGbJf0q07i6p2RZc SecumWnK4JnUmMWqThdaCsNwqibI5l4QZEItZaZGa62WsAXEiStUs44KfWkb8npeVW9P LFUA== X-Gm-Message-State: AOJu0Yx3ikS2uWzDS2KpteuRM9CiPjB1OJYDSeH8uYLWxqznxNQeMuub hKAPIgvhBrMi9Tmg4GPW6j8wVIRHcIgfPjHYCFvWBGkQjRMBnYwKs7mbbwKivY/CjRWNU4WACso G8EM= X-Google-Smtp-Source: AGHT+IHt7ZXVs2AH96FOh6Pke1sLaH3j+YnGUcjX0ZZovLLxnfo4LY6WW3btJo3QUhzWSA9VhEYc1w== X-Received: by 2002:a05:6214:3c9f:b0:6b5:d95c:692d with SMTP id 6a1803df08f44-6bf5f9c0984mr1111586d6.13.1723584272055; Tue, 13 Aug 2024 14:24:32 -0700 (PDT) Received: from n36-183-057.byted.org ([130.44.212.116]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6bf432fb61dsm21390786d6.52.2024.08.13.14.24.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 14:24:31 -0700 (PDT) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, houtao@huaweicloud.com, sinquersw@gmail.com, davemarchevsky@fb.com, ameryhung@gmail.com, Hou Tao , Amery Hung Subject: [PATCH v4 bpf-next 2/5] bpf: Search for kptrs in prog BTF structs Date: Tue, 13 Aug 2024 21:24:21 +0000 Message-Id: <20240813212424.2871455-3-amery.hung@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240813212424.2871455-1-amery.hung@bytedance.com> References: <20240813212424.2871455-1-amery.hung@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Dave Marchevsky Currently btf_parse_fields is used in two places to create struct btf_record's for structs: when looking at mapval type, and when looking at any struct in program BTF. The former looks for kptr fields while the latter does not. This patch modifies the btf_parse_fields call made when looking at prog BTF struct types to search for kptrs as well. Before this series there was no reason to search for kptrs in non-mapval types: a referenced kptr needs some owner to guarantee resource cleanup, and map values were the only owner that supported this. If a struct with a kptr field were to have some non-kptr-aware owner, the kptr field might not be properly cleaned up and result in resources leaking. Only searching for kptr fields in mapval was a simple way to avoid this problem. In practice, though, searching for BPF_KPTR when populating struct_meta_tab does not expose us to this risk, as struct_meta_tab is only accessed through btf_find_struct_meta helper, and that helper is only called in contexts where recognizing the kptr field is safe: * PTR_TO_BTF_ID reg w/ MEM_ALLOC flag * Such a reg is a local kptr and must be free'd via bpf_obj_drop, which will correctly handle kptr field * When handling specific kfuncs which either expect MEM_ALLOC input or return MEM_ALLOC output (obj_{new,drop}, percpu_obj_{new,drop}, list+rbtree funcs, refcount_acquire) * Will correctly handle kptr field for same reasons as above * When looking at kptr pointee type * Called by functions which implement "correct kptr resource handling" * In btf_check_and_fixup_fields * Helper that ensures no ownership loops for lists and rbtrees, doesn't care about kptr field existence So we should be able to find BPF_KPTR fields in all prog BTF structs without leaking resources. Further patches in the series will build on this change to support kptr_xchg into non-mapval local kptr. Without this change there would be no kptr field found in such a type. Acked-by: Martin KaFai Lau Acked-by: Hou Tao Signed-off-by: Dave Marchevsky Signed-off-by: Amery Hung --- kernel/bpf/btf.c | 70 +++++++++++++++++++++++++++++++++++------------- 1 file changed, 52 insertions(+), 18 deletions(-) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index deacf9d7b276..ed7758936424 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -5517,36 +5517,70 @@ static const char *alloc_obj_fields[] = { static struct btf_struct_metas * btf_parse_struct_metas(struct bpf_verifier_log *log, struct btf *btf) { - union { - struct btf_id_set set; - struct { - u32 _cnt; - u32 _ids[ARRAY_SIZE(alloc_obj_fields)]; - } _arr; - } aof; struct btf_struct_metas *tab = NULL; + struct btf_id_set *aof; int i, n, id, ret; BUILD_BUG_ON(offsetof(struct btf_id_set, cnt) != 0); BUILD_BUG_ON(sizeof(struct btf_id_set) != sizeof(u32)); - memset(&aof, 0, sizeof(aof)); + aof = kmalloc(sizeof(*aof), GFP_KERNEL | __GFP_NOWARN); + if (!aof) + return ERR_PTR(-ENOMEM); + aof->cnt = 0; + for (i = 0; i < ARRAY_SIZE(alloc_obj_fields); i++) { /* Try to find whether this special type exists in user BTF, and * if so remember its ID so we can easily find it among members * of structs that we iterate in the next loop. */ + struct btf_id_set *new_aof; + id = btf_find_by_name_kind(btf, alloc_obj_fields[i], BTF_KIND_STRUCT); if (id < 0) continue; - aof.set.ids[aof.set.cnt++] = id; + + new_aof = krealloc(aof, offsetof(struct btf_id_set, ids[aof->cnt + 1]), + GFP_KERNEL | __GFP_NOWARN); + if (!new_aof) { + ret = -ENOMEM; + goto free_aof; + } + aof = new_aof; + aof->ids[aof->cnt++] = id; + } + + n = btf_nr_types(btf); + for (i = 1; i < n; i++) { + /* Try to find if there are kptrs in user BTF and remember their ID */ + struct btf_id_set *new_aof; + struct btf_field_info tmp; + const struct btf_type *t; + + t = btf_type_by_id(btf, i); + if (!t) { + ret = -EINVAL; + goto free_aof; + } + + ret = btf_find_kptr(btf, t, 0, 0, &tmp); + if (ret != BTF_FIELD_FOUND) + continue; + + new_aof = krealloc(aof, offsetof(struct btf_id_set, ids[aof->cnt + 1]), + GFP_KERNEL | __GFP_NOWARN); + if (!new_aof) { + ret = -ENOMEM; + goto free_aof; + } + aof = new_aof; + aof->ids[aof->cnt++] = i; } - if (!aof.set.cnt) + if (!aof->cnt) return NULL; - sort(&aof.set.ids, aof.set.cnt, sizeof(aof.set.ids[0]), btf_id_cmp_func, NULL); + sort(&aof->ids, aof->cnt, sizeof(aof->ids[0]), btf_id_cmp_func, NULL); - n = btf_nr_types(btf); for (i = 1; i < n; i++) { struct btf_struct_metas *new_tab; const struct btf_member *member; @@ -5556,17 +5590,13 @@ btf_parse_struct_metas(struct bpf_verifier_log *log, struct btf *btf) int j, tab_cnt; t = btf_type_by_id(btf, i); - if (!t) { - ret = -EINVAL; - goto free; - } if (!__btf_type_is_struct(t)) continue; cond_resched(); for_each_member(j, t, member) { - if (btf_id_set_contains(&aof.set, member->type)) + if (btf_id_set_contains(aof, member->type)) goto parse; } continue; @@ -5585,7 +5615,8 @@ btf_parse_struct_metas(struct bpf_verifier_log *log, struct btf *btf) type = &tab->types[tab->cnt]; type->btf_id = i; record = btf_parse_fields(btf, t, BPF_SPIN_LOCK | BPF_LIST_HEAD | BPF_LIST_NODE | - BPF_RB_ROOT | BPF_RB_NODE | BPF_REFCOUNT, t->size); + BPF_RB_ROOT | BPF_RB_NODE | BPF_REFCOUNT | + BPF_KPTR, t->size); /* The record cannot be unset, treat it as an error if so */ if (IS_ERR_OR_NULL(record)) { ret = PTR_ERR_OR_ZERO(record) ?: -EFAULT; @@ -5594,9 +5625,12 @@ btf_parse_struct_metas(struct bpf_verifier_log *log, struct btf *btf) type->record = record; tab->cnt++; } + kfree(aof); return tab; free: btf_struct_metas_free(tab); +free_aof: + kfree(aof); return ERR_PTR(ret); } From patchwork Tue Aug 13 21:24:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13762589 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yb1-f174.google.com (mail-yb1-f174.google.com [209.85.219.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F0281A76CD for ; Tue, 13 Aug 2024 21:24:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584275; cv=none; b=Mwn4/U89/2eMCUs5fSijZYNXIo9Az7fumDebFsaQ+K6JlLfsr0cb+KRJEX2MqGh2qUQdwop5Hnb7eemM/E+GvHbBypGf9vF3Vq9pixFIpbF25oEllTOyhLPaKfJ+MTbj9Clo8k3YYp1n3RiiwJ74083MJxPskaV+CkQrnJnguSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584275; c=relaxed/simple; bh=CBTN153jJzwjME+dO9P/o9STqxs6koh4Yk5fYysLgEo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R3Xoaj8kIeiNCUaK04M5pY6m8yWSLRjW/+vDuk1pysNmHXpeHv8JIt3m/efvsgGQESLbS0N7RBP/r1XpEg4Xwlyryc9looKYkPZNwbDcoAMCGUK0uh+I7Hyaj2nwSWMHehALkGn446C/cRI0QcVAdUMTFy6T3bP721MT/asw4t8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=X06mwvHs; arc=none smtp.client-ip=209.85.219.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="X06mwvHs" Received: by mail-yb1-f174.google.com with SMTP id 3f1490d57ef6-e0bfa541c05so5234040276.0 for ; Tue, 13 Aug 2024 14:24:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1723584273; x=1724189073; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q6p8c1OUEhv+/PTai01JOAF17KOu2jscbjKAA2ruSl0=; b=X06mwvHs7t7ucHp7jVdyRvWt1GsI7n6tf8DqCXRQpGG4in5VufC4lhBx5TTK5QlKIh jNIweqZfBdmxvANo15FlqMruRCPkfoyB0ADOA1Sx5WgUqqpu757ywCML8GOJBJromT5i mV4x90qWuTIhoBjNQnDoJeGaLQEb1HW1/EhWb3MvC/qWIM6nj86FWvXbkGoBK49SNSwS 7K+WMe06bgFTZV9kVGsaHAJPRmCsonm9Ccw4/2gQo2G92I28zogGsaRHrgVnCnvyNuFh BHQrE2MhFR+wqFA1lVTjo4VxnVwLnxWEHjuOf+5RJifwjU0DlhXZq71JN7goRMcRkKMK k0Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723584273; x=1724189073; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q6p8c1OUEhv+/PTai01JOAF17KOu2jscbjKAA2ruSl0=; b=dpbotxVXBzDMSLjGE2/7Q6+qBqX6Vsb+YZ52Oxd2lWSDeYcRCy+nlIE8gHN4Oz5eG+ TXN+DpHuQZGt76sBk+OeYY2WZ8sjwTqSmGEi/PhhPjyK/AuCWApjpswVzmUcmX2I3Yq+ HiAjNolAfItlZvpCoaGZkCbnntcDwKwWCboJjY/EG7+oZBxQrdHb7LPwbZ+RHSLKqNSm pGLn3czv2xPmkap0n+Dn+cTQM9OcOJGUtU4qgqblrdP6HRfRsAlFCniLi7YTxycRALyV 8FRpNusdsx5p5IWo9COqPMf5Qlm/N+laaPCfOGhDibakfDD/QrzXx9dMQl3jjgdtcjrM pmfw== X-Gm-Message-State: AOJu0YwifX3xEuFemAp1Xh9AdxUGo3DDkszJK3UvaqkW7C1IQYbRmKan A3rbLF2OQvYZuf4IGZpboqCWA1KIvxT+G+fxEdqqr4EYsoyCtkMDFMZjGZHdVkauHn5K0dbKAsI 4X6c= X-Google-Smtp-Source: AGHT+IFpN/c0k0Ealux7lB+a8xpJ82Of6TwMQndmbFPYTJPbxW2IP0KZEJ9qUkXje59QO3zZnbQphQ== X-Received: by 2002:a05:6902:2102:b0:e0e:3d6a:8713 with SMTP id 3f1490d57ef6-e1155bc7f51mr792086276.56.1723584272930; Tue, 13 Aug 2024 14:24:32 -0700 (PDT) Received: from n36-183-057.byted.org ([130.44.212.116]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6bf432fb61dsm21390786d6.52.2024.08.13.14.24.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 14:24:32 -0700 (PDT) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, houtao@huaweicloud.com, sinquersw@gmail.com, davemarchevsky@fb.com, ameryhung@gmail.com, Amery Hung Subject: [PATCH v4 bpf-next 3/5] bpf: Rename ARG_PTR_TO_KPTR -> ARG_KPTR_XCHG_DEST Date: Tue, 13 Aug 2024 21:24:22 +0000 Message-Id: <20240813212424.2871455-4-amery.hung@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240813212424.2871455-1-amery.hung@bytedance.com> References: <20240813212424.2871455-1-amery.hung@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Dave Marchevsky ARG_PTR_TO_KPTR is currently only used by the bpf_kptr_xchg helper. Although it limits reg types for that helper's first arg to PTR_TO_MAP_VALUE, any arbitrary mapval won't do: further custom verification logic ensures that the mapval reg being xchgd-into is pointing to a kptr field. If this is not the case, it's not safe to xchg into that reg's pointee. Let's rename the bpf_arg_type to more accurately describe the fairly specific expectations that this arg type encodes. This is a nonfunctional change. Acked-by: Martin KaFai Lau Signed-off-by: Dave Marchevsky Signed-off-by: Amery Hung --- include/linux/bpf.h | 2 +- kernel/bpf/helpers.c | 2 +- kernel/bpf/verifier.c | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 7ad37cbdc815..f853e350c057 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -744,7 +744,7 @@ enum bpf_arg_type { ARG_PTR_TO_STACK, /* pointer to stack */ ARG_PTR_TO_CONST_STR, /* pointer to a null terminated read-only string */ ARG_PTR_TO_TIMER, /* pointer to bpf_timer */ - ARG_PTR_TO_KPTR, /* pointer to referenced kptr */ + ARG_KPTR_XCHG_DEST, /* pointer to destination that kptrs are bpf_kptr_xchg'd into */ ARG_PTR_TO_DYNPTR, /* pointer to bpf_dynptr. See bpf_type_flag for dynptr type */ __BPF_ARG_TYPE_MAX, diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index d02ae323996b..8ecd8dc95f16 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1636,7 +1636,7 @@ static const struct bpf_func_proto bpf_kptr_xchg_proto = { .gpl_only = false, .ret_type = RET_PTR_TO_BTF_ID_OR_NULL, .ret_btf_id = BPF_PTR_POISON, - .arg1_type = ARG_PTR_TO_KPTR, + .arg1_type = ARG_KPTR_XCHG_DEST, .arg2_type = ARG_PTR_TO_BTF_ID_OR_NULL | OBJ_RELEASE, .arg2_btf_id = BPF_PTR_POISON, }; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 1f5302fb0957..9f2964b13b46 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -8399,7 +8399,7 @@ static const struct bpf_reg_types func_ptr_types = { .types = { PTR_TO_FUNC } }; static const struct bpf_reg_types stack_ptr_types = { .types = { PTR_TO_STACK } }; static const struct bpf_reg_types const_str_ptr_types = { .types = { PTR_TO_MAP_VALUE } }; static const struct bpf_reg_types timer_types = { .types = { PTR_TO_MAP_VALUE } }; -static const struct bpf_reg_types kptr_types = { .types = { PTR_TO_MAP_VALUE } }; +static const struct bpf_reg_types kptr_xchg_dest_types = { .types = { PTR_TO_MAP_VALUE } }; static const struct bpf_reg_types dynptr_types = { .types = { PTR_TO_STACK, @@ -8431,7 +8431,7 @@ static const struct bpf_reg_types *compatible_reg_types[__BPF_ARG_TYPE_MAX] = { [ARG_PTR_TO_STACK] = &stack_ptr_types, [ARG_PTR_TO_CONST_STR] = &const_str_ptr_types, [ARG_PTR_TO_TIMER] = &timer_types, - [ARG_PTR_TO_KPTR] = &kptr_types, + [ARG_KPTR_XCHG_DEST] = &kptr_xchg_dest_types, [ARG_PTR_TO_DYNPTR] = &dynptr_types, }; @@ -9031,7 +9031,7 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, return err; break; } - case ARG_PTR_TO_KPTR: + case ARG_KPTR_XCHG_DEST: err = process_kptr_func(env, regno, meta); if (err) return err; From patchwork Tue Aug 13 21:24:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13762591 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C6DD43155 for ; Tue, 13 Aug 2024 21:24:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584276; cv=none; b=KyMWLd+DVF068rpclaYWapKZ8Nk7Lx425Ci5cnx7P8g/Sogh6CKPuNmTkdUmvydVffZxNepVtQOCGYWJ3kAN7k8unUIWUlSh+zhZjFwWqxoc+R/ND6udoPqei8fH4IlBrDrrHI3pMT1T1khl0FoJqBrX3iAqbKLOiSmWQ5hByEY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584276; c=relaxed/simple; bh=9bVt2RqQHMBiLL+bNtCoVf9m2FTiqTamJTbmrLYY5Ks=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tAv1AM9xA7HQyEepWMeBJsypRlR67F+t5LGFJIXo/t3SRm+LiOE/uuAyFCwVckf6tCp6G8Q/W9FExeoWOUF88sedGSgkRz3v45SNAUM7ogTQAS1Jo9Okm1L4E7ZGSpirqfZuiZ/wYINj4kEX+DXE/52rv3lWk2TbRbKPQuY2gTo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=dYaFL/to; arc=none smtp.client-ip=209.85.219.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="dYaFL/to" Received: by mail-qv1-f42.google.com with SMTP id 6a1803df08f44-6b5dfcfb165so32999356d6.0 for ; Tue, 13 Aug 2024 14:24:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1723584273; x=1724189073; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kSdXUIc8I8ipGRJ21IkX+BG97y/uudEncK49JhUrZvI=; b=dYaFL/toHjoBhetM5GF7I5MBkYFc85Otg3hmz3spgWCw1QrJ49tw6Ca+4+JJFuQ1wg FO0/XlN46lqLaU+46fbHThpDBmO92emAOPCQr0yaUsM93V3s5Kjk3sd5wX9bvqUprTfL aT8r52bltaz5fsfqy1HC4DGkVrjmBUbHpAs0W44HEwLpHZL9jk+IljoXSTdZQiJ4qUWS 4P/dJHxt0HKwo4GwLzrrySqZLHyvo8MHeAwCyeDcMtwBY7nfHTQGtwIafy2jlTX9XC8r BtBeNVxQ2gFCmsiHkkjhxtTDpp+dy+bZcLJ/y5rMHR6NIpHn42WvJ6tDDe9UDNPF7Qy1 Lktg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723584273; x=1724189073; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kSdXUIc8I8ipGRJ21IkX+BG97y/uudEncK49JhUrZvI=; b=VWXy8YjRGLnbJx/sgjCN85ykdzew8tqr13U8A1EVueFKSNA3D3WoLUJJRHLNJtjV1j XriRHfWUR2PvQUhEIpA0X8mi+NpW4XGJXHwPuDLhXnwZygy48v13gKKP3+KFXCZ3yVhE 1o3DXg03rJISKnlYPmlo4iTMfdb4shtXSZsGxjcsQChquh5Ytn5nEkMfzcBoASW4j6ws sC7Ri2EzHgZwZ9aC0zSSIsAQEpWk0c4DLFSizhpqVeIDUbYJAgHLdQ2YA1etkLREvBVV qMaXFPDrpDszuPcohetUz9E6LvxjCJ724RnupMlW+R74c7devm937UOt6mPcG38JuRrr EMZw== X-Gm-Message-State: AOJu0YxJbs+gdvSdPRD/BDP/qK21FNiq2XK8zJ9Uiqr25ZRg2inC/s+g 10dP1KnnM37UV4I4FNPth6Dujp2LcEX5fpX76cgW1zL9QTfNnIt9/tqcocDE7/B77/eM5NAIPeI mh1Y= X-Google-Smtp-Source: AGHT+IHzZYqKxQdetXbSVaqyg1P1QFqpicvfxJoEByjWY3ztNbUASprpU4oVq2mBS1UhQVXLsnEfeQ== X-Received: by 2002:a05:6214:4413:b0:6b7:ab54:3b90 with SMTP id 6a1803df08f44-6bf5d22db83mr8767996d6.35.1723584273466; Tue, 13 Aug 2024 14:24:33 -0700 (PDT) Received: from n36-183-057.byted.org ([130.44.212.116]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6bf432fb61dsm21390786d6.52.2024.08.13.14.24.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 14:24:33 -0700 (PDT) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, houtao@huaweicloud.com, sinquersw@gmail.com, davemarchevsky@fb.com, ameryhung@gmail.com, Amery Hung Subject: [PATCH v4 bpf-next 4/5] bpf: Support bpf_kptr_xchg into local kptr Date: Tue, 13 Aug 2024 21:24:23 +0000 Message-Id: <20240813212424.2871455-5-amery.hung@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240813212424.2871455-1-amery.hung@bytedance.com> References: <20240813212424.2871455-1-amery.hung@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Dave Marchevsky Currently, users can only stash kptr into map values with bpf_kptr_xchg(). This patch further supports stashing kptr into local kptr by adding local kptr as a valid destination type. When stashing into local kptr, btf_record in program BTF is used instead of btf_record in map to search for the btf_field of the local kptr. The local kptr specific checks in check_reg_type() only apply when the source argument of bpf_kptr_xchg() is local kptr. Therefore, we make the scope of the check explicit as the destination now can also be local kptr. Acked-by: Martin KaFai Lau Signed-off-by: Dave Marchevsky Signed-off-by: Amery Hung --- include/uapi/linux/bpf.h | 9 ++++---- kernel/bpf/helpers.c | 4 ++-- kernel/bpf/verifier.c | 44 +++++++++++++++++++++++++++------------- 3 files changed, 37 insertions(+), 20 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 35bcf52dbc65..e2629457d72d 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -5519,11 +5519,12 @@ union bpf_attr { * **-EOPNOTSUPP** if the hash calculation failed or **-EINVAL** if * invalid arguments are passed. * - * void *bpf_kptr_xchg(void *map_value, void *ptr) + * void *bpf_kptr_xchg(void *dst, void *ptr) * Description - * Exchange kptr at pointer *map_value* with *ptr*, and return the - * old value. *ptr* can be NULL, otherwise it must be a referenced - * pointer which will be released when this helper is called. + * Exchange kptr at pointer *dst* with *ptr*, and return the old value. + * *dst* can be map value or local kptr. *ptr* can be NULL, otherwise + * it must be a referenced pointer which will be released when this helper + * is called. * Return * The old value of kptr (which can be NULL). The returned pointer * if not NULL, is a reference which must be released using its diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 8ecd8dc95f16..d1a39734894c 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1619,9 +1619,9 @@ void bpf_wq_cancel_and_free(void *val) schedule_work(&work->delete_work); } -BPF_CALL_2(bpf_kptr_xchg, void *, map_value, void *, ptr) +BPF_CALL_2(bpf_kptr_xchg, void *, dst, void *, ptr) { - unsigned long *kptr = map_value; + unsigned long *kptr = dst; /* This helper may be inlined by verifier. */ return xchg(kptr, (unsigned long)ptr); diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 9f2964b13b46..5a4ca7e29272 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7803,29 +7803,38 @@ static int process_kptr_func(struct bpf_verifier_env *env, int regno, struct bpf_call_arg_meta *meta) { struct bpf_reg_state *regs = cur_regs(env), *reg = ®s[regno]; - struct bpf_map *map_ptr = reg->map_ptr; struct btf_field *kptr_field; + struct bpf_map *map_ptr; + struct btf_record *rec; u32 kptr_off; + if (type_is_ptr_alloc_obj(reg->type)) { + rec = reg_btf_record(reg); + } else { /* PTR_TO_MAP_VALUE */ + map_ptr = reg->map_ptr; + if (!map_ptr->btf) { + verbose(env, "map '%s' has to have BTF in order to use bpf_kptr_xchg\n", + map_ptr->name); + return -EINVAL; + } + rec = map_ptr->record; + meta->map_ptr = map_ptr; + } + if (!tnum_is_const(reg->var_off)) { verbose(env, "R%d doesn't have constant offset. kptr has to be at the constant offset\n", regno); return -EINVAL; } - if (!map_ptr->btf) { - verbose(env, "map '%s' has to have BTF in order to use bpf_kptr_xchg\n", - map_ptr->name); - return -EINVAL; - } - if (!btf_record_has_field(map_ptr->record, BPF_KPTR)) { - verbose(env, "map '%s' has no valid kptr\n", map_ptr->name); + + if (!btf_record_has_field(rec, BPF_KPTR)) { + verbose(env, "R%d has no valid kptr\n", regno); return -EINVAL; } - meta->map_ptr = map_ptr; kptr_off = reg->off + reg->var_off.value; - kptr_field = btf_record_find(map_ptr->record, kptr_off, BPF_KPTR); + kptr_field = btf_record_find(rec, kptr_off, BPF_KPTR); if (!kptr_field) { verbose(env, "off=%d doesn't point to kptr\n", kptr_off); return -EACCES; @@ -8399,7 +8408,12 @@ static const struct bpf_reg_types func_ptr_types = { .types = { PTR_TO_FUNC } }; static const struct bpf_reg_types stack_ptr_types = { .types = { PTR_TO_STACK } }; static const struct bpf_reg_types const_str_ptr_types = { .types = { PTR_TO_MAP_VALUE } }; static const struct bpf_reg_types timer_types = { .types = { PTR_TO_MAP_VALUE } }; -static const struct bpf_reg_types kptr_xchg_dest_types = { .types = { PTR_TO_MAP_VALUE } }; +static const struct bpf_reg_types kptr_xchg_dest_types = { + .types = { + PTR_TO_MAP_VALUE, + PTR_TO_BTF_ID | MEM_ALLOC + } +}; static const struct bpf_reg_types dynptr_types = { .types = { PTR_TO_STACK, @@ -8470,7 +8484,8 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, if (base_type(arg_type) == ARG_PTR_TO_MEM) type &= ~DYNPTR_TYPE_FLAG_MASK; - if (meta->func_id == BPF_FUNC_kptr_xchg && type_is_alloc(type)) { + /* Local kptr types are allowed as the source argument of bpf_kptr_xchg */ + if (meta->func_id == BPF_FUNC_kptr_xchg && type_is_alloc(type) && regno == BPF_REG_2) { type &= ~MEM_ALLOC; type &= ~MEM_PERCPU; } @@ -8563,7 +8578,8 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, verbose(env, "verifier internal error: unimplemented handling of MEM_ALLOC\n"); return -EFAULT; } - if (meta->func_id == BPF_FUNC_kptr_xchg) { + /* Check if local kptr in src arg matches kptr in dst arg */ + if (meta->func_id == BPF_FUNC_kptr_xchg && regno == BPF_REG_2) { if (map_kptr_match_type(env, meta->kptr_field, reg, regno)) return -EACCES; } @@ -8874,7 +8890,7 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, meta->release_regno = regno; } - if (reg->ref_obj_id) { + if (reg->ref_obj_id && base_type(arg_type) != ARG_KPTR_XCHG_DEST) { if (meta->ref_obj_id) { verbose(env, "verifier internal error: more than one arg with ref_obj_id R%d %u %u\n", regno, reg->ref_obj_id, From patchwork Tue Aug 13 21:24:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13762590 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49BC41A76DD for ; Tue, 13 Aug 2024 21:24:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584276; cv=none; b=EZiqyUx2H6vv+GkdpdyIl1solLs0hRPU1586+G2ka2anDwDE4gMJVCvq4dQwuKtodkW5PJ1yFM6weZ5Pv8Hg/mFakOms1HQavgNgNxSSFXI9Eh4yXCmCVwx+FrbAlPisYrNuHo7pf0q5SEpYZKP+D4lsO2lDiH+/euWhy6zQ9WY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723584276; c=relaxed/simple; bh=l7cXzz1Fi3bJLPmo/erBU+6IMXHygyE9Oz7+u83z9tw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aMCMLa3Y1oKwtauUQZWRu+B6mAeoL+8IsiHWxzZZ+ytDduKL/ygrVnbaZCnnTDDmtgMp2bLnd6f7/JF6WPzGw7+i8GgFWmgWCK0p3bPKR0xcK0Y6Mj9jBlB6k+sS166JMOrr/jNj6iQ6HGghW1VIB80liIETAAUTOy5dMO5W4sI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=PiDQZrcO; arc=none smtp.client-ip=209.85.219.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="PiDQZrcO" Received: by mail-yb1-f180.google.com with SMTP id 3f1490d57ef6-dfe43dca3bfso5261565276.0 for ; Tue, 13 Aug 2024 14:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1723584274; x=1724189074; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=M0j/9W5NoTZNr/s7dgUSvKvFbmQwhYnbQ8NOOXdGYL0=; b=PiDQZrcOCq/c9842/0Mb4kfUrUYG73U152xsficWkkpce1x5r804vNp6rtkHObilPc /kBpHwfcc8WI4bHvDa7K6mFx+F4lG47Ml2OBYOkWa+TK+R+93nzKwPuubwOlhZG8Im1V yNk8pLIhJTaWslmgddoOjT9z0dkoizBfN7k5utBqnwSuPP4gU2beklI8L86bTbXxR/2Z VZ0hfqMWJpYYGegmc745vrlJkYtbZs0EcgxPKdXHux1FPoRZNm6qEVC4OlsxKgWFcOZv yIdaziGm7IR9HNZvN0Fdnr2GB0eq8HoAXBY05YcdrOBkGx0guUtPI5uDH7mUhhjSg7RD z2Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723584274; x=1724189074; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M0j/9W5NoTZNr/s7dgUSvKvFbmQwhYnbQ8NOOXdGYL0=; b=J97msIn4OsUSqZCVT3Og47gtOPn5gaK0LeP702i7LMlj/d8e5RZOV4zKytGQNsRKVc /eI8HRlw8eLmeTlNk36v4AM24xtyDdzYv9Fu8miqftrK+W22ypMiyOlDfHXNiny2NBGI YOOipKQtFTJDVUsIwqWo9rzpxUztYvWGLMVtoRvKaRHSFUEcGa0QEK4/3Y3TU3PMZcCV GuleF6/Du1lhJFmZYyV5ZRMv0ihBG1rq+wzF14XlsGLdBwDc2R8KrwFBl+aZQ0KPXTEV 1bZnNeJ9TQELpvEjlqlZ429fUCy3UfrGL5Fyhzi4QDINrxNDOPNlVTn+gdjziR6umlLF Gzcw== X-Gm-Message-State: AOJu0YztKZ/xphJDSZWmCg6UE9yhMy6eRXChk3MiqkJCictiTbTZ4Gb6 +qhoPj4ua0gFteg90/MrcooTP7Ib/u3DiCCjeeePT9gTgd3KSJPVnD6YuzpVMZNNZUoQJdS6af3 RBCk= X-Google-Smtp-Source: AGHT+IGfEhTfnqg1fyGB1/YFtby+56d7HbRJkeg0IJY7ajTxiQvvsK3UkvbjRYwk+Vd9aOVobUTp9g== X-Received: by 2002:a05:6902:e10:b0:e0b:d7da:adf9 with SMTP id 3f1490d57ef6-e1155ab305cmr996249276.19.1723584274034; Tue, 13 Aug 2024 14:24:34 -0700 (PDT) Received: from n36-183-057.byted.org ([130.44.212.116]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6bf432fb61dsm21390786d6.52.2024.08.13.14.24.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Aug 2024 14:24:33 -0700 (PDT) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, houtao@huaweicloud.com, sinquersw@gmail.com, davemarchevsky@fb.com, ameryhung@gmail.com, Hou Tao , Amery Hung Subject: [PATCH v4 bpf-next 5/5] selftests/bpf: Test bpf_kptr_xchg stashing into local kptr Date: Tue, 13 Aug 2024 21:24:24 +0000 Message-Id: <20240813212424.2871455-6-amery.hung@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240813212424.2871455-1-amery.hung@bytedance.com> References: <20240813212424.2871455-1-amery.hung@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Dave Marchevsky Test stashing both referenced kptr and local kptr into local kptrs. Then, test unstashing them. Acked-by: Martin KaFai Lau Acked-by: Hou Tao Signed-off-by: Dave Marchevsky Signed-off-by: Amery Hung --- .../selftests/bpf/progs/local_kptr_stash.c | 30 +++++++++++++++++-- .../selftests/bpf/progs/task_kfunc_success.c | 26 +++++++++++++++- 2 files changed, 53 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/progs/local_kptr_stash.c b/tools/testing/selftests/bpf/progs/local_kptr_stash.c index 75043ffc5dad..b092a72b2c9d 100644 --- a/tools/testing/selftests/bpf/progs/local_kptr_stash.c +++ b/tools/testing/selftests/bpf/progs/local_kptr_stash.c @@ -8,9 +8,12 @@ #include "../bpf_experimental.h" #include "../bpf_testmod/bpf_testmod_kfunc.h" +struct plain_local; + struct node_data { long key; long data; + struct plain_local __kptr * stashed_in_local_kptr; struct bpf_rb_node node; }; @@ -85,6 +88,7 @@ static bool less(struct bpf_rb_node *a, const struct bpf_rb_node *b) static int create_and_stash(int idx, int val) { + struct plain_local *inner_local_kptr; struct map_value *mapval; struct node_data *res; @@ -92,11 +96,25 @@ static int create_and_stash(int idx, int val) if (!mapval) return 1; + inner_local_kptr = bpf_obj_new(typeof(*inner_local_kptr)); + if (!inner_local_kptr) + return 2; + res = bpf_obj_new(typeof(*res)); - if (!res) - return 1; + if (!res) { + bpf_obj_drop(inner_local_kptr); + return 3; + } res->key = val; + inner_local_kptr = bpf_kptr_xchg(&res->stashed_in_local_kptr, inner_local_kptr); + if (inner_local_kptr) { + /* Should never happen, we just obj_new'd res */ + bpf_obj_drop(inner_local_kptr); + bpf_obj_drop(res); + return 4; + } + res = bpf_kptr_xchg(&mapval->node, res); if (res) bpf_obj_drop(res); @@ -169,6 +187,7 @@ long stash_local_with_root(void *ctx) SEC("tc") long unstash_rb_node(void *ctx) { + struct plain_local *inner_local_kptr = NULL; struct map_value *mapval; struct node_data *res; long retval; @@ -180,6 +199,13 @@ long unstash_rb_node(void *ctx) res = bpf_kptr_xchg(&mapval->node, NULL); if (res) { + inner_local_kptr = bpf_kptr_xchg(&res->stashed_in_local_kptr, inner_local_kptr); + if (!inner_local_kptr) { + bpf_obj_drop(res); + return 1; + } + bpf_obj_drop(inner_local_kptr); + retval = res->key; bpf_obj_drop(res); return retval; diff --git a/tools/testing/selftests/bpf/progs/task_kfunc_success.c b/tools/testing/selftests/bpf/progs/task_kfunc_success.c index 70df695312dc..3138bb689b0b 100644 --- a/tools/testing/selftests/bpf/progs/task_kfunc_success.c +++ b/tools/testing/selftests/bpf/progs/task_kfunc_success.c @@ -5,6 +5,7 @@ #include #include +#include "../bpf_experimental.h" #include "task_kfunc_common.h" char _license[] SEC("license") = "GPL"; @@ -143,7 +144,7 @@ SEC("tp_btf/task_newtask") int BPF_PROG(test_task_xchg_release, struct task_struct *task, u64 clone_flags) { struct task_struct *kptr; - struct __tasks_kfunc_map_value *v; + struct __tasks_kfunc_map_value *v, *local; long status; if (!is_test_kfunc_task()) @@ -167,6 +168,29 @@ int BPF_PROG(test_task_xchg_release, struct task_struct *task, u64 clone_flags) return 0; } + local = bpf_obj_new(typeof(*local)); + if (!local) { + err = 4; + bpf_task_release(kptr); + return 0; + } + + kptr = bpf_kptr_xchg(&local->task, kptr); + if (kptr) { + err = 5; + bpf_obj_drop(local); + bpf_task_release(kptr); + return 0; + } + + kptr = bpf_kptr_xchg(&local->task, NULL); + if (!kptr) { + err = 6; + bpf_obj_drop(local); + return 0; + } + + bpf_obj_drop(local); bpf_task_release(kptr); return 0;