| Message ID | 5b8a8e318c9d661f495b6d0be2b7a776de7da7a1.1729588019.git.tanggeliang@kylinos.cn (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Delegated to: | Matthieu Baerts |
| Headers | show |
| Series | BPF path manager | expand |
| Context | Check | Description |
|---|---|---|
| matttbe/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 122 lines checked |
| matttbe/shellcheck | success | MPTCP selftests files have not been modified |
| matttbe/build | warning | Build error with: make C=1 net/mptcp/bpf.o |
| matttbe/KVM_Validation__normal | success | Success! ✅ |
| matttbe/KVM_Validation__debug | success | Success! ✅ |
| matttbe/KVM_Validation__btf-normal__only_bpftest_all_ | success | Success! ✅ |
| matttbe/KVM_Validation__btf-debug__only_bpftest_all_ | success | Success! ✅ |
Hi Geliang, On 22/10/2024 11:14, Geliang Tang wrote: > From: Geliang Tang <tanggeliang@kylinos.cn> > > The following code in mptcp_pm_nl_subflow_create_doit() that assigns struct > mptcp_pm_addr_entry "entry" to the local struct mptcp_pm_local variable > "local" is not allowed in BPF if we use the same code to implement the > subflow_create() interface of a BFP path manager: > > struct mptcp_pm_local local; > > local.addr = entry.addr; > local.flags = entry.flags; > local.ifindex = entry.ifindex; Here as well, I'm not sure to understand the issue. Is it because you cannot allocate this structure locally with BPF? No alternatives? > We should avoid this type of assignment from struct mptcp_pm_addr_entry to > struct mptcp_pm_local. > > In fact, there is no need to add a dedicated address entry type for local > address entry. All its fields are the same as struct mptcp_pm_addr_entry, > except that it lacks a "lsk" for the listening socket. So we can use struct > mptcp_pm_addr_entry directly. This makes the path manager code simpler. There was a need for this structure, see this discussion: https://lore.kernel.org/mptcp/587e096f-0e76-40fd-9220-6d8dd2930838@redhat.com/ and the changelog in v5: https://lore.kernel.org/mptcp/20240726-mptcp-pm-avail-v5-13-fb1117ddeef6@kernel.org/ In short ... > > Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn> > --- > net/mptcp/pm_netlink.c | 22 ++++++++-------------- > net/mptcp/pm_userspace.c | 7 +------ > net/mptcp/protocol.h | 8 +------- > net/mptcp/subflow.c | 2 +- > 4 files changed, 11 insertions(+), 28 deletions(-) > > diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c > index 287e715bcf68..15f97e7af182 100644 > --- a/net/mptcp/pm_netlink.c > +++ b/net/mptcp/pm_netlink.c (...) > @@ -721,7 +715,7 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk, > > static void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk) > { > - struct mptcp_pm_local locals[MPTCP_PM_ADDR_MAX]; > + struct mptcp_pm_addr_entry locals[MPTCP_PM_ADDR_MAX]; ... we don't want that: that's reserving too much memory, and we don't need half of it. *If* it is really needed to change it for BPF (I'm surprised there are no alternatives), then maybe we can change the signature of __mptcp_subflow_connect(), but not change the code here. In this case, only one "struct mptcp_pm_addr_entry" will be needed, and the "struct mptcp_pm_local" can be defined in this pm_netlink.c file if it is only used here. But please mention that in the commit message. > struct sock *sk = (struct sock *)msk; > unsigned int add_addr_accept_max; > struct mptcp_addr_info remote; (...) Cheers, Matt
Hi Matt, On Mon, 2024-11-04 at 20:08 +0100, Matthieu Baerts wrote: > Hi Geliang, > > On 22/10/2024 11:14, Geliang Tang wrote: > > From: Geliang Tang <tanggeliang@kylinos.cn> > > > > The following code in mptcp_pm_nl_subflow_create_doit() that > > assigns struct > > mptcp_pm_addr_entry "entry" to the local struct mptcp_pm_local > > variable > > "local" is not allowed in BPF if we use the same code to implement > > the > > subflow_create() interface of a BFP path manager: > > > > struct mptcp_pm_local local; > > > > local.addr = entry.addr; > > local.flags = entry.flags; > > local.ifindex = entry.ifindex; > > Here as well, I'm not sure to understand the issue. Is it because you > cannot allocate this structure locally with BPF? No alternatives? Here "struct mptcp_pm_local local" is a "scalar", when passing it to __mptcp_subflow_connect(), this error occurs: # ; err = __mptcp_subflow_connect(sk, &local, remote); @ mptcp_bpf_userspace_pm.c:242 # 53: (bf) r1 = r6 ; R1_w=trusted_ptr_mptcp_sock() R6=trusted_ptr_mptcp_sock() # 54: (bf) r3 = r8 ; R3_w=trusted_ptr_mptcp_addr_info() R8=trusted_ptr_mptcp_addr_info() # 55: (85) call __mptcp_subflow_connect#34404 # max struct nesting depth exceeded # arg#1 pointer type STRUCT mptcp_pm_local must point to scalar, or struct with scalar Also, assigning an address to an address like this is not allowed in BPF: local.addr = entry.addr; > > > We should avoid this type of assignment from struct > > mptcp_pm_addr_entry to > > struct mptcp_pm_local. > > > > In fact, there is no need to add a dedicated address entry type for > > local > > address entry. All its fields are the same as struct > > mptcp_pm_addr_entry, > > except that it lacks a "lsk" for the listening socket. So we can > > use struct > > mptcp_pm_addr_entry directly. This makes the path manager code > > simpler. > > There was a need for this structure, see this discussion: > > > https://lore.kernel.org/mptcp/587e096f-0e76-40fd-9220-6d8dd2930838@redhat.com/ > > and the changelog in v5: > > > https://lore.kernel.org/mptcp/20240726-mptcp-pm-avail-v5-13-fb1117ddeef6@kernel.org/ > > In short ... > > > > > Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn> > > --- > > net/mptcp/pm_netlink.c | 22 ++++++++-------------- > > net/mptcp/pm_userspace.c | 7 +------ > > net/mptcp/protocol.h | 8 +------- > > net/mptcp/subflow.c | 2 +- > > 4 files changed, 11 insertions(+), 28 deletions(-) > > > > diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c > > index 287e715bcf68..15f97e7af182 100644 > > --- a/net/mptcp/pm_netlink.c > > +++ b/net/mptcp/pm_netlink.c > > (...) > > > @@ -721,7 +715,7 @@ static unsigned int > > fill_local_addresses_vec(struct mptcp_sock *msk, > > > > static void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk) > > { > > - struct mptcp_pm_local locals[MPTCP_PM_ADDR_MAX]; > > + struct mptcp_pm_addr_entry locals[MPTCP_PM_ADDR_MAX]; > > ... we don't want that: that's reserving too much memory, and we > don't > need half of it. Thanks for your information. > > *If* it is really needed to change it for BPF (I'm surprised there > are > no alternatives), then maybe we can change the signature of > __mptcp_subflow_connect(), but not change the code here. > > In this case, only one "struct mptcp_pm_addr_entry" will be needed, > and > the "struct mptcp_pm_local" can be defined in this pm_netlink.c file > if > it is only used here. > > But please mention that in the commit message. I updated it in v3 as you suggested. Thanks, -Geliang > > > struct sock *sk = (struct sock *)msk; > > unsigned int add_addr_accept_max; > > struct mptcp_addr_info remote; > (...) > > Cheers, > Matt
Hi Geliang, On 07/11/2024 08:29, Geliang Tang wrote: > Hi Matt, > > On Mon, 2024-11-04 at 20:08 +0100, Matthieu Baerts wrote: >> Hi Geliang, >> >> On 22/10/2024 11:14, Geliang Tang wrote: >>> From: Geliang Tang <tanggeliang@kylinos.cn> >>> >>> The following code in mptcp_pm_nl_subflow_create_doit() that >>> assigns struct >>> mptcp_pm_addr_entry "entry" to the local struct mptcp_pm_local >>> variable >>> "local" is not allowed in BPF if we use the same code to implement >>> the >>> subflow_create() interface of a BFP path manager: >>> >>> struct mptcp_pm_local local; >>> >>> local.addr = entry.addr; >>> local.flags = entry.flags; >>> local.ifindex = entry.ifindex; >> >> Here as well, I'm not sure to understand the issue. Is it because you >> cannot allocate this structure locally with BPF? No alternatives? > > Here "struct mptcp_pm_local local" is a "scalar", when passing it to > __mptcp_subflow_connect(), this error occurs: > > # ; err = __mptcp_subflow_connect(sk, &local, remote); @ > mptcp_bpf_userspace_pm.c:242 > # 53: (bf) r1 = r6 ; R1_w=trusted_ptr_mptcp_sock() > R6=trusted_ptr_mptcp_sock() > # 54: (bf) r3 = r8 ; > R3_w=trusted_ptr_mptcp_addr_info() R8=trusted_ptr_mptcp_addr_info() > # 55: (85) call __mptcp_subflow_connect#34404 > # max struct nesting depth exceeded > # arg#1 pointer type STRUCT mptcp_pm_local must point to scalar, or > struct with scalar > > Also, assigning an address to an address like this is not allowed in > BPF: > > local.addr = entry.addr; OK, thank you, that's a bit clearer (even if it is not clear what BPF code was causing that). For this kind of issue, would it work to add a BPF-specific kfunc, eventually with additional annotations or arguments, or taking a "struct mptcp_pm_addr_entry" in argument, and "simply" calling __mptcp_subflow_connect() with a "struct mptcp_pm_local"? So something specific to BPF, without impacting the rest? Cheers, Matt
Hi Matt, On Thu, 2024-11-07 at 11:00 +0100, Matthieu Baerts wrote: > Hi Geliang, > > On 07/11/2024 08:29, Geliang Tang wrote: > > Hi Matt, > > > > On Mon, 2024-11-04 at 20:08 +0100, Matthieu Baerts wrote: > > > Hi Geliang, > > > > > > On 22/10/2024 11:14, Geliang Tang wrote: > > > > From: Geliang Tang <tanggeliang@kylinos.cn> > > > > > > > > The following code in mptcp_pm_nl_subflow_create_doit() that > > > > assigns struct > > > > mptcp_pm_addr_entry "entry" to the local struct mptcp_pm_local > > > > variable > > > > "local" is not allowed in BPF if we use the same code to > > > > implement > > > > the > > > > subflow_create() interface of a BFP path manager: > > > > > > > > struct mptcp_pm_local local; > > > > > > > > local.addr = entry.addr; > > > > local.flags = entry.flags; > > > > local.ifindex = entry.ifindex; > > > > > > Here as well, I'm not sure to understand the issue. Is it because > > > you > > > cannot allocate this structure locally with BPF? No alternatives? > > > > Here "struct mptcp_pm_local local" is a "scalar", when passing it > > to > > __mptcp_subflow_connect(), this error occurs: > > > > # ; err = __mptcp_subflow_connect(sk, &local, remote); @ > > mptcp_bpf_userspace_pm.c:242 > > # 53: (bf) r1 = r6 ; > > R1_w=trusted_ptr_mptcp_sock() > > R6=trusted_ptr_mptcp_sock() > > # 54: (bf) r3 = r8 ; > > R3_w=trusted_ptr_mptcp_addr_info() R8=trusted_ptr_mptcp_addr_info() > > # 55: (85) call __mptcp_subflow_connect#34404 > > # max struct nesting depth exceeded > > # arg#1 pointer type STRUCT mptcp_pm_local must point to scalar, or > > struct with scalar > > > > Also, assigning an address to an address like this is not allowed > > in > > BPF: > > > > local.addr = entry.addr; > > OK, thank you, that's a bit clearer (even if it is not clear what BPF > code was causing that). For this kind of issue, would it work to add > a > BPF-specific kfunc, eventually with additional annotations or > arguments, > or taking a "struct mptcp_pm_addr_entry" in argument, and "simply" > calling __mptcp_subflow_connect() with a "struct mptcp_pm_local"? It works indeed. But I prefer the version in v3, "mptcp: use mptcp_pm_local in pm_netlink only". It makes more sense to move mptcp_pm_local in pm_netlink.c. WDYT? Thanks, -Geliang > > So something specific to BPF, without impacting the rest? > > Cheers, > Matt
Hi Matt, On Thu, 2024-11-07 at 18:19 +0800, Geliang Tang wrote: > Hi Matt, > > On Thu, 2024-11-07 at 11:00 +0100, Matthieu Baerts wrote: > > Hi Geliang, > > > > On 07/11/2024 08:29, Geliang Tang wrote: > > > Hi Matt, > > > > > > On Mon, 2024-11-04 at 20:08 +0100, Matthieu Baerts wrote: > > > > Hi Geliang, > > > > > > > > On 22/10/2024 11:14, Geliang Tang wrote: > > > > > From: Geliang Tang <tanggeliang@kylinos.cn> > > > > > > > > > > The following code in mptcp_pm_nl_subflow_create_doit() that > > > > > assigns struct > > > > > mptcp_pm_addr_entry "entry" to the local struct > > > > > mptcp_pm_local > > > > > variable > > > > > "local" is not allowed in BPF if we use the same code to > > > > > implement > > > > > the > > > > > subflow_create() interface of a BFP path manager: > > > > > > > > > > struct mptcp_pm_local local; > > > > > > > > > > local.addr = entry.addr; > > > > > local.flags = entry.flags; > > > > > local.ifindex = entry.ifindex; > > > > > > > > Here as well, I'm not sure to understand the issue. Is it > > > > because > > > > you > > > > cannot allocate this structure locally with BPF? No > > > > alternatives? > > > > > > Here "struct mptcp_pm_local local" is a "scalar", when passing it > > > to > > > __mptcp_subflow_connect(), this error occurs: > > > > > > # ; err = __mptcp_subflow_connect(sk, &local, remote); @ > > > mptcp_bpf_userspace_pm.c:242 > > > # 53: (bf) r1 = r6 ; > > > R1_w=trusted_ptr_mptcp_sock() > > > R6=trusted_ptr_mptcp_sock() > > > # 54: (bf) r3 = r8 ; > > > R3_w=trusted_ptr_mptcp_addr_info() > > > R8=trusted_ptr_mptcp_addr_info() > > > # 55: (85) call __mptcp_subflow_connect#34404 > > > # max struct nesting depth exceeded > > > # arg#1 pointer type STRUCT mptcp_pm_local must point to scalar, > > > or > > > struct with scalar > > > > > > Also, assigning an address to an address like this is not allowed > > > in > > > BPF: > > > > > > local.addr = entry.addr; > > > > OK, thank you, that's a bit clearer (even if it is not clear what > > BPF > > code was causing that). For this kind of issue, would it work to > > add > > a > > BPF-specific kfunc, eventually with additional annotations or > > arguments, > > or taking a "struct mptcp_pm_addr_entry" in argument, and "simply" > > calling __mptcp_subflow_connect() with a "struct mptcp_pm_local"? > > It works indeed. But I prefer the version in v3, "mptcp: use > mptcp_pm_local in pm_netlink only". It makes more sense to move > mptcp_pm_local in pm_netlink.c. I thought about it again and felt that "mptcp: use mptcp_pm_local in pm_netlink only" was not a good solution. If we want to add fullmesh support to userspace pm in the future, we also need to use struct mptcp_pm_local in pm_netlink.c. So I decided to deprecate "mptcp: use mptcp_pm_local in pm_netlink only" and keep the code for struct mptcp_pm_local as it is. As you suggested, add a BPF-specific kfunc to solve the issue. Thanks, -Geliang > > WDYT? > > Thanks, > -Geliang > > > > > So something specific to BPF, without impacting the rest? > > > > Cheers, > > Matt > >
diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c index 287e715bcf68..15f97e7af182 100644 --- a/net/mptcp/pm_netlink.c +++ b/net/mptcp/pm_netlink.c @@ -149,7 +149,7 @@ static bool lookup_subflow_by_daddr(const struct list_head *list, static bool select_local_address(const struct pm_nl_pernet *pernet, const struct mptcp_sock *msk, - struct mptcp_pm_local *new_local) + struct mptcp_pm_addr_entry *new_local) { struct mptcp_pm_addr_entry *entry; bool found = false; @@ -164,9 +164,7 @@ select_local_address(const struct pm_nl_pernet *pernet, if (!test_bit(entry->addr.id, msk->pm.id_avail_bitmap.map)) continue; - new_local->addr = entry->addr; - new_local->flags = entry->flags; - new_local->ifindex = entry->ifindex; + *new_local = *entry; found = true; break; } @@ -177,7 +175,7 @@ select_local_address(const struct pm_nl_pernet *pernet, static bool select_signal_address(struct pm_nl_pernet *pernet, const struct mptcp_sock *msk, - struct mptcp_pm_local *new_local) + struct mptcp_pm_addr_entry *new_local) { struct mptcp_pm_addr_entry *entry; bool found = false; @@ -195,9 +193,7 @@ select_signal_address(struct pm_nl_pernet *pernet, const struct mptcp_sock *msk, if (!(entry->flags & MPTCP_PM_ADDR_FLAG_SIGNAL)) continue; - new_local->addr = entry->addr; - new_local->flags = entry->flags; - new_local->ifindex = entry->ifindex; + *new_local = *entry; found = true; break; } @@ -534,11 +530,11 @@ __lookup_addr(struct pm_nl_pernet *pernet, const struct mptcp_addr_info *info) static void mptcp_pm_create_subflow_or_signal_addr(struct mptcp_sock *msk) { struct sock *sk = (struct sock *)msk; + struct mptcp_pm_addr_entry local; unsigned int add_addr_signal_max; bool signal_and_subflow = false; unsigned int local_addr_max; struct pm_nl_pernet *pernet; - struct mptcp_pm_local local; unsigned int subflows_max; pernet = pm_nl_get_pernet(sock_net(sk)); @@ -660,7 +656,7 @@ static void mptcp_pm_nl_subflow_established(struct mptcp_sock *msk) */ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk, struct mptcp_addr_info *remote, - struct mptcp_pm_local *locals) + struct mptcp_pm_addr_entry *locals) { struct sock *sk = (struct sock *)msk; struct mptcp_pm_addr_entry *entry; @@ -683,9 +679,7 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk, continue; if (msk->pm.subflows < subflows_max) { - locals[i].addr = entry->addr; - locals[i].flags = entry->flags; - locals[i].ifindex = entry->ifindex; + locals[i] = *entry; /* Special case for ID0: set the correct ID */ if (mptcp_addresses_equal(&locals[i].addr, &mpc_addr, locals[i].addr.port)) @@ -721,7 +715,7 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk, static void mptcp_pm_nl_add_addr_received(struct mptcp_sock *msk) { - struct mptcp_pm_local locals[MPTCP_PM_ADDR_MAX]; + struct mptcp_pm_addr_entry locals[MPTCP_PM_ADDR_MAX]; struct sock *sk = (struct sock *)msk; unsigned int add_addr_accept_max; struct mptcp_addr_info remote; diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c index 0012b1965421..c1bc668cce1e 100644 --- a/net/mptcp/pm_userspace.c +++ b/net/mptcp/pm_userspace.c @@ -339,7 +339,6 @@ int mptcp_pm_nl_subflow_create_doit(struct sk_buff *skb, struct genl_info *info) struct nlattr *laddr = info->attrs[MPTCP_PM_ATTR_ADDR]; struct mptcp_pm_addr_entry entry = { 0 }; struct mptcp_addr_info addr_r; - struct mptcp_pm_local local; struct mptcp_sock *msk; int err = -EINVAL; struct sock *sk; @@ -386,12 +385,8 @@ int mptcp_pm_nl_subflow_create_doit(struct sk_buff *skb, struct genl_info *info) goto create_err; } - local.addr = entry.addr; - local.flags = entry.flags; - local.ifindex = entry.ifindex; - lock_sock(sk); - err = __mptcp_subflow_connect(sk, &local, &addr_r); + err = __mptcp_subflow_connect(sk, &entry, &addr_r); release_sock(sk); spin_lock_bh(&msk->pm.lock); diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 8efd288bd247..d9badf270b15 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -240,12 +240,6 @@ struct mptcp_pm_data { struct mptcp_rm_list rm_list_rx; }; -struct mptcp_pm_local { - struct mptcp_addr_info addr; - u8 flags; - int ifindex; -}; - struct mptcp_pm_addr_entry { struct list_head list; struct mptcp_addr_info addr; @@ -745,7 +739,7 @@ bool mptcp_addresses_equal(const struct mptcp_addr_info *a, void mptcp_local_address(const struct sock_common *skc, struct mptcp_addr_info *addr); /* called with sk socket lock held */ -int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_pm_local *local, +int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_pm_addr_entry *local, const struct mptcp_addr_info *remote); int mptcp_subflow_create_socket(struct sock *sk, unsigned short family, struct socket **new_sock); diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 07352b15f145..2ae8f467abc1 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -1586,7 +1586,7 @@ void mptcp_info2sockaddr(const struct mptcp_addr_info *info, #endif } -int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_pm_local *local, +int __mptcp_subflow_connect(struct sock *sk, const struct mptcp_pm_addr_entry *local, const struct mptcp_addr_info *remote) { struct mptcp_sock *msk = mptcp_sk(sk);