| Message ID | cover.1543343124.git.lorenzo.bianconi@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | add XDP support to mt76x2e/mt76x0e drivers | expand |
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > This series is intended as a playground to start experimenting/developing > with XDP/eBPF over WiFi and collect ideas/concerns about it. > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > actions are: > - XDP_PASS > - XDP_ABORTED > - XDP_DROP > Introduce ndo_bpf mac80211 callback in order to to load a bpf > program into low level driver XDP rx hook. > This series has been tested through a simple bpf program (available here: > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > used to count frame types received by the device. > Possible eBPF use cases could be: > - implement new statistics through bpf maps > - implement fast packet filtering (e.g in monitor mode) > - ... This is most likely a stupid question, but why do this in the driver and not in mac80211 so that all drivers could benefit from it? I guess there are reasons for that, I just can't figure that out.
> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > > This series is intended as a playground to start experimenting/developing > > with XDP/eBPF over WiFi and collect ideas/concerns about it. > > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > > actions are: > > - XDP_PASS > > - XDP_ABORTED > > - XDP_DROP > > Introduce ndo_bpf mac80211 callback in order to to load a bpf > > program into low level driver XDP rx hook. > > This series has been tested through a simple bpf program (available here: > > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > > used to count frame types received by the device. > > Possible eBPF use cases could be: > > - implement new statistics through bpf maps > > - implement fast packet filtering (e.g in monitor mode) > > - ... Hi Kalle, > > This is most likely a stupid question, but why do this in the driver and > not in mac80211 so that all drivers could benefit from it? I guess there > are reasons for that, I just can't figure that out. > > -- > Kalle Valo I thought about that possibility (and I think it is definitely valuable) but I preferred to work as close as possible to the hw running the bpf program before skb allocation. My primary goal when I started thinking about eBPF over WiFi was to perform fast packet filtering. I think these two possibility are not mutually exclusive, we can fall-back to mac80211 if hw driver does not support XDP. This is an early stage implementation, at this point I would collect other people opinions/concerns about using bpf/xdp directly on 802.11 frames. Regards, Lorenzo
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> > This series is intended as a playground to start experimenting/developing >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported >> > actions are: >> > - XDP_PASS >> > - XDP_ABORTED >> > - XDP_DROP >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf >> > program into low level driver XDP rx hook. >> > This series has been tested through a simple bpf program (available here: >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) >> > used to count frame types received by the device. >> > Possible eBPF use cases could be: >> > - implement new statistics through bpf maps >> > - implement fast packet filtering (e.g in monitor mode) >> > - ... > > Hi Kalle, > >> >> This is most likely a stupid question, but why do this in the driver and >> not in mac80211 so that all drivers could benefit from it? I guess there >> are reasons for that, I just can't figure that out. XDP achieves its speedup by running the eBPF program inside the driver NAPI loop, before the kernel even touches the data in any other capacity (and in particular, before it allocates an SKB). Which kinda means the hook needs to be in the driver... Could be a fallback in mac80211, though; although we'd have to figure out how that interacts with Generic XDP. > This is an early stage implementation, at this point I would collect > other people opinions/concerns about using bpf/xdp directly on 802.11 > frames. Thanks for looking into this! I have two concerns with running XDP on 802.11 frames: 1. It makes it more difficult to add other XDP actions (such as REDIRECT), as the XDP program would then have to make sure that the outer packet headers are removed before, say, redirecting the packet out of an ethernet interface. Also, if we do add redirect, we would be bypassing mac80211 entirely; to what extent would that mess up internal state? 2. UI consistency; suddenly, the user needs to know which kind of frames to expect, and XDP program reuse becomes more difficult. This may be unavoidable given the nature of XDP, but some thought needs to go into this. Especially since we wouldn't necessarily be consistent between WiFi drivers (there are fullmac devices that remove 802.11 headers before sending up the frame, right?). Adding in Jesper; maybe he has some thoughts on this? -Toke
On Wed, 28 Nov 2018 at 13:39, Toke Høiland-Jørgensen <toke@toke.dk> wrote:[...] > > This is an early stage implementation, at this point I would collect > > other people opinions/concerns about using bpf/xdp directly on 802.11 > > frames. > > Thanks for looking into this! I'm really hyped up with this especially when I think this could be maybe offloaded to wlan cpu's for e.g. deep powersaving (allow host system to suspend) without relying on blackbox logic you get today, or 802.11 -> 802.3 conversion (to offload host cpu). > I have two concerns with running XDP on 802.11 frames: > > 1. It makes it more difficult to add other XDP actions (such as > REDIRECT), as the XDP program would then have to make sure that the > outer packet headers are removed before, say, redirecting the packet > out of an ethernet interface. Also, if we do add redirect, we would > be bypassing mac80211 entirely; to what extent would that mess up > internal state? > > 2. UI consistency; suddenly, the user needs to know which kind of > frames to expect, and XDP program reuse becomes more difficult. This > may be unavoidable given the nature of XDP, but some thought needs to > go into this. Especially since we wouldn't necessarily be consistent > between WiFi drivers (there are fullmac devices that remove 802.11 > headers before sending up the frame, right?). Yep - you can expect 802.3 frames or amsdu subframes (DA+SA+length) too. In some cases you could also expect internal events (broadcom if I understand their fullmac arch). Michał
> >> > This series is intended as a playground to start experimenting/developing > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > >> > actions are: > >> > - XDP_PASS > >> > - XDP_ABORTED > >> > - XDP_DROP > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf > >> > program into low level driver XDP rx hook. > >> > This series has been tested through a simple bpf program (available here: > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > >> > used to count frame types received by the device. > >> > Possible eBPF use cases could be: > >> > - implement new statistics through bpf maps > >> > - implement fast packet filtering (e.g in monitor mode) > >> > - ... > > > > Hi Kalle, > > > >> > >> This is most likely a stupid question, but why do this in the driver and > >> not in mac80211 so that all drivers could benefit from it? I guess there > >> are reasons for that, I just can't figure that out. > > XDP achieves its speedup by running the eBPF program inside the driver > NAPI loop, before the kernel even touches the data in any other capacity > (and in particular, before it allocates an SKB). Which kinda means the > hook needs to be in the driver... Could be a fallback in mac80211, > though; although we'd have to figure out how that interacts with Generic > XDP. > > > This is an early stage implementation, at this point I would collect > > other people opinions/concerns about using bpf/xdp directly on 802.11 > > frames. > > Thanks for looking into this! Hi Toke, > > I have two concerns with running XDP on 802.11 frames: > > 1. It makes it more difficult to add other XDP actions (such as > REDIRECT), as the XDP program would then have to make sure that the > outer packet headers are removed before, say, redirecting the packet > out of an ethernet interface. Also, if we do add redirect, we would > be bypassing mac80211 entirely; to what extent would that mess up > internal state? > You are right, my assumption here is the logic/complexity is moved to the bpf program that needs to take care of all possible issues that can be introduced. More or less it is the same if a bpf program mess up with TCP segments on a wired connection, isn't it? > 2. UI consistency; suddenly, the user needs to know which kind of > frames to expect, and XDP program reuse becomes more difficult. This > may be unavoidable given the nature of XDP, but some thought needs to > go into this. Especially since we wouldn't necessarily be consistent > between WiFi drivers (there are fullmac devices that remove 802.11 > headers before sending up the frame, right?). > Right, maybe can we have some kind of 'wifi' bpf helpers? Regards, Lorenzo > > Adding in Jesper; maybe he has some thoughts on this? > > -Toke
Michał Kazior <kazikcz@gmail.com> writes: > On Wed, 28 Nov 2018 at 13:39, Toke Høiland-Jørgensen <toke@toke.dk> wrote:[...] >> > This is an early stage implementation, at this point I would collect >> > other people opinions/concerns about using bpf/xdp directly on 802.11 >> > frames. >> >> Thanks for looking into this! > > I'm really hyped up with this especially when I think this could be > maybe offloaded to wlan cpu's for e.g. deep powersaving (allow host > system to suspend) without relying on blackbox logic you get today, or > 802.11 -> 802.3 conversion (to offload host cpu). Yeah, good point! >> I have two concerns with running XDP on 802.11 frames: >> >> 1. It makes it more difficult to add other XDP actions (such as >> REDIRECT), as the XDP program would then have to make sure that the >> outer packet headers are removed before, say, redirecting the packet >> out of an ethernet interface. Also, if we do add redirect, we would >> be bypassing mac80211 entirely; to what extent would that mess up >> internal state? >> >> 2. UI consistency; suddenly, the user needs to know which kind of >> frames to expect, and XDP program reuse becomes more difficult. This >> may be unavoidable given the nature of XDP, but some thought needs to >> go into this. Especially since we wouldn't necessarily be consistent >> between WiFi drivers (there are fullmac devices that remove 802.11 >> headers before sending up the frame, right?). > > Yep - you can expect 802.3 frames or amsdu subframes (DA+SA+length) > too. In some cases you could also expect internal events (broadcom if > I understand their fullmac arch). Oh boy. I figured it was going to be a can of worms; didn't realise it was quite that big :/ -Toke
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> > This series is intended as a playground to start experimenting/developing >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported >> >> > actions are: >> >> > - XDP_PASS >> >> > - XDP_ABORTED >> >> > - XDP_DROP >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf >> >> > program into low level driver XDP rx hook. >> >> > This series has been tested through a simple bpf program (available here: >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) >> >> > used to count frame types received by the device. >> >> > Possible eBPF use cases could be: >> >> > - implement new statistics through bpf maps >> >> > - implement fast packet filtering (e.g in monitor mode) >> >> > - ... >> > >> > Hi Kalle, >> > >> >> >> >> This is most likely a stupid question, but why do this in the driver and >> >> not in mac80211 so that all drivers could benefit from it? I guess there >> >> are reasons for that, I just can't figure that out. >> >> XDP achieves its speedup by running the eBPF program inside the driver >> NAPI loop, before the kernel even touches the data in any other capacity >> (and in particular, before it allocates an SKB). Which kinda means the >> hook needs to be in the driver... Could be a fallback in mac80211, >> though; although we'd have to figure out how that interacts with Generic >> XDP. >> >> > This is an early stage implementation, at this point I would collect >> > other people opinions/concerns about using bpf/xdp directly on 802.11 >> > frames. >> >> Thanks for looking into this! > > Hi Toke, > >> >> I have two concerns with running XDP on 802.11 frames: >> >> 1. It makes it more difficult to add other XDP actions (such as >> REDIRECT), as the XDP program would then have to make sure that the >> outer packet headers are removed before, say, redirecting the packet >> out of an ethernet interface. Also, if we do add redirect, we would >> be bypassing mac80211 entirely; to what extent would that mess up >> internal state? >> > > You are right, my assumption here is the logic/complexity is moved to > the bpf program that needs to take care of all possible issues that > can be introduced. More or less it is the same if a bpf program mess > up with TCP segments on a wired connection, isn't it? No, I guess not; except here it potentially applies to all packets (things like BAW tracking), and it is *in addition* to TCP. >> 2. UI consistency; suddenly, the user needs to know which kind of >> frames to expect, and XDP program reuse becomes more difficult. This >> may be unavoidable given the nature of XDP, but some thought needs to >> go into this. Especially since we wouldn't necessarily be consistent >> between WiFi drivers (there are fullmac devices that remove 802.11 >> headers before sending up the frame, right?). >> > > Right, maybe can we have some kind of 'wifi' bpf helpers? Yeah, I guess we would at least need helpers to update any internal state in mac80211 (such as BAW), or BPF programs wouldn't even be able to drop packets without messing things up... -Toke
> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > >> >> > This series is intended as a playground to start experimenting/developing > >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. > >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > >> >> > actions are: > >> >> > - XDP_PASS > >> >> > - XDP_ABORTED > >> >> > - XDP_DROP > >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf > >> >> > program into low level driver XDP rx hook. > >> >> > This series has been tested through a simple bpf program (available here: > >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > >> >> > used to count frame types received by the device. > >> >> > Possible eBPF use cases could be: > >> >> > - implement new statistics through bpf maps > >> >> > - implement fast packet filtering (e.g in monitor mode) > >> >> > - ... > >> > > >> > Hi Kalle, > >> > > >> >> > >> >> This is most likely a stupid question, but why do this in the driver and > >> >> not in mac80211 so that all drivers could benefit from it? I guess there > >> >> are reasons for that, I just can't figure that out. > >> > >> XDP achieves its speedup by running the eBPF program inside the driver > >> NAPI loop, before the kernel even touches the data in any other capacity > >> (and in particular, before it allocates an SKB). Which kinda means the > >> hook needs to be in the driver... Could be a fallback in mac80211, > >> though; although we'd have to figure out how that interacts with Generic > >> XDP. > >> > >> > This is an early stage implementation, at this point I would collect > >> > other people opinions/concerns about using bpf/xdp directly on 802.11 > >> > frames. > >> > >> Thanks for looking into this! > > > > Hi Toke, > > > >> > >> I have two concerns with running XDP on 802.11 frames: > >> > >> 1. It makes it more difficult to add other XDP actions (such as > >> REDIRECT), as the XDP program would then have to make sure that the > >> outer packet headers are removed before, say, redirecting the packet > >> out of an ethernet interface. Also, if we do add redirect, we would > >> be bypassing mac80211 entirely; to what extent would that mess up > >> internal state? > >> > > > > You are right, my assumption here is the logic/complexity is moved to > > the bpf program that needs to take care of all possible issues that > > can be introduced. More or less it is the same if a bpf program mess > > up with TCP segments on a wired connection, isn't it? > > No, I guess not; except here it potentially applies to all packets > (things like BAW tracking), and it is *in addition* to TCP. Yes, here it is a little bit harder, but I was meaning that the bpf program has to be very careful when dropping a packet :) > > >> 2. UI consistency; suddenly, the user needs to know which kind of > >> frames to expect, and XDP program reuse becomes more difficult. This > >> may be unavoidable given the nature of XDP, but some thought needs to > >> go into this. Especially since we wouldn't necessarily be consistent > >> between WiFi drivers (there are fullmac devices that remove 802.11 > >> headers before sending up the frame, right?). > >> > > > > Right, maybe can we have some kind of 'wifi' bpf helpers? > > Yeah, I guess we would at least need helpers to update any internal > state in mac80211 (such as BAW), or BPF programs wouldn't even be able > to drop packets without messing things up... > Correct. Regards, Lorenzo > -Toke
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> >> > This series is intended as a playground to start experimenting/developing >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported >> >> >> > actions are: >> >> >> > - XDP_PASS >> >> >> > - XDP_ABORTED >> >> >> > - XDP_DROP >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf >> >> >> > program into low level driver XDP rx hook. >> >> >> > This series has been tested through a simple bpf program (available here: >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) >> >> >> > used to count frame types received by the device. >> >> >> > Possible eBPF use cases could be: >> >> >> > - implement new statistics through bpf maps >> >> >> > - implement fast packet filtering (e.g in monitor mode) >> >> >> > - ... >> >> > >> >> > Hi Kalle, >> >> > >> >> >> >> >> >> This is most likely a stupid question, but why do this in the driver and >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there >> >> >> are reasons for that, I just can't figure that out. >> >> >> >> XDP achieves its speedup by running the eBPF program inside the driver >> >> NAPI loop, before the kernel even touches the data in any other capacity >> >> (and in particular, before it allocates an SKB). Which kinda means the >> >> hook needs to be in the driver... Could be a fallback in mac80211, >> >> though; although we'd have to figure out how that interacts with Generic >> >> XDP. >> >> >> >> > This is an early stage implementation, at this point I would collect >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11 >> >> > frames. >> >> >> >> Thanks for looking into this! >> > >> > Hi Toke, >> > >> >> >> >> I have two concerns with running XDP on 802.11 frames: >> >> >> >> 1. It makes it more difficult to add other XDP actions (such as >> >> REDIRECT), as the XDP program would then have to make sure that the >> >> outer packet headers are removed before, say, redirecting the packet >> >> out of an ethernet interface. Also, if we do add redirect, we would >> >> be bypassing mac80211 entirely; to what extent would that mess up >> >> internal state? >> >> >> > >> > You are right, my assumption here is the logic/complexity is moved to >> > the bpf program that needs to take care of all possible issues that >> > can be introduced. More or less it is the same if a bpf program mess >> > up with TCP segments on a wired connection, isn't it? >> >> No, I guess not; except here it potentially applies to all packets >> (things like BAW tracking), and it is *in addition* to TCP. > > Yes, here it is a little bit harder, but I was meaning that the bpf program > has to be very careful when dropping a packet :) Yeah. What kind of filtering were you thinking you would use this for in the short term? -Toke
On Nov 28, Toke Høiland-Jørgensen wrote: > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > >> > >> >> >> > This series is intended as a playground to start experimenting/developing > >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. > >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > >> >> >> > actions are: > >> >> >> > - XDP_PASS > >> >> >> > - XDP_ABORTED > >> >> >> > - XDP_DROP > >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf > >> >> >> > program into low level driver XDP rx hook. > >> >> >> > This series has been tested through a simple bpf program (available here: > >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > >> >> >> > used to count frame types received by the device. > >> >> >> > Possible eBPF use cases could be: > >> >> >> > - implement new statistics through bpf maps > >> >> >> > - implement fast packet filtering (e.g in monitor mode) > >> >> >> > - ... > >> >> > > >> >> > Hi Kalle, > >> >> > > >> >> >> > >> >> >> This is most likely a stupid question, but why do this in the driver and > >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there > >> >> >> are reasons for that, I just can't figure that out. > >> >> > >> >> XDP achieves its speedup by running the eBPF program inside the driver > >> >> NAPI loop, before the kernel even touches the data in any other capacity > >> >> (and in particular, before it allocates an SKB). Which kinda means the > >> >> hook needs to be in the driver... Could be a fallback in mac80211, > >> >> though; although we'd have to figure out how that interacts with Generic > >> >> XDP. > >> >> > >> >> > This is an early stage implementation, at this point I would collect > >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11 > >> >> > frames. > >> >> > >> >> Thanks for looking into this! > >> > > >> > Hi Toke, > >> > > >> >> > >> >> I have two concerns with running XDP on 802.11 frames: > >> >> > >> >> 1. It makes it more difficult to add other XDP actions (such as > >> >> REDIRECT), as the XDP program would then have to make sure that the > >> >> outer packet headers are removed before, say, redirecting the packet > >> >> out of an ethernet interface. Also, if we do add redirect, we would > >> >> be bypassing mac80211 entirely; to what extent would that mess up > >> >> internal state? > >> >> > >> > > >> > You are right, my assumption here is the logic/complexity is moved to > >> > the bpf program that needs to take care of all possible issues that > >> > can be introduced. More or less it is the same if a bpf program mess > >> > up with TCP segments on a wired connection, isn't it? > >> > >> No, I guess not; except here it potentially applies to all packets > >> (things like BAW tracking), and it is *in addition* to TCP. > > > > Yes, here it is a little bit harder, but I was meaning that the bpf program > > has to be very careful when dropping a packet :) > > Yeah. What kind of filtering were you thinking you would use this for in > the short term? > When I started working on XDP for mt76 I was thinking about BSSID filtering but I was looking for a more general solution respect to add that feature in the driver. Moreover we could use bpf for fast packet filtering when you add an interface in monitor mode. Nevertheless I guess there could be other use cases not limited to frame filtering. My primary goal with this series is to collect ideas/concerns on WiFi XDP/eBPF possible uses cases. Regards, Lorenzo > -Toke
On Wed, 28 Nov 2018 13:36:26 +0100 Toke Høiland-Jørgensen <toke@toke.dk> wrote: > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > >> > >> > This series is intended as a playground to start experimenting/developing > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > >> > actions are: > >> > - XDP_PASS > >> > - XDP_ABORTED > >> > - XDP_DROP > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf > >> > program into low level driver XDP rx hook. > >> > This series has been tested through a simple bpf program (available here: > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > >> > used to count frame types received by the device. > >> > Possible eBPF use cases could be: > >> > - implement new statistics through bpf maps > >> > - implement fast packet filtering (e.g in monitor mode) > >> > - ... > > > > Hi Kalle, > > > >> > >> This is most likely a stupid question, but why do this in the driver and > >> not in mac80211 so that all drivers could benefit from it? I guess there > >> are reasons for that, I just can't figure that out. > > XDP achieves its speedup by running the eBPF program inside the driver > NAPI loop, before the kernel even touches the data in any other capacity > (and in particular, before it allocates an SKB). Which kinda means the > hook needs to be in the driver... Could be a fallback in mac80211, > though; although we'd have to figure out how that interacts with Generic > XDP. > > > This is an early stage implementation, at this point I would collect > > other people opinions/concerns about using bpf/xdp directly on 802.11 > > frames. > > Thanks for looking into this! > > I have two concerns with running XDP on 802.11 frames: > > 1. It makes it more difficult to add other XDP actions (such as > REDIRECT), as the XDP program would then have to make sure that the > outer packet headers are removed before, say, redirecting the packet > out of an ethernet interface. Also, if we do add redirect, we would > be bypassing mac80211 entirely; to what extent would that mess up > internal state? > > 2. UI consistency; suddenly, the user needs to know which kind of > frames to expect, and XDP program reuse becomes more difficult. This > may be unavoidable given the nature of XDP, but some thought needs to > go into this. Especially since we wouldn't necessarily be consistent > between WiFi drivers (there are fullmac devices that remove 802.11 > headers before sending up the frame, right?). > > > Adding in Jesper; maybe he has some thoughts on this? Today XDP assumes the frame is an Ethernet frame. With WiFi I guess this assumption change, right? I worry a bit about this, as XDP is all about performance, and I don't want to add performance regressions, by requiring all XDP programs or core-code to having to check-frame-type before proceeding. That said, I do think it is doable, without adding performance regressions. Option #1 is to move the check-frame-type to setup time. By either having frame-type be part of eBPF prog, or supply frame-type as option XDP attach call. And then reject attaching XDP prog to a device, where the expected frame-type does not match. Option#2, leave it up to eBPF-programmer if they want to add runtime checks. By extending xdp_rxq_info with frame-type (default to Ethernet), which allow the eBPF-programmer choose to write a generic XDP program that both work on Ethernet and WiFi, or skip-check as they know this will e.g. only run on Wifi. (Note xdp_rxq_info is static read-only info per RX-queue, will all Wifi frames have same frame-type?. Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to an Ethernet NIC. It would of-cause be cool to get this working cross, Wifi-Ethernet. Option#3 is to say, Wifi XDP is so different that we should create a new (enum) bpf_prog_type. And then still see if we can leverage some of the same core-code (as long as it doesn't slowdown performance).
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > On Nov 28, Toke Høiland-Jørgensen wrote: >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> >> >> >> > This series is intended as a playground to start experimenting/developing >> >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. >> >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported >> >> >> >> > actions are: >> >> >> >> > - XDP_PASS >> >> >> >> > - XDP_ABORTED >> >> >> >> > - XDP_DROP >> >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf >> >> >> >> > program into low level driver XDP rx hook. >> >> >> >> > This series has been tested through a simple bpf program (available here: >> >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) >> >> >> >> > used to count frame types received by the device. >> >> >> >> > Possible eBPF use cases could be: >> >> >> >> > - implement new statistics through bpf maps >> >> >> >> > - implement fast packet filtering (e.g in monitor mode) >> >> >> >> > - ... >> >> >> > >> >> >> > Hi Kalle, >> >> >> > >> >> >> >> >> >> >> >> This is most likely a stupid question, but why do this in the driver and >> >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there >> >> >> >> are reasons for that, I just can't figure that out. >> >> >> >> >> >> XDP achieves its speedup by running the eBPF program inside the driver >> >> >> NAPI loop, before the kernel even touches the data in any other capacity >> >> >> (and in particular, before it allocates an SKB). Which kinda means the >> >> >> hook needs to be in the driver... Could be a fallback in mac80211, >> >> >> though; although we'd have to figure out how that interacts with Generic >> >> >> XDP. >> >> >> >> >> >> > This is an early stage implementation, at this point I would collect >> >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11 >> >> >> > frames. >> >> >> >> >> >> Thanks for looking into this! >> >> > >> >> > Hi Toke, >> >> > >> >> >> >> >> >> I have two concerns with running XDP on 802.11 frames: >> >> >> >> >> >> 1. It makes it more difficult to add other XDP actions (such as >> >> >> REDIRECT), as the XDP program would then have to make sure that the >> >> >> outer packet headers are removed before, say, redirecting the packet >> >> >> out of an ethernet interface. Also, if we do add redirect, we would >> >> >> be bypassing mac80211 entirely; to what extent would that mess up >> >> >> internal state? >> >> >> >> >> > >> >> > You are right, my assumption here is the logic/complexity is moved to >> >> > the bpf program that needs to take care of all possible issues that >> >> > can be introduced. More or less it is the same if a bpf program mess >> >> > up with TCP segments on a wired connection, isn't it? >> >> >> >> No, I guess not; except here it potentially applies to all packets >> >> (things like BAW tracking), and it is *in addition* to TCP. >> > >> > Yes, here it is a little bit harder, but I was meaning that the bpf program >> > has to be very careful when dropping a packet :) >> >> Yeah. What kind of filtering were you thinking you would use this for in >> the short term? >> > > When I started working on XDP for mt76 I was thinking about BSSID > filtering but I was looking for a more general solution respect to add > that feature in the driver. Moreover we could use bpf for fast packet > filtering when you add an interface in monitor mode. Yup, both of these make sense. > Nevertheless I guess there could be other use cases not limited to > frame filtering. My primary goal with this series is to collect > ideas/concerns on WiFi XDP/eBPF possible uses cases. Well, Michał's idea about offloading is cool if it is possible to get vendors to implement it. Other than that, if we can solve the issues with differences between 802.11 and plain Ethernet frames, I see no reason why it wouldn't be possible to implement an XDP fast-path for WiFi-to-Ethernet forwarding, which might be useful in an access point, especially as WiFi speeds increase. The other direction will probably be more difficult, at least if 802.11 frames need to be built in software. It *might* be possible with the XDP egress hook we are planning (with a suitable set of helpers, the eBPF program could build the 802.11 frames), but I'm not really sure if that is worth doing as I'm quite sure there are some hairy edge cases there... -Toke
> On Wed, 28 Nov 2018 13:36:26 +0100 > Toke Høiland-Jørgensen <toke@toke.dk> wrote: > > > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > > > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > >> > > >> > This series is intended as a playground to start experimenting/developing > > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. > > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > > >> > actions are: > > >> > - XDP_PASS > > >> > - XDP_ABORTED > > >> > - XDP_DROP > > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf > > >> > program into low level driver XDP rx hook. > > >> > This series has been tested through a simple bpf program (available here: > > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > > >> > used to count frame types received by the device. > > >> > Possible eBPF use cases could be: > > >> > - implement new statistics through bpf maps > > >> > - implement fast packet filtering (e.g in monitor mode) > > >> > - ... > > > > > > Hi Kalle, > > > > > >> > > >> This is most likely a stupid question, but why do this in the driver and > > >> not in mac80211 so that all drivers could benefit from it? I guess there > > >> are reasons for that, I just can't figure that out. > > > > XDP achieves its speedup by running the eBPF program inside the driver > > NAPI loop, before the kernel even touches the data in any other capacity > > (and in particular, before it allocates an SKB). Which kinda means the > > hook needs to be in the driver... Could be a fallback in mac80211, > > though; although we'd have to figure out how that interacts with Generic > > XDP. > > > > > This is an early stage implementation, at this point I would collect > > > other people opinions/concerns about using bpf/xdp directly on 802.11 > > > frames. > > > > Thanks for looking into this! > > > > I have two concerns with running XDP on 802.11 frames: > > > > 1. It makes it more difficult to add other XDP actions (such as > > REDIRECT), as the XDP program would then have to make sure that the > > outer packet headers are removed before, say, redirecting the packet > > out of an ethernet interface. Also, if we do add redirect, we would > > be bypassing mac80211 entirely; to what extent would that mess up > > internal state? > > > > 2. UI consistency; suddenly, the user needs to know which kind of > > frames to expect, and XDP program reuse becomes more difficult. This > > may be unavoidable given the nature of XDP, but some thought needs to > > go into this. Especially since we wouldn't necessarily be consistent > > between WiFi drivers (there are fullmac devices that remove 802.11 > > headers before sending up the frame, right?). > > > > > > Adding in Jesper; maybe he has some thoughts on this? Hi Jesper, > > Today XDP assumes the frame is an Ethernet frame. With WiFi I guess > this assumption change, right? yes correct, SoftMAC devices report 802.11 frames to the stack > I worry a bit about this, as XDP is all about performance, and I don't > want to add performance regressions, by requiring all XDP programs or > core-code to having to check-frame-type before proceeding. That said, I > do think it is doable, without adding performance regressions. > > Option #1 is to move the check-frame-type to setup time. By either > having frame-type be part of eBPF prog, or supply frame-type as option > XDP attach call. And then reject attaching XDP prog to a device, where > the expected frame-type does not match. > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on a 802.11 netdevice (and vice versa). We could add a flag (or something similar) in XDP_SETUP_PROG section of netdev_bpf data structure and use ieee80211_ptr netdevice pointer in order to guarantee that the bpf program will work on the expected 'frame-type' > Option#2, leave it up to eBPF-programmer if they want to add runtime > checks. By extending xdp_rxq_info with frame-type (default to > Ethernet), which allow the eBPF-programmer choose to write a generic > XDP program that both work on Ethernet and WiFi, or skip-check as they > know this will e.g. only run on Wifi. (Note xdp_rxq_info is static > read-only info per RX-queue, will all Wifi frames have same frame-type?. > 802.11 standards define three frame subtype (data, management and control). Subtypes could be detected parsing 802.11 header > > Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to > an Ethernet NIC. It would of-cause be cool to get this working cross, > Wifi-Ethernet. > Very cool :) On tx side the driver will accept standard ethernet frames in ndo_xdp_xmit pointer > Option#3 is to say, Wifi XDP is so different that we should create a > new (enum) bpf_prog_type. And then still see if we can leverage some > of the same core-code (as long as it doesn't slowdown performance). > Do you think that Option#3 will be more 'future-proof' respect to Option#1? Regards, Lorenzo > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer
> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > > On Nov 28, Toke Høiland-Jørgensen wrote: > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > >> > >> >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > >> >> > >> >> >> >> > This series is intended as a playground to start experimenting/developing > >> >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. > >> >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > >> >> >> >> > actions are: > >> >> >> >> > - XDP_PASS > >> >> >> >> > - XDP_ABORTED > >> >> >> >> > - XDP_DROP > >> >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf > >> >> >> >> > program into low level driver XDP rx hook. > >> >> >> >> > This series has been tested through a simple bpf program (available here: > >> >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > >> >> >> >> > used to count frame types received by the device. > >> >> >> >> > Possible eBPF use cases could be: > >> >> >> >> > - implement new statistics through bpf maps > >> >> >> >> > - implement fast packet filtering (e.g in monitor mode) > >> >> >> >> > - ... > >> >> >> > > >> >> >> > Hi Kalle, > >> >> >> > > >> >> >> >> > >> >> >> >> This is most likely a stupid question, but why do this in the driver and > >> >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there > >> >> >> >> are reasons for that, I just can't figure that out. > >> >> >> > >> >> >> XDP achieves its speedup by running the eBPF program inside the driver > >> >> >> NAPI loop, before the kernel even touches the data in any other capacity > >> >> >> (and in particular, before it allocates an SKB). Which kinda means the > >> >> >> hook needs to be in the driver... Could be a fallback in mac80211, > >> >> >> though; although we'd have to figure out how that interacts with Generic > >> >> >> XDP. > >> >> >> > >> >> >> > This is an early stage implementation, at this point I would collect > >> >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11 > >> >> >> > frames. > >> >> >> > >> >> >> Thanks for looking into this! > >> >> > > >> >> > Hi Toke, > >> >> > > >> >> >> > >> >> >> I have two concerns with running XDP on 802.11 frames: > >> >> >> > >> >> >> 1. It makes it more difficult to add other XDP actions (such as > >> >> >> REDIRECT), as the XDP program would then have to make sure that the > >> >> >> outer packet headers are removed before, say, redirecting the packet > >> >> >> out of an ethernet interface. Also, if we do add redirect, we would > >> >> >> be bypassing mac80211 entirely; to what extent would that mess up > >> >> >> internal state? > >> >> >> > >> >> > > >> >> > You are right, my assumption here is the logic/complexity is moved to > >> >> > the bpf program that needs to take care of all possible issues that > >> >> > can be introduced. More or less it is the same if a bpf program mess > >> >> > up with TCP segments on a wired connection, isn't it? > >> >> > >> >> No, I guess not; except here it potentially applies to all packets > >> >> (things like BAW tracking), and it is *in addition* to TCP. > >> > > >> > Yes, here it is a little bit harder, but I was meaning that the bpf program > >> > has to be very careful when dropping a packet :) > >> > >> Yeah. What kind of filtering were you thinking you would use this for in > >> the short term? > >> > > > > When I started working on XDP for mt76 I was thinking about BSSID > > filtering but I was looking for a more general solution respect to add > > that feature in the driver. Moreover we could use bpf for fast packet > > filtering when you add an interface in monitor mode. > > Yup, both of these make sense. > > > Nevertheless I guess there could be other use cases not limited to > > frame filtering. My primary goal with this series is to collect > > ideas/concerns on WiFi XDP/eBPF possible uses cases. > > Well, Michał's idea about offloading is cool if it is possible to get > vendors to implement it. > Yep, would be very cool :) > Other than that, if we can solve the issues with differences between > 802.11 and plain Ethernet frames, I see no reason why it wouldn't be > possible to implement an XDP fast-path for WiFi-to-Ethernet forwarding, > which might be useful in an access point, especially as WiFi speeds > increase. > Agree > The other direction will probably be more difficult, at least if 802.11 > frames need to be built in software. It *might* be possible with the XDP > egress hook we are planning (with a suitable set of helpers, the eBPF > program could build the 802.11 frames), but I'm not really sure if that > is worth doing as I'm quite sure there are some hairy edge cases > there... The possible issue with XDP_DROP action you are referring to here is A-MPDU reordering on rx side, right? If so I guess the issue will be fixed by tid_agg_rx->reorder_timer. Are you referring to other possible edge cases? Regards, Lorenzo > > -Toke
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> On Wed, 28 Nov 2018 13:36:26 +0100 >> Toke Høiland-Jørgensen <toke@toke.dk> wrote: >> >> > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> > >> > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> > >> >> > >> > This series is intended as a playground to start experimenting/developing >> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. >> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported >> > >> > actions are: >> > >> > - XDP_PASS >> > >> > - XDP_ABORTED >> > >> > - XDP_DROP >> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf >> > >> > program into low level driver XDP rx hook. >> > >> > This series has been tested through a simple bpf program (available here: >> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) >> > >> > used to count frame types received by the device. >> > >> > Possible eBPF use cases could be: >> > >> > - implement new statistics through bpf maps >> > >> > - implement fast packet filtering (e.g in monitor mode) >> > >> > - ... >> > > >> > > Hi Kalle, >> > > >> > >> >> > >> This is most likely a stupid question, but why do this in the driver and >> > >> not in mac80211 so that all drivers could benefit from it? I guess there >> > >> are reasons for that, I just can't figure that out. >> > >> > XDP achieves its speedup by running the eBPF program inside the driver >> > NAPI loop, before the kernel even touches the data in any other capacity >> > (and in particular, before it allocates an SKB). Which kinda means the >> > hook needs to be in the driver... Could be a fallback in mac80211, >> > though; although we'd have to figure out how that interacts with Generic >> > XDP. >> > >> > > This is an early stage implementation, at this point I would collect >> > > other people opinions/concerns about using bpf/xdp directly on 802.11 >> > > frames. >> > >> > Thanks for looking into this! >> > >> > I have two concerns with running XDP on 802.11 frames: >> > >> > 1. It makes it more difficult to add other XDP actions (such as >> > REDIRECT), as the XDP program would then have to make sure that the >> > outer packet headers are removed before, say, redirecting the packet >> > out of an ethernet interface. Also, if we do add redirect, we would >> > be bypassing mac80211 entirely; to what extent would that mess up >> > internal state? >> > >> > 2. UI consistency; suddenly, the user needs to know which kind of >> > frames to expect, and XDP program reuse becomes more difficult. This >> > may be unavoidable given the nature of XDP, but some thought needs to >> > go into this. Especially since we wouldn't necessarily be consistent >> > between WiFi drivers (there are fullmac devices that remove 802.11 >> > headers before sending up the frame, right?). >> > >> > >> > Adding in Jesper; maybe he has some thoughts on this? > > Hi Jesper, > >> >> Today XDP assumes the frame is an Ethernet frame. With WiFi I guess >> this assumption change, right? > > yes correct, SoftMAC devices report 802.11 frames to the stack > >> I worry a bit about this, as XDP is all about performance, and I don't >> want to add performance regressions, by requiring all XDP programs or >> core-code to having to check-frame-type before proceeding. That said, I >> do think it is doable, without adding performance regressions. >> >> Option #1 is to move the check-frame-type to setup time. By either >> having frame-type be part of eBPF prog, or supply frame-type as option >> XDP attach call. And then reject attaching XDP prog to a device, where >> the expected frame-type does not match. >> > > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on > a 802.11 netdevice (and vice versa). We could add a flag (or something > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf > program will work on the expected 'frame-type' Yeah, a flag would be good; we've been discussing that for other XDP use cases; it's not a done deal yet, but I think it would be useful. > >> Option#2, leave it up to eBPF-programmer if they want to add runtime >> checks. By extending xdp_rxq_info with frame-type (default to >> Ethernet), which allow the eBPF-programmer choose to write a generic >> XDP program that both work on Ethernet and WiFi, or skip-check as they >> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static >> read-only info per RX-queue, will all Wifi frames have same frame-type?. >> > > 802.11 standards define three frame subtype (data, management and control). > Subtypes could be detected parsing 802.11 header > >> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to >> an Ethernet NIC. It would of-cause be cool to get this working cross, >> Wifi-Ethernet. >> > > Very cool :) On tx side the driver will accept standard ethernet frames in > ndo_xdp_xmit pointer How do you envision that will work with drivers that build software 802.11 frames? The TX hook would have to be in mac80211 somewhere, wouldn't it? >> Option#3 is to say, Wifi XDP is so different that we should create a >> new (enum) bpf_prog_type. And then still see if we can leverage some >> of the same core-code (as long as it doesn't slowdown performance). >> > > Do you think that Option#3 will be more 'future-proof' respect to > Option#1? My feeling is that WiFi devices are not sufficiently different to warrant a whole new program type. We risk combinatorial explosion for all the stuff that is the same, but now need to be tested for two (or N) types... -Toke
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> > On Nov 28, Toke Høiland-Jørgensen wrote: >> >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> >> >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> >> >> >> >> >> > This series is intended as a playground to start experimenting/developing >> >> >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. >> >> >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported >> >> >> >> >> > actions are: >> >> >> >> >> > - XDP_PASS >> >> >> >> >> > - XDP_ABORTED >> >> >> >> >> > - XDP_DROP >> >> >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf >> >> >> >> >> > program into low level driver XDP rx hook. >> >> >> >> >> > This series has been tested through a simple bpf program (available here: >> >> >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) >> >> >> >> >> > used to count frame types received by the device. >> >> >> >> >> > Possible eBPF use cases could be: >> >> >> >> >> > - implement new statistics through bpf maps >> >> >> >> >> > - implement fast packet filtering (e.g in monitor mode) >> >> >> >> >> > - ... >> >> >> >> > >> >> >> >> > Hi Kalle, >> >> >> >> > >> >> >> >> >> >> >> >> >> >> This is most likely a stupid question, but why do this in the driver and >> >> >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there >> >> >> >> >> are reasons for that, I just can't figure that out. >> >> >> >> >> >> >> >> XDP achieves its speedup by running the eBPF program inside the driver >> >> >> >> NAPI loop, before the kernel even touches the data in any other capacity >> >> >> >> (and in particular, before it allocates an SKB). Which kinda means the >> >> >> >> hook needs to be in the driver... Could be a fallback in mac80211, >> >> >> >> though; although we'd have to figure out how that interacts with Generic >> >> >> >> XDP. >> >> >> >> >> >> >> >> > This is an early stage implementation, at this point I would collect >> >> >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11 >> >> >> >> > frames. >> >> >> >> >> >> >> >> Thanks for looking into this! >> >> >> > >> >> >> > Hi Toke, >> >> >> > >> >> >> >> >> >> >> >> I have two concerns with running XDP on 802.11 frames: >> >> >> >> >> >> >> >> 1. It makes it more difficult to add other XDP actions (such as >> >> >> >> REDIRECT), as the XDP program would then have to make sure that the >> >> >> >> outer packet headers are removed before, say, redirecting the packet >> >> >> >> out of an ethernet interface. Also, if we do add redirect, we would >> >> >> >> be bypassing mac80211 entirely; to what extent would that mess up >> >> >> >> internal state? >> >> >> >> >> >> >> > >> >> >> > You are right, my assumption here is the logic/complexity is moved to >> >> >> > the bpf program that needs to take care of all possible issues that >> >> >> > can be introduced. More or less it is the same if a bpf program mess >> >> >> > up with TCP segments on a wired connection, isn't it? >> >> >> >> >> >> No, I guess not; except here it potentially applies to all packets >> >> >> (things like BAW tracking), and it is *in addition* to TCP. >> >> > >> >> > Yes, here it is a little bit harder, but I was meaning that the bpf program >> >> > has to be very careful when dropping a packet :) >> >> >> >> Yeah. What kind of filtering were you thinking you would use this for in >> >> the short term? >> >> >> > >> > When I started working on XDP for mt76 I was thinking about BSSID >> > filtering but I was looking for a more general solution respect to add >> > that feature in the driver. Moreover we could use bpf for fast packet >> > filtering when you add an interface in monitor mode. >> >> Yup, both of these make sense. >> >> > Nevertheless I guess there could be other use cases not limited to >> > frame filtering. My primary goal with this series is to collect >> > ideas/concerns on WiFi XDP/eBPF possible uses cases. >> >> Well, Michał's idea about offloading is cool if it is possible to get >> vendors to implement it. >> > > Yep, would be very cool :) > >> Other than that, if we can solve the issues with differences between >> 802.11 and plain Ethernet frames, I see no reason why it wouldn't be >> possible to implement an XDP fast-path for WiFi-to-Ethernet forwarding, >> which might be useful in an access point, especially as WiFi speeds >> increase. >> > > Agree > >> The other direction will probably be more difficult, at least if 802.11 >> frames need to be built in software. It *might* be possible with the XDP >> egress hook we are planning (with a suitable set of helpers, the eBPF >> program could build the 802.11 frames), but I'm not really sure if that >> is worth doing as I'm quite sure there are some hairy edge cases >> there... > > The possible issue with XDP_DROP action you are referring to here is > A-MPDU reordering on rx side, right? If so I guess the issue will be > fixed by tid_agg_rx->reorder_timer. Are you referring to other > possible edge cases? I'm not sure, which is why I was being deliberately vague... ;) Just something to be aware of when going beyond proof of concept, I guess: here be dragons. -Toke
On Thu, 29 Nov 2018 at 14:31, Toke Høiland-Jørgensen <toke@toke.dk> wrote: [...] > >> Option#3 is to say, Wifi XDP is so different that we should create a > >> new (enum) bpf_prog_type. And then still see if we can leverage some > >> of the same core-code (as long as it doesn't slowdown performance). > >> > > > > Do you think that Option#3 will be more 'future-proof' respect to > > Option#1? > > My feeling is that WiFi devices are not sufficiently different to > warrant a whole new program type. We risk combinatorial explosion for > all the stuff that is the same, but now need to be tested for two (or N) > types... I'm not sure if my understanding is correct, but XDP seems like it can (and intends to be able to?) act as a general purpose packet accelerator (REDIRECT action was mentioned so I'm inferring..). In such case you'll need to be able to perform transformations on packets too, e.g. strip/prepend vlan tags, gre headers and what have you. The 802.3 <-> 802.11 conversion could be treated on equal terms. Michał
On Thu, 29 Nov 2018 at 14:00, Lorenzo Bianconi <lorenzo.bianconi@redhat.com> wrote: [...] > > The other direction will probably be more difficult, at least if 802.11 > > frames need to be built in software. It *might* be possible with the XDP > > egress hook we are planning (with a suitable set of helpers, the eBPF > > program could build the 802.11 frames), but I'm not really sure if that > > is worth doing as I'm quite sure there are some hairy edge cases > > there... > > The possible issue with XDP_DROP action you are referring to here is A-MPDU > reordering on rx side, right? If so I guess the issue will be fixed by > tid_agg_rx->reorder_timer. Are you referring to other possible edge cases? What I'm thinking is reordering could be one of possible things to offload to an XDP program. It would require per-station data structure to keep track of the frame sequence numbers (among other things). Same could be said for crypto offloads (would require XDP programs to be able to use crypto apis I guess?). Michał
Michał Kazior <kazikcz@gmail.com> writes: > On Thu, 29 Nov 2018 at 14:31, Toke Høiland-Jørgensen <toke@toke.dk> wrote: > [...] >> >> Option#3 is to say, Wifi XDP is so different that we should create a >> >> new (enum) bpf_prog_type. And then still see if we can leverage some >> >> of the same core-code (as long as it doesn't slowdown performance). >> >> >> > >> > Do you think that Option#3 will be more 'future-proof' respect to >> > Option#1? >> >> My feeling is that WiFi devices are not sufficiently different to >> warrant a whole new program type. We risk combinatorial explosion for >> all the stuff that is the same, but now need to be tested for two (or N) >> types... > > I'm not sure if my understanding is correct, but XDP seems like it can > (and intends to be able to?) act as a general purpose packet > accelerator (REDIRECT action was mentioned so I'm inferring..). In > such case you'll need to be able to perform transformations on packets > too, e.g. strip/prepend vlan tags, gre headers and what have you. The > 802.3 <-> 802.11 conversion could be treated on equal terms. Sure, that is perfectly possible. It just needs to be implemented by the eBPF program being loaded; and there is a facility to shrink/expand the packet size for encapsulation processing. It's just that since currently all interfaces process Ethernet frames, this frame type assumption has kinda been built in. So something needs to be done to handle that. The minimum is just to ignore the issue and make it up to the program writer to handle this or risk breaking things, but something a bit friendlier (such as an ability for the loaded program to indicate which frame type it is prepared to handle, which can be sanity-checked by the loader) might be a good idea... Especially if different WiFi devices will emit different frame types (so we can't just go "it's a WiFi device, you should know this"). Then there's the additional issue that since mac80211 and/or the driver may have internal state, we need to expose appropriate helpers for an XDP program to be able to update this as it's processing packets (before bypassing the stack, for instance). -Toke
Michał Kazior <kazikcz@gmail.com> writes: > On Thu, 29 Nov 2018 at 14:00, Lorenzo Bianconi > <lorenzo.bianconi@redhat.com> wrote: > [...] >> > The other direction will probably be more difficult, at least if 802.11 >> > frames need to be built in software. It *might* be possible with the XDP >> > egress hook we are planning (with a suitable set of helpers, the eBPF >> > program could build the 802.11 frames), but I'm not really sure if that >> > is worth doing as I'm quite sure there are some hairy edge cases >> > there... >> >> The possible issue with XDP_DROP action you are referring to here is A-MPDU >> reordering on rx side, right? If so I guess the issue will be fixed by >> tid_agg_rx->reorder_timer. Are you referring to other possible edge cases? > > What I'm thinking is reordering could be one of possible things to > offload to an XDP program. It would require per-station data structure > to keep track of the frame sequence numbers (among other things). Same > could be said for crypto offloads (would require XDP programs to be > able to use crypto apis I guess?). In principle, all of this could be done. But we need to think carefully about what things it really makes sense to offload to XDP. In the general case, we will end up re-implementing all of mac80211 in eBPF, which is obviously not ideal. However, fast-path handling in XDP, which will punt to the full stack on edge cases, is probably doable (and is the general model we envision for XDP programs). For crypto in particular, I wonder if there is much of a speedup to be had if the crypto is in software anyway. Wouldn't that dominate the processing time? Whereas, if the crypto is offloaded to hardware, fast-path packet processing in XDP might make sense. This would translate to the fallback mode I mention above: If hardware crypto is enabled, handle in XDP fast-path, otherwise punt to mac80211 for full (crypto) processing. -Toke
> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > >> On Wed, 28 Nov 2018 13:36:26 +0100 > >> Toke Høiland-Jørgensen <toke@toke.dk> wrote: > >> > >> > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > >> > > >> > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > >> > >> > >> > >> > This series is intended as a playground to start experimenting/developing > >> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. > >> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported > >> > >> > actions are: > >> > >> > - XDP_PASS > >> > >> > - XDP_ABORTED > >> > >> > - XDP_DROP > >> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf > >> > >> > program into low level driver XDP rx hook. > >> > >> > This series has been tested through a simple bpf program (available here: > >> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) > >> > >> > used to count frame types received by the device. > >> > >> > Possible eBPF use cases could be: > >> > >> > - implement new statistics through bpf maps > >> > >> > - implement fast packet filtering (e.g in monitor mode) > >> > >> > - ... > >> > > > >> > > Hi Kalle, > >> > > > >> > >> > >> > >> This is most likely a stupid question, but why do this in the driver and > >> > >> not in mac80211 so that all drivers could benefit from it? I guess there > >> > >> are reasons for that, I just can't figure that out. > >> > > >> > XDP achieves its speedup by running the eBPF program inside the driver > >> > NAPI loop, before the kernel even touches the data in any other capacity > >> > (and in particular, before it allocates an SKB). Which kinda means the > >> > hook needs to be in the driver... Could be a fallback in mac80211, > >> > though; although we'd have to figure out how that interacts with Generic > >> > XDP. > >> > > >> > > This is an early stage implementation, at this point I would collect > >> > > other people opinions/concerns about using bpf/xdp directly on 802.11 > >> > > frames. > >> > > >> > Thanks for looking into this! > >> > > >> > I have two concerns with running XDP on 802.11 frames: > >> > > >> > 1. It makes it more difficult to add other XDP actions (such as > >> > REDIRECT), as the XDP program would then have to make sure that the > >> > outer packet headers are removed before, say, redirecting the packet > >> > out of an ethernet interface. Also, if we do add redirect, we would > >> > be bypassing mac80211 entirely; to what extent would that mess up > >> > internal state? > >> > > >> > 2. UI consistency; suddenly, the user needs to know which kind of > >> > frames to expect, and XDP program reuse becomes more difficult. This > >> > may be unavoidable given the nature of XDP, but some thought needs to > >> > go into this. Especially since we wouldn't necessarily be consistent > >> > between WiFi drivers (there are fullmac devices that remove 802.11 > >> > headers before sending up the frame, right?). > >> > > >> > > >> > Adding in Jesper; maybe he has some thoughts on this? > > > > Hi Jesper, > > > >> > >> Today XDP assumes the frame is an Ethernet frame. With WiFi I guess > >> this assumption change, right? > > > > yes correct, SoftMAC devices report 802.11 frames to the stack > > > >> I worry a bit about this, as XDP is all about performance, and I don't > >> want to add performance regressions, by requiring all XDP programs or > >> core-code to having to check-frame-type before proceeding. That said, I > >> do think it is doable, without adding performance regressions. > >> > >> Option #1 is to move the check-frame-type to setup time. By either > >> having frame-type be part of eBPF prog, or supply frame-type as option > >> XDP attach call. And then reject attaching XDP prog to a device, where > >> the expected frame-type does not match. > >> > > > > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on > > a 802.11 netdevice (and vice versa). We could add a flag (or something > > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and > > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf > > program will work on the expected 'frame-type' > > Yeah, a flag would be good; we've been discussing that for other XDP use > cases; it's not a done deal yet, but I think it would be useful. Do you think something wifi specific is ok (e.g bool wifi) or do you prefer something more general (e.g u32 frame_type)? > > > >> Option#2, leave it up to eBPF-programmer if they want to add runtime > >> checks. By extending xdp_rxq_info with frame-type (default to > >> Ethernet), which allow the eBPF-programmer choose to write a generic > >> XDP program that both work on Ethernet and WiFi, or skip-check as they > >> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static > >> read-only info per RX-queue, will all Wifi frames have same frame-type?. > >> > > > > 802.11 standards define three frame subtype (data, management and control). > > Subtypes could be detected parsing 802.11 header > > > >> > >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to > >> an Ethernet NIC. It would of-cause be cool to get this working cross, > >> Wifi-Ethernet. > >> > > > > Very cool :) On tx side the driver will accept standard ethernet frames in > > ndo_xdp_xmit pointer > > How do you envision that will work with drivers that build software > 802.11 frames? The TX hook would have to be in mac80211 somewhere, > wouldn't it? In order to perform 802.3 --> 802.11 xdp forwarding my current idea is is to have ndo_xdp_xmit pointer in mac80211 that will forward the frame to the low-level driver (more or less what I did in the RFC series to upload the bpf program to mt76). We will probably need to pass some info to the driver from mac80211 (e.g sequence number or hw key idx to use) > > >> Option#3 is to say, Wifi XDP is so different that we should create a > >> new (enum) bpf_prog_type. And then still see if we can leverage some > >> of the same core-code (as long as it doesn't slowdown performance). > >> > > > > Do you think that Option#3 will be more 'future-proof' respect to > > Option#1? > > My feeling is that WiFi devices are not sufficiently different to > warrant a whole new program type. We risk combinatorial explosion for > all the stuff that is the same, but now need to be tested for two (or N) > types... Agree Regards, Lorenzo > > -Toke
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> On Wed, 28 Nov 2018 13:36:26 +0100 >> >> Toke Høiland-Jørgensen <toke@toke.dk> wrote: >> >> >> >> > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> > >> >> > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> > >> >> >> > >> > This series is intended as a playground to start experimenting/developing >> >> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it. >> >> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported >> >> > >> > actions are: >> >> > >> > - XDP_PASS >> >> > >> > - XDP_ABORTED >> >> > >> > - XDP_DROP >> >> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf >> >> > >> > program into low level driver XDP rx hook. >> >> > >> > This series has been tested through a simple bpf program (available here: >> >> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats) >> >> > >> > used to count frame types received by the device. >> >> > >> > Possible eBPF use cases could be: >> >> > >> > - implement new statistics through bpf maps >> >> > >> > - implement fast packet filtering (e.g in monitor mode) >> >> > >> > - ... >> >> > > >> >> > > Hi Kalle, >> >> > > >> >> > >> >> >> > >> This is most likely a stupid question, but why do this in the driver and >> >> > >> not in mac80211 so that all drivers could benefit from it? I guess there >> >> > >> are reasons for that, I just can't figure that out. >> >> > >> >> > XDP achieves its speedup by running the eBPF program inside the driver >> >> > NAPI loop, before the kernel even touches the data in any other capacity >> >> > (and in particular, before it allocates an SKB). Which kinda means the >> >> > hook needs to be in the driver... Could be a fallback in mac80211, >> >> > though; although we'd have to figure out how that interacts with Generic >> >> > XDP. >> >> > >> >> > > This is an early stage implementation, at this point I would collect >> >> > > other people opinions/concerns about using bpf/xdp directly on 802.11 >> >> > > frames. >> >> > >> >> > Thanks for looking into this! >> >> > >> >> > I have two concerns with running XDP on 802.11 frames: >> >> > >> >> > 1. It makes it more difficult to add other XDP actions (such as >> >> > REDIRECT), as the XDP program would then have to make sure that the >> >> > outer packet headers are removed before, say, redirecting the packet >> >> > out of an ethernet interface. Also, if we do add redirect, we would >> >> > be bypassing mac80211 entirely; to what extent would that mess up >> >> > internal state? >> >> > >> >> > 2. UI consistency; suddenly, the user needs to know which kind of >> >> > frames to expect, and XDP program reuse becomes more difficult. This >> >> > may be unavoidable given the nature of XDP, but some thought needs to >> >> > go into this. Especially since we wouldn't necessarily be consistent >> >> > between WiFi drivers (there are fullmac devices that remove 802.11 >> >> > headers before sending up the frame, right?). >> >> > >> >> > >> >> > Adding in Jesper; maybe he has some thoughts on this? >> > >> > Hi Jesper, >> > >> >> >> >> Today XDP assumes the frame is an Ethernet frame. With WiFi I guess >> >> this assumption change, right? >> > >> > yes correct, SoftMAC devices report 802.11 frames to the stack >> > >> >> I worry a bit about this, as XDP is all about performance, and I don't >> >> want to add performance regressions, by requiring all XDP programs or >> >> core-code to having to check-frame-type before proceeding. That said, I >> >> do think it is doable, without adding performance regressions. >> >> >> >> Option #1 is to move the check-frame-type to setup time. By either >> >> having frame-type be part of eBPF prog, or supply frame-type as option >> >> XDP attach call. And then reject attaching XDP prog to a device, where >> >> the expected frame-type does not match. >> >> >> > >> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on >> > a 802.11 netdevice (and vice versa). We could add a flag (or something >> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and >> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf >> > program will work on the expected 'frame-type' >> >> Yeah, a flag would be good; we've been discussing that for other XDP use >> cases; it's not a done deal yet, but I think it would be useful. > > Do you think something wifi specific is ok (e.g bool wifi) or do you prefer > something more general (e.g u32 frame_type)? My thought was a feature flag where the program can set a flag which means "I expect 802.11 frames", and the driver can set a flag saying "I emit 802.11 frames", and if those two flags don't match, the verifier can refuse to load the program. This would not be fool-proof (an XDP program can still corrupt things if written incorrectly), but it would at least protect against the most obvious mistakes. >> >> Option#2, leave it up to eBPF-programmer if they want to add runtime >> >> checks. By extending xdp_rxq_info with frame-type (default to >> >> Ethernet), which allow the eBPF-programmer choose to write a generic >> >> XDP program that both work on Ethernet and WiFi, or skip-check as they >> >> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static >> >> read-only info per RX-queue, will all Wifi frames have same frame-type?. >> >> >> > >> > 802.11 standards define three frame subtype (data, management and control). >> > Subtypes could be detected parsing 802.11 header >> > >> >> >> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to >> >> an Ethernet NIC. It would of-cause be cool to get this working cross, >> >> Wifi-Ethernet. >> >> >> > >> > Very cool :) On tx side the driver will accept standard ethernet frames in >> > ndo_xdp_xmit pointer >> >> How do you envision that will work with drivers that build software >> 802.11 frames? The TX hook would have to be in mac80211 somewhere, >> wouldn't it? > > In order to perform 802.3 --> 802.11 xdp forwarding my current idea is > is to have ndo_xdp_xmit pointer in mac80211 that will forward the > frame to the low-level driver (more or less what I did in the RFC > series to upload the bpf program to mt76). We will probably need to > pass some info to the driver from mac80211 (e.g sequence number or hw > key idx to use) So this means that the driver will need to do the 802.11 encapsulation? I guess we could have a fallback implementation in mac80211; but there is possibly quite a bit of refactoring needed to make the existing code work without an skb. Also, we need to think about queueing; I'm not sure it's a good idea to have redirected frames bypass the TXQs... -Toke
> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: > >> > >> >> On Wed, 28 Nov 2018 13:36:26 +0100 > >> >> Toke Høiland-Jørgensen <toke@toke.dk> wrote: > >> >> [...] > >> > > >> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on > >> > a 802.11 netdevice (and vice versa). We could add a flag (or something > >> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and > >> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf > >> > program will work on the expected 'frame-type' > >> > >> Yeah, a flag would be good; we've been discussing that for other XDP use > >> cases; it's not a done deal yet, but I think it would be useful. > > > > Do you think something wifi specific is ok (e.g bool wifi) or do you prefer > > something more general (e.g u32 frame_type)? > > My thought was a feature flag where the program can set a flag which > means "I expect 802.11 frames", and the driver can set a flag saying "I > emit 802.11 frames", and if those two flags don't match, the verifier > can refuse to load the program. This would not be fool-proof (an XDP > program can still corrupt things if written incorrectly), but it would > at least protect against the most obvious mistakes. I guess we can use iee80211_ptr in dev_xdp_install to double check if it is allowed to upload a 802.11 (or 802.3) bpf program > > >> >> Option#2, leave it up to eBPF-programmer if they want to add runtime > >> >> checks. By extending xdp_rxq_info with frame-type (default to > >> >> Ethernet), which allow the eBPF-programmer choose to write a generic > >> >> XDP program that both work on Ethernet and WiFi, or skip-check as they > >> >> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static > >> >> read-only info per RX-queue, will all Wifi frames have same frame-type?. > >> >> > >> > > >> > 802.11 standards define three frame subtype (data, management and control). > >> > Subtypes could be detected parsing 802.11 header > >> > > >> >> > >> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to > >> >> an Ethernet NIC. It would of-cause be cool to get this working cross, > >> >> Wifi-Ethernet. > >> >> > >> > > >> > Very cool :) On tx side the driver will accept standard ethernet frames in > >> > ndo_xdp_xmit pointer > >> > >> How do you envision that will work with drivers that build software > >> 802.11 frames? The TX hook would have to be in mac80211 somewhere, > >> wouldn't it? > > > > In order to perform 802.3 --> 802.11 xdp forwarding my current idea is > > is to have ndo_xdp_xmit pointer in mac80211 that will forward the > > frame to the low-level driver (more or less what I did in the RFC > > series to upload the bpf program to mt76). We will probably need to > > pass some info to the driver from mac80211 (e.g sequence number or hw > > key idx to use) > > So this means that the driver will need to do the 802.11 encapsulation? > I guess we could have a fallback implementation in mac80211; but there > is possibly quite a bit of refactoring needed to make the existing code > work without an skb. Also, we need to think about queueing; I'm not sure > it's a good idea to have redirected frames bypass the TXQs... > good point :) Regards, Lorenzo > -Toke
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes: >> >> >> >> >> On Wed, 28 Nov 2018 13:36:26 +0100 >> >> >> Toke Høiland-Jørgensen <toke@toke.dk> wrote: >> >> >> > > [...] > >> >> > >> >> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on >> >> > a 802.11 netdevice (and vice versa). We could add a flag (or something >> >> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and >> >> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf >> >> > program will work on the expected 'frame-type' >> >> >> >> Yeah, a flag would be good; we've been discussing that for other XDP use >> >> cases; it's not a done deal yet, but I think it would be useful. >> > >> > Do you think something wifi specific is ok (e.g bool wifi) or do you prefer >> > something more general (e.g u32 frame_type)? >> >> My thought was a feature flag where the program can set a flag which >> means "I expect 802.11 frames", and the driver can set a flag saying "I >> emit 802.11 frames", and if those two flags don't match, the verifier >> can refuse to load the program. This would not be fool-proof (an XDP >> program can still corrupt things if written incorrectly), but it would >> at least protect against the most obvious mistakes. > > I guess we can use iee80211_ptr in dev_xdp_install to double check if it is > allowed to upload a 802.11 (or 802.3) bpf program Yeah, I think it's more an issue of convincing the wider XDP community that support for feature flags is in fact needed ;) -Toke
Hi Toke, all, Sorry I wasn't around for the discussion, I've been travelling and sick. I'm picking this as an arbitrary point in the discussion to reply to, hope you don't feel bad about that. >> [A-MPDU reordering] > In principle, all of this could be done. But we need to think carefully > about what things it really makes sense to offload to XDP. In the > general case, we will end up re-implementing all of mac80211 in eBPF, > which is obviously not ideal. Agree with this, completely. Mind you, it's not even simple to do that, because each kind of hardware behaves differently. See below. > For crypto in particular, I wonder if there is much of a speedup to be > had if the crypto is in software anyway. Wouldn't that dominate the > processing time? Yeah, I'd also think the cost of skb allocation is dwarfed by the cost of doing the crypto, but not really sure, maybe you have a fast special AES instruction? :) Anyway, here are my thoughts after reading most of the thread and having discussed it with Lorenzo a while back and today: 1) I'm going to restrict most of the discussion to mac80211, but there are interesting cases outside of it as well. Much of the same arguments apply, though it means *more* feature combinatorial explosion to be able to handle everything. 2) We don't yet have it (but it's been suggested for years and we might also get it on Intel hardware eventually), but if we have TX and/or RX 802.3/802.11 frame conversion offload then I think we have no argument - XDP applies pure and simple as is in each direction you have offloads for, unless you do something really stupid in HW like still having to do reorder buffer or PN checking based on RX frame metadata. I hope nobody does that, but then I suppose you could do it before the XDP hooks w/o allocating SKBs. 3) TX from XDP (i.e. X -> wifi) could be implemented in mac80211, but is subject to some rather iffy details, for example: a) mac80211 may need more headroom than the input has, do we have to copy the whole data? depending on xdp_mem_type I think we may need that, rather than being able to make an skb with frag pages and extra headroom? We don't know it was just a single page and I guess we can't ref that page anyway, but maybe we could meddle with the SKB frag data somehow to avoid that. b) mac80211 may need to software encrypt - copy the whole data c) lifetime issues - we need to call xdp_return_frame(), which I guess we can tie to the skb we'd create anyway [2] (and IMHO we need to allocate an skb anyway, due to TXQs like Toke mentioned and much other possible software handling) 4) XDP during RX - well, that's _mostly_ what this thread has been about, but really I think it's a can of worms. Consider: a) drivers may or may not do SW decryption b) drivers may or may not do PN checking - so your XDP program might end up having to do that or risk security/replay bugs! (which, btw, you can only do after reordering so see below) c) drivers may or may not do A-MSDU deaggregation, do you[1] really want to handle that in XDP? You could argue outer code should walk over the subframes, but then dropping becomes hard - and this really should only happen after a) and b) Oh, and IIRC there are cases that use A-MSDUs for single frames just to encapsulate different MAC addresses. Speaking of which, you have to validate the inner MAC addresses (but it's not a trivial comparison) or risk security bugs again. d) drivers may or may not do A-MPDU reordering (this was discussed earlier in the thread), which means even if you drop a frame you haven't just affected that connection but potentially stalled everything until the reorder buffer times out (~100ms), or we need a way for mac80211 to insert a fake skb into the reorder buffer at that spot (but then you allocate an skb again and high-speed traffic where you'd care most is always in an A-MPDU session) e) speaking of which, Intel's device has hardware assist things for reordering, but not fully offloaded reordering, so you might end up with hardware-specific ways of doing this in XDP? f) Some data frames are used internally in the stack, e.g. for TDLS. g) data frames may also affect the powersave state of a station, so if you have e.g. ath9k and the station sends a frame to wake up, but you decide you XDP_DROP it, the powersave state will get messed up, unless ... special handling (either mac80211 or the bpf program) again. Also, U-APSD. h) There are also simple things like QoS/non-QoS that add to the combinatorial explosion of things you have to handle i) data frames could be sent by anyone, not just STAs connected to your network, so I guess you'd have to filter that and mac80211 would have to expose a station table lookup helper or something? Alternatively shuffle the code around so mac80211 does that first, but then we already have an SKB anyway since we got it from the driver, and you want this stuff to run in the driver. j) in AP mode, if data frames are sent by somebody not belonging to the network, we should send an event to hostapd so it can tell them they're not on the network k) some drivers implement client-side powersave in mac80211 and need the more-data bit even if you were to XDP_DROP a frame. I hate this code with a passion, but it's there for now. m) did somebody mention fragmentation yet? and the special security handling for CCMP/GCMP there? n) I guess checking for port authorized/not would be one of the more trivial things, just adds another helper/bit/state/something. o) not supported yet by anyone (well, mac80211 at least), but the standard is adding provisions to leave out the SNAP/LLC header, which requires more out-of-band data (i.e. data not directly in the frame) p) there's also iwlwifi with it's multiqueue RX, but let's not go there now [Toke, do you still think it's not sufficiently different? ;-P] Of course with XDP anyone can shoot themselves in the foot easily, but I also think we shouldn't make it too easy to do things like XDP_REDIRECT on a frame that hasn't even had its security properties validated. It'd work, but an attacker's frame could be accepted as well. We have enough security bugs as is, I'm not sure I'd want that extra surface... Now, of course you can pick each one of these points and say "but that's not relevant for my ${favourite_hw}", which is true. Each hardware (family) will likely make *most* of fixed or not relevant. Restrict sufficiently though and you'll end up with *exactly* ${favourite_hw} doing it the way you want to assume it is like. E.g. let's say you require PN checking in hardware - maybe 2 or 3 drivers (I'm thinking ath10k, maybe mt76, maybe some ralink family)? Or let's say you want A-MPDU reordering offload to not have to worry about that and replays and all - I'm guessing only ath10k now and that perhaps even only in 802.3 mode? Or let's say you have enough filtering to not have to worry about 4j), what if you enable monitor mode at the same time? Can you tell the difference in the driver before calling the XDP program? As fun as it sounds (on paper) to have XDP available for wifi, I'm not sure we really should do that. We'd have to handle many of the issues outlined above (which is at minimum a LOT of helper code that each at least somewhat secure and portable XDP-wifi program would have to call), and it looks like in the future we'll be moving more towards offloaded 802.11/.3 frame conversion anyway, at which point it all becomes more or less useless. So I see a few ways this might go: A) We basically allow everything, and let the XDP program authors figure it out. They'll eventually learn to handle the whole mac80211 datapath (items in 4)), and will add a TON of helper code for it. B) We try to foresee all the needed helper code, and essentially build a parallel mac80211 datapath equivalent to the one today, but without SKBs. C) We rebuild the mac80211 datapath w/o SKBs so we can put XDP behind it. Ugh. D) We restrict it to ~1 driver that does most of the important things anyway in the device, thus hardcoding most of the variables, but then we expand out to A) or B) E) Like D, but then that device learns to do 802.3/.11 conversion. Oops. F) Like D, but then that *driver* learns to do 802.3/.11 conversion. Also oops. Maybe F) kind of has a good direction in it? If the device does enough that we don't have to worry about most things, then - provided we actually implement the code to allow 802.3/.11 conversion offload in mac80211 - the driver can probably easily do that itself rather than punting to mac80211's fast-RX/fast-TX, which would likely be used? Or perhaps that means we should teach mac80211 fast-RX to be skb-less, or something like that? But I think that doesn't solve the reorder buffer issue either. (Let's also not forget that the original use case was something around defective HW filters for BSSIDs - IMHO that should just be *hardcoded* into the driver anyway because all other frames are useless to you, except perhaps if also monitor mode is active but then you need an SKB!) Ok, this got longer than I wanted... johannes [1] generally speaking, no particular "you" intended [2] btw, i40e_clean_xdp_tx_buffer() looks kinda weird? it returns first and then DMA-unmaps, you'd think it should be done the other way around?
Hi Johannes
I think your email can be basically summed up to:
> [ ... ] but really I think it's a can of worms.
...right? :)
I sort of had a feeling it would be, but thank you for spelling out in
excruciating detail why that is so.
Given this, I think I agree that it's not worth it for now, and we
should hold off on adding XDP support until we have 802.3/.11 conversion
offload working... Which I think is also where you ended up? :)
-Toke
On Mon, 2018-12-03 at 21:36 +0200, Toke Høiland-Jørgensen wrote: > Hi Johannes > > I think your email can be basically summed up to: > > > [ ... ] but really I think it's a can of worms. > > ...right? :) Heh, yeah :) > I sort of had a feeling it would be, but thank you for spelling out in > excruciating detail why that is so. :-) > Given this, I think I agree that it's not worth it for now, and we > should hold off on adding XDP support until we have 802.3/.11 conversion > offload working... Which I think is also where you ended up? :) That case is at least easy, yeah. And it seems kinda likely that we'll end up with that in all well-maintained drivers in the relatively near future anyway? BTW, in a sense I still kind of want to add eBPF to the mac80211 ingress path, just not in the XDP sense. For example, I had a proposal a while ago to add a filter to the monitor mode RX path(s) in eBPF; I still think that's useful. I also think it may be useful to put eBPF programs into per-netdev ingress path, in order to e.g. collect statistics, rather than hard- coding all kinds of statistics into mac80211. All of these things I consider absolutely useful and helpful. I like eBPF and the flexibility it affords. I just really don't think we should call it XDP or let it do similar things to XDP like dropping or redirecting frames. johannes
Johannes Berg <johannes@sipsolutions.net> writes: > On Mon, 2018-12-03 at 21:36 +0200, Toke Høiland-Jørgensen wrote: >> Hi Johannes >> >> I think your email can be basically summed up to: >> >> > [ ... ] but really I think it's a can of worms. >> >> ...right? :) > > Heh, yeah :) > >> I sort of had a feeling it would be, but thank you for spelling out in >> excruciating detail why that is so. > > :-) > >> Given this, I think I agree that it's not worth it for now, and we >> should hold off on adding XDP support until we have 802.3/.11 >> conversion offload working... Which I think is also where you ended >> up? :) > > That case is at least easy, yeah. And it seems kinda likely that we'll > end up with that in all well-maintained drivers in the relatively near > future anyway? Right; you probably know that better than me :) > BTW, in a sense I still kind of want to add eBPF to the mac80211 ingress > path, just not in the XDP sense. For example, I had a proposal a while > ago to add a filter to the monitor mode RX path(s) in eBPF; I still > think that's useful. > > I also think it may be useful to put eBPF programs into per-netdev > ingress path, in order to e.g. collect statistics, rather than hard- > coding all kinds of statistics into mac80211. > > All of these things I consider absolutely useful and helpful. I like > eBPF and the flexibility it affords. I just really don't think we should > call it XDP or let it do similar things to XDP like dropping or > redirecting frames. Absolutely; totally on board with that! -Toke
> On Mon, 2018-12-03 at 21:36 +0200, Toke Høiland-Jørgensen wrote: > > Hi Johannes > > > > I think your email can be basically summed up to: > > > > > [ ... ] but really I think it's a can of worms. > > > > ...right? :) > > Heh, yeah :) > > > I sort of had a feeling it would be, but thank you for spelling out in > > excruciating detail why that is so. > > :-) > > > Given this, I think I agree that it's not worth it for now, and we > > should hold off on adding XDP support until we have 802.3/.11 conversion > > offload working... Which I think is also where you ended up? :) > > That case is at least easy, yeah. And it seems kinda likely that we'll > end up with that in all well-maintained drivers in the relatively near > future anyway? Hi all, thanks for sharing your thoughts and concerns, as already stated the main goal of this series is get feedbacks and/or blocking points to start experimenting with eBPF over 802.11 devices. Reviewing the points indicated by Johannes, 802.11 protocol is too complicated to be managed without skb allocation. I agree that for the moment XDP/eBPF can be useful for devices that supports hw offloading (or FullMac devices?) since all the tricky aspects are already managed in the firmware and we can take care of XDP stuff. Probably in the near future we will respin this argument :) > > BTW, in a sense I still kind of want to add eBPF to the mac80211 ingress > path, just not in the XDP sense. For example, I had a proposal a while > ago to add a filter to the monitor mode RX path(s) in eBPF; I still > think that's useful. > > I also think it may be useful to put eBPF programs into per-netdev > ingress path, in order to e.g. collect statistics, rather than hard- > coding all kinds of statistics into mac80211. > > All of these things I consider absolutely useful and helpful. I like > eBPF and the flexibility it affords. I just really don't think we should > call it XDP or let it do similar things to XDP like dropping or > redirecting frames. > +1 :) Regards, Lorenzo > johannes >