From patchwork Fri Oct 18 14:40:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 11198597 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 366FC112C for ; Fri, 18 Oct 2019 14:40:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 17C61222BD for ; Fri, 18 Oct 2019 14:40:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="G6jOAMwY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2408509AbfJROk3 (ORCPT ); Fri, 18 Oct 2019 10:40:29 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:59819 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2389378AbfJROk2 (ORCPT ); Fri, 18 Oct 2019 10:40:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1571409626; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Eg6GWygKeYzqmkq+LDj844ZOc9L0SpguuNdAt7HOGwI=; b=G6jOAMwYm+aL6DQegZPO0zxfRrTTcAXqafgytbDUS96JVHu9siaUm2IbM7Hal5a5TRx+Z6 OIAraMPGZoHGf9q8JdlEC6Uq7aYpjVkkR6j3MEwesjpp2Oit9xG0FazdE9iDEh1R3uUA2M i08inJE5iP/ZFVrW2xxlYKC5AbDej78= Received: from mail-lj1-f197.google.com (mail-lj1-f197.google.com [209.85.208.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-249-pPthp24zMfG9FMVpm_gpRA-1; Fri, 18 Oct 2019 10:40:25 -0400 Received: by mail-lj1-f197.google.com with SMTP id g88so1125233lje.10 for ; Fri, 18 Oct 2019 07:40:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=gQX28cV4kR5e8hFv0Sv15bTp7rYQDznh9B4z2Gx6gas=; b=LbvXW/LdFWSMvdAhuYWAe6vtE3e6YYWG1HWBaqlduo6i7kp6zndM2zGICp8229NEOg cVtrG1nVEBqs/Gzzw8fzQdBaus2RbPJCYThkPYzJl9iliMbfJPTjQwhSbnoOFG8LL2wP 4ypvhIiFfkvja/KURFEbv6IsO7SmY4fg91+c6pZAOyXgm38gnrsLVtfqTpflSC1VoyKG GD2Po5ugSiUI19c5KTfFO1kRX+neFkNOHlP13x/3oT0jIbIrdCGFsrGs+vKh/GgcP8W3 zmwZgjJYeCtwDvB0Wak9Dsp4n03MmDlYK6pHi9UOMmaMnhz880DCfGb5SwD8ryRjkGMA Wqfw== X-Gm-Message-State: APjAAAV3r40i+xd49qGFHuFfuTj+nE3v4THM6OAVcFXSYiqUySsljVSS k6f71FQ2uNiBDene2NzkY9EudNWJTixHaPulYEXN9CjczfXRwvX9b+Kzb6SjcQ8WAbP8eksjzyz OnnWPVmeNB3zi0vbS15gR6xqpgMg= X-Received: by 2002:a2e:b17b:: with SMTP id a27mr6357138ljm.7.1571409623703; Fri, 18 Oct 2019 07:40:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqx1YvkIc2YHpk1/SSCOw+Ds7+tH99z4j9/nltzLAYUnxbi1nOpFOFqviaE5EgAWL5eFjRejZw== X-Received: by 2002:a2e:b17b:: with SMTP id a27mr6357117ljm.7.1571409623508; Fri, 18 Oct 2019 07:40:23 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk (borgediget.toke.dk. [85.204.121.218]) by smtp.gmail.com with ESMTPSA id k13sm2644724ljc.96.2019.10.18.07.40.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2019 07:40:22 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 20DE21804C9; Fri, 18 Oct 2019 16:40:22 +0200 (CEST) Subject: [PATCH v3 1/4] mac80211: Shrink the size of ack_frame_id to make room for tx_time_est From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: Johannes Berg Cc: linux-wireless@vger.kernel.org, make-wifi-fast@lists.bufferbloat.net, ath10k@lists.infradead.org, John Crispin , Lorenzo Bianconi , Felix Fietkau , Kan Yan , Rajkumar Manoharan , Kevin Hayes Date: Fri, 18 Oct 2019 16:40:22 +0200 Message-ID: <157140962205.2866668.16480110695725359529.stgit@toke.dk> In-Reply-To: <157140962094.2866668.3383184317264069493.stgit@toke.dk> References: <157140962094.2866668.3383184317264069493.stgit@toke.dk> User-Agent: StGit/0.20 MIME-Version: 1.0 X-MC-Unique: pPthp24zMfG9FMVpm_gpRA-1 X-Mimecast-Spam-Score: 0 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Toke Høiland-Jørgensen To implement airtime queue limiting, we need to keep a running account of the estimated airtime of all skbs queued into the device. Do to this correctly, we need to store the airtime estimate into the skb so we can decrease the outstanding balance when the skb is freed. This means that the time estimate must be stored somewhere that will survive for the lifetime of the skb. To get this, decrease the size of the ack_frame_id field to 6 bits, and lower the size of the ID space accordingly. This leaves 10 bits for use for tx_time_est, which is enough to store a maximum of 4096 us, if we shift the values so they become units of 4us. Signed-off-by: Toke Høiland-Jørgensen --- include/net/mac80211.h | 4 +++- net/mac80211/cfg.c | 2 +- net/mac80211/tx.c | 2 +- 3 files changed, 5 insertions(+), 3 deletions(-) diff --git a/include/net/mac80211.h b/include/net/mac80211.h index d69081c38788..4288ace72c2b 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -967,6 +967,7 @@ ieee80211_rate_get_vht_nss(const struct ieee80211_tx_rate *rate) * @band: the band to transmit on (use for checking for races) * @hw_queue: HW queue to put the frame on, skb_get_queue_mapping() gives the AC * @ack_frame_id: internal frame ID for TX status, used internally + * @tx_time_est: TX time estimate in units of 4us, used internally * @control: union part for control data * @control.rates: TX rates array to try * @control.rts_cts_rate_idx: rate for RTS or CTS @@ -1007,7 +1008,8 @@ struct ieee80211_tx_info { u8 hw_queue; - u16 ack_frame_id; + u16 ack_frame_id:6; + u16 tx_time_est:10; union { struct { diff --git a/net/mac80211/cfg.c b/net/mac80211/cfg.c index 70739e746c13..4fb7f1f12109 100644 --- a/net/mac80211/cfg.c +++ b/net/mac80211/cfg.c @@ -3428,7 +3428,7 @@ int ieee80211_attach_ack_skb(struct ieee80211_local *local, struct sk_buff *skb, spin_lock_irqsave(&local->ack_status_lock, spin_flags); id = idr_alloc(&local->ack_status_frames, ack_skb, - 1, 0x10000, GFP_ATOMIC); + 1, 0x40, GFP_ATOMIC); spin_unlock_irqrestore(&local->ack_status_lock, spin_flags); if (id < 0) { diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 938c10f7955b..a16c2f863702 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -2719,7 +2719,7 @@ static struct sk_buff *ieee80211_build_hdr(struct ieee80211_sub_if_data *sdata, spin_lock_irqsave(&local->ack_status_lock, flags); id = idr_alloc(&local->ack_status_frames, ack_skb, - 1, 0x10000, GFP_ATOMIC); + 1, 0x40, GFP_ATOMIC); spin_unlock_irqrestore(&local->ack_status_lock, flags); if (id >= 0) { From patchwork Fri Oct 18 14:40:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 11198601 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9AA13112C for ; Fri, 18 Oct 2019 14:40:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6AA69222C6 for ; Fri, 18 Oct 2019 14:40:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="J9dsp02o" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2410103AbfJROka (ORCPT ); Fri, 18 Oct 2019 10:40:30 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:35072 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2389378AbfJROk3 (ORCPT ); Fri, 18 Oct 2019 10:40:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1571409629; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=037kYcrwRSTSMLRfx4CfzNcNZlFM0quGkGQoN08cL8c=; b=J9dsp02ouGPR5B3BRznrOMxhDK99vPmlhkmFILmzOONi1mSQDybgXEUb9c/d5wH9th+Czy RXdMv4vsEtUHlG9V3IK9i02OKuT9G4PFK1BGjWdESucJzwaZYyhUQUJuj8uE438mYcNPd4 HgAyZST0XZ9zJzl1nGMJwXwgz4fwU9g= Received: from mail-lf1-f69.google.com (mail-lf1-f69.google.com [209.85.167.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-237-qJetDeSuOtCJy1_V9i4-Jg-1; Fri, 18 Oct 2019 10:40:27 -0400 Received: by mail-lf1-f69.google.com with SMTP id f3so1318673lfa.16 for ; Fri, 18 Oct 2019 07:40:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=8YRb2N6dvs3xtiXuol7/wWZTQ7TYCWwqAjekk5alzxE=; b=MvEAyg0AHtRsQnrx0OS8aMo7tgSkdzRcUUnIJkX7LhDcpOMgLC0PEzJQcIhOH4JNwi tDu8HidR8rC501uloSJEdET5Lby/NI7f18CBo68Zd3jtEDnWoqEELN4rcaTH35T30lg8 n37IAWFRqG2zbQ+9KXnlcfcsBfX6CkufIlLy0vcvHtTJUFbWt+0a8MWJbbXcJRtbbbfm 35z98ro2UzId25RW+WsBHTrZbf0uZNj8v2jJ9iKEyy2bAgMzX7dKOX/Xn6RPnnVIQPhw DHBecNOJoQJwUnH0ZM/jyE8apvawOLJ0giySSsiJ0w5i9v2aUw5fyLjEvsxWrYNDI/V5 m/qA== X-Gm-Message-State: APjAAAU4p3USgXqqGhd8gXgpxGJGBlu2zXu+cSEj2qAGwnU91oy5IrEo UV9kvt8fGGynn1MPS+QjE4LU5T+vDGkVDAvI5p4Vej8Wx7nggN7fo3XfG1ljWJUtDlxPnT8QWuk JEOR2ns5U61MxOuL3qmqq15xqG4o= X-Received: by 2002:a05:6512:25c:: with SMTP id b28mr6242359lfo.35.1571409625286; Fri, 18 Oct 2019 07:40:25 -0700 (PDT) X-Google-Smtp-Source: APXvYqxvJyABn3lw1TrzJM5AUSpQnDQzGZxqESuzJeHWgUsJII7TdTdKX31hKrPUtdDqmXTeaGOaIA== X-Received: by 2002:a05:6512:25c:: with SMTP id b28mr6242326lfo.35.1571409624840; Fri, 18 Oct 2019 07:40:24 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a00:7660:6da:443::2]) by smtp.gmail.com with ESMTPSA id y2sm2231820lfl.47.2019.10.18.07.40.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2019 07:40:23 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 36BC61804B6; Fri, 18 Oct 2019 16:40:23 +0200 (CEST) Subject: [PATCH v3 2/4] mac80211: Import airtime calculation code from mt76 From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: Johannes Berg Cc: linux-wireless@vger.kernel.org, make-wifi-fast@lists.bufferbloat.net, ath10k@lists.infradead.org, John Crispin , Lorenzo Bianconi , Felix Fietkau , Kan Yan , Rajkumar Manoharan , Kevin Hayes Date: Fri, 18 Oct 2019 16:40:23 +0200 Message-ID: <157140962313.2866668.14014088713159413172.stgit@toke.dk> In-Reply-To: <157140962094.2866668.3383184317264069493.stgit@toke.dk> References: <157140962094.2866668.3383184317264069493.stgit@toke.dk> User-Agent: StGit/0.20 MIME-Version: 1.0 X-MC-Unique: qJetDeSuOtCJy1_V9i4-Jg-1 X-Mimecast-Spam-Score: 0 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Toke Høiland-Jørgensen Felix recently added code to calculate airtime of packets to the mt76 driver. Import this into mac80211 so we can use it for airtime queue limit calculations later. The airtime.c file is copied verbatim from the mt76 driver, and adjusted to use mac80211 data structures instead (which is fairly straight forward). The per-rate TX rate calculation is split out to its own function (ieee80211_calc_tx_airtime_rate()) so it can be used directly for the AQL calculations added in a subsequent patch. The only thing that it was not possible to port directly was the bit that read the internal driver flags of struct ieee80211_rate to determine whether a rate is using CCK or OFDM encoding. Instead, just look at the rate index, since at least mt76 and ath10k both seem to have the same number of CCK rates (4) in their tables. Signed-off-by: Toke Høiland-Jørgensen --- include/net/mac80211.h | 14 ++ net/mac80211/Makefile | 3 net/mac80211/airtime.c | 377 ++++++++++++++++++++++++++++++++++++++++++++ net/mac80211/ieee80211_i.h | 4 4 files changed, 397 insertions(+), 1 deletion(-) create mode 100644 net/mac80211/airtime.c diff --git a/include/net/mac80211.h b/include/net/mac80211.h index 4288ace72c2b..f058386e3fef 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -6424,4 +6424,18 @@ void ieee80211_nan_func_match(struct ieee80211_vif *vif, struct cfg80211_nan_match_params *match, gfp_t gfp); +/** + * ieee80211_calc_tx_airtime - calculate estimated transmission airtime. + * + * This function calculates the estimated airtime usage of a frame based on the + * rate information in the TX info struct and the frame length. + * + * @hw: pointer as obtained from ieee80211_alloc_hw() + * @info: &struct ieee80211_tx_info of the frame. + * @len: frame length in bytes + */ +u32 ieee80211_calc_tx_airtime(struct ieee80211_hw *hw, + struct ieee80211_tx_info *info, + int len); + #endif /* MAC80211_H */ diff --git a/net/mac80211/Makefile b/net/mac80211/Makefile index 4f03ebe732fa..6cbb1286d6c0 100644 --- a/net/mac80211/Makefile +++ b/net/mac80211/Makefile @@ -32,7 +32,8 @@ mac80211-y := \ chan.o \ trace.o mlme.o \ tdls.o \ - ocb.o + ocb.o \ + airtime.o mac80211-$(CONFIG_MAC80211_LEDS) += led.o mac80211-$(CONFIG_MAC80211_DEBUGFS) += \ diff --git a/net/mac80211/airtime.c b/net/mac80211/airtime.c new file mode 100644 index 000000000000..c8d0cee61366 --- /dev/null +++ b/net/mac80211/airtime.c @@ -0,0 +1,377 @@ +// SPDX-License-Identifier: ISC +/* + * Copyright (C) 2019 Felix Fietkau + */ + +#include +#include "ieee80211_i.h" +#include "sta_info.h" + +#define AVG_PKT_SIZE 1024 + +/* Number of bits for an average sized packet */ +#define MCS_NBITS (AVG_PKT_SIZE << 3) + +/* Number of symbols for a packet with (bps) bits per symbol */ +#define MCS_NSYMS(bps) DIV_ROUND_UP(MCS_NBITS, (bps)) + +/* Transmission time (1024 usec) for a packet containing (syms) * symbols */ +#define MCS_SYMBOL_TIME(sgi, syms) \ + (sgi ? \ + ((syms) * 18 * 1024 + 4 * 1024) / 5 : /* syms * 3.6 us */ \ + ((syms) * 1024) << 2 /* syms * 4 us */ \ + ) + +/* Transmit duration for the raw data part of an average sized packet */ +#define MCS_DURATION(streams, sgi, bps) \ + MCS_SYMBOL_TIME(sgi, MCS_NSYMS((streams) * (bps))) + +#define BW_20 0 +#define BW_40 1 +#define BW_80 2 + +/* + * Define group sort order: HT40 -> SGI -> #streams + */ +#define IEEE80211_MAX_STREAMS 4 +#define IEEE80211_HT_STREAM_GROUPS 4 /* BW(=2) * SGI(=2) */ +#define IEEE80211_VHT_STREAM_GROUPS 6 /* BW(=3) * SGI(=2) */ + +#define IEEE80211_HT_GROUPS_NB (IEEE80211_MAX_STREAMS * \ + IEEE80211_HT_STREAM_GROUPS) +#define IEEE80211_VHT_GROUPS_NB (IEEE80211_MAX_STREAMS * \ + IEEE80211_VHT_STREAM_GROUPS) +#define IEEE80211_GROUPS_NB (IEEE80211_HT_GROUPS_NB + \ + IEEE80211_VHT_GROUPS_NB) + +#define IEEE80211_HT_GROUP_0 0 +#define IEEE80211_VHT_GROUP_0 (IEEE80211_HT_GROUP_0 + IEEE80211_HT_GROUPS_NB) + +#define MCS_GROUP_RATES 10 +#define CCK_NUM_RATES 4 + +#define HT_GROUP_IDX(_streams, _sgi, _ht40) \ + IEEE80211_HT_GROUP_0 + \ + IEEE80211_MAX_STREAMS * 2 * _ht40 + \ + IEEE80211_MAX_STREAMS * _sgi + \ + _streams - 1 + +#define _MAX(a, b) (((a)>(b))?(a):(b)) + +#define GROUP_SHIFT(duration) \ + _MAX(0, 16 - __builtin_clz(duration)) + +/* MCS rate information for an MCS group */ +#define __MCS_GROUP(_streams, _sgi, _ht40, _s) \ + [HT_GROUP_IDX(_streams, _sgi, _ht40)] = { \ + .shift = _s, \ + .duration = { \ + MCS_DURATION(_streams, _sgi, _ht40 ? 54 : 26) >> _s, \ + MCS_DURATION(_streams, _sgi, _ht40 ? 108 : 52) >> _s, \ + MCS_DURATION(_streams, _sgi, _ht40 ? 162 : 78) >> _s, \ + MCS_DURATION(_streams, _sgi, _ht40 ? 216 : 104) >> _s, \ + MCS_DURATION(_streams, _sgi, _ht40 ? 324 : 156) >> _s, \ + MCS_DURATION(_streams, _sgi, _ht40 ? 432 : 208) >> _s, \ + MCS_DURATION(_streams, _sgi, _ht40 ? 486 : 234) >> _s, \ + MCS_DURATION(_streams, _sgi, _ht40 ? 540 : 260) >> _s \ + } \ +} + +#define MCS_GROUP_SHIFT(_streams, _sgi, _ht40) \ + GROUP_SHIFT(MCS_DURATION(_streams, _sgi, _ht40 ? 54 : 26)) + +#define MCS_GROUP(_streams, _sgi, _ht40) \ + __MCS_GROUP(_streams, _sgi, _ht40, \ + MCS_GROUP_SHIFT(_streams, _sgi, _ht40)) + +#define VHT_GROUP_IDX(_streams, _sgi, _bw) \ + (IEEE80211_VHT_GROUP_0 + \ + IEEE80211_MAX_STREAMS * 2 * (_bw) + \ + IEEE80211_MAX_STREAMS * (_sgi) + \ + (_streams) - 1) + +#define BW2VBPS(_bw, r3, r2, r1) \ + (_bw == BW_80 ? r3 : _bw == BW_40 ? r2 : r1) + +#define __VHT_GROUP(_streams, _sgi, _bw, _s) \ + [VHT_GROUP_IDX(_streams, _sgi, _bw)] = { \ + .shift = _s, \ + .duration = { \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 117, 54, 26)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 234, 108, 52)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 351, 162, 78)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 468, 216, 104)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 702, 324, 156)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 936, 432, 208)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 1053, 486, 234)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 1170, 540, 260)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 1404, 648, 312)) >> _s, \ + MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 1560, 720, 346)) >> _s \ + } \ +} + +#define VHT_GROUP_SHIFT(_streams, _sgi, _bw) \ + GROUP_SHIFT(MCS_DURATION(_streams, _sgi, \ + BW2VBPS(_bw, 117, 54, 26))) + +#define VHT_GROUP(_streams, _sgi, _bw) \ + __VHT_GROUP(_streams, _sgi, _bw, \ + VHT_GROUP_SHIFT(_streams, _sgi, _bw)) + +struct mcs_group { + u8 shift; + u16 duration[MCS_GROUP_RATES]; +}; + +static const struct mcs_group airtime_mcs_groups[] = { + MCS_GROUP(1, 0, BW_20), + MCS_GROUP(2, 0, BW_20), + MCS_GROUP(3, 0, BW_20), + MCS_GROUP(4, 0, BW_20), + + MCS_GROUP(1, 1, BW_20), + MCS_GROUP(2, 1, BW_20), + MCS_GROUP(3, 1, BW_20), + MCS_GROUP(4, 1, BW_20), + + MCS_GROUP(1, 0, BW_40), + MCS_GROUP(2, 0, BW_40), + MCS_GROUP(3, 0, BW_40), + MCS_GROUP(4, 0, BW_40), + + MCS_GROUP(1, 1, BW_40), + MCS_GROUP(2, 1, BW_40), + MCS_GROUP(3, 1, BW_40), + MCS_GROUP(4, 1, BW_40), + + VHT_GROUP(1, 0, BW_20), + VHT_GROUP(2, 0, BW_20), + VHT_GROUP(3, 0, BW_20), + VHT_GROUP(4, 0, BW_20), + + VHT_GROUP(1, 1, BW_20), + VHT_GROUP(2, 1, BW_20), + VHT_GROUP(3, 1, BW_20), + VHT_GROUP(4, 1, BW_20), + + VHT_GROUP(1, 0, BW_40), + VHT_GROUP(2, 0, BW_40), + VHT_GROUP(3, 0, BW_40), + VHT_GROUP(4, 0, BW_40), + + VHT_GROUP(1, 1, BW_40), + VHT_GROUP(2, 1, BW_40), + VHT_GROUP(3, 1, BW_40), + VHT_GROUP(4, 1, BW_40), + + VHT_GROUP(1, 0, BW_80), + VHT_GROUP(2, 0, BW_80), + VHT_GROUP(3, 0, BW_80), + VHT_GROUP(4, 0, BW_80), + + VHT_GROUP(1, 1, BW_80), + VHT_GROUP(2, 1, BW_80), + VHT_GROUP(3, 1, BW_80), + VHT_GROUP(4, 1, BW_80), +}; + +static u32 +ieee80211_calc_legacy_rate_duration(const struct ieee80211_rate *rate, + bool short_pre, bool cck, + int len) +{ + u32 duration; + + if (cck) { + duration = 144 + 48; /* preamble + PLCP */ + if (short_pre) + duration >>= 1; + + duration += 10; /* SIFS */ + } else { + duration = 20 + 16; /* premable + SIFS */ + } + + len <<= 3; + duration += (len * 10) / rate->bitrate; + + return duration; +} + +u32 ieee80211_calc_rx_airtime(struct ieee80211_hw *hw, + struct ieee80211_rx_status *status, + int len) +{ + struct ieee80211_supported_band *sband; + const struct ieee80211_rate *rate; + bool sgi = status->enc_flags & RX_ENC_FLAG_SHORT_GI; + bool sp = status->enc_flags & RX_ENC_FLAG_SHORTPRE; + int bw, streams; + u32 duration; + int group, idx; + bool cck; + + switch (status->bw) { + case RATE_INFO_BW_20: + bw = BW_20; + break; + case RATE_INFO_BW_40: + bw = BW_40; + break; + case RATE_INFO_BW_80: + bw = BW_80; + break; + default: + WARN_ON_ONCE(1); + return 0; + } + + switch (status->encoding) { + case RX_ENC_LEGACY: + if (WARN_ON_ONCE(status->band > NL80211_BAND_5GHZ)) + return 0; + + sband = hw->wiphy->bands[status->band]; + if (!sband || status->rate_idx > sband->n_bitrates) + return 0; + + rate = &sband->bitrates[status->rate_idx]; + cck = (status->rate_idx < CCK_NUM_RATES); + + return ieee80211_calc_legacy_rate_duration(rate, sp, cck, len); + + case RX_ENC_VHT: + streams = status->nss; + idx = status->rate_idx; + group = VHT_GROUP_IDX(streams, sgi, bw); + break; + case RX_ENC_HT: + streams = ((status->rate_idx >> 3) & 3) + 1; + idx = status->rate_idx & 7; + group = HT_GROUP_IDX(streams, sgi, bw); + break; + default: + WARN_ON_ONCE(1); + return 0; + } + + if (WARN_ON_ONCE(streams > 4)) + return 0; + + duration = airtime_mcs_groups[group].duration[idx]; + duration <<= airtime_mcs_groups[group].shift; + duration *= len; + duration /= AVG_PKT_SIZE; + duration /= 1024; + + duration += 36 + (streams << 2); + + return duration; +} + +u32 ieee80211_calc_tx_airtime_rate(struct ieee80211_hw *hw, + struct ieee80211_tx_rate *rate, + u8 band, int len) +{ + struct ieee80211_rx_status stat = { + .band = band, + }; + + if (rate->idx < 0 || !rate->count) + return 0; + + if (rate->flags & IEEE80211_TX_RC_80_MHZ_WIDTH) + stat.bw = RATE_INFO_BW_80; + else if (rate->flags & IEEE80211_TX_RC_40_MHZ_WIDTH) + stat.bw = RATE_INFO_BW_40; + else + stat.bw = RATE_INFO_BW_20; + + stat.enc_flags = 0; + if (rate->flags & IEEE80211_TX_RC_USE_SHORT_PREAMBLE) + stat.enc_flags |= RX_ENC_FLAG_SHORTPRE; + if (rate->flags & IEEE80211_TX_RC_SHORT_GI) + stat.enc_flags |= RX_ENC_FLAG_SHORT_GI; + + stat.rate_idx = rate->idx; + if (rate->flags & IEEE80211_TX_RC_VHT_MCS) { + stat.encoding = RX_ENC_VHT; + stat.rate_idx = ieee80211_rate_get_vht_mcs(rate); + stat.nss = ieee80211_rate_get_vht_nss(rate); + } else if (rate->flags & IEEE80211_TX_RC_MCS) { + stat.encoding = RX_ENC_HT; + } else { + stat.encoding = RX_ENC_LEGACY; + } + + return ieee80211_calc_rx_airtime(hw, &stat, len); +} + +u32 ieee80211_calc_tx_airtime(struct ieee80211_hw *hw, + struct ieee80211_tx_info *info, + int len) +{ + u32 duration = 0; + int i; + + for (i = 0; i < ARRAY_SIZE(info->status.rates); i++) { + struct ieee80211_tx_rate *rate = &info->status.rates[i]; + u32 cur_duration; + + cur_duration = ieee80211_calc_tx_airtime_rate(hw, rate, + info->band, len); + if (!cur_duration) + break; + + duration += cur_duration * rate->count; + } + + return duration; +} +EXPORT_SYMBOL_GPL(ieee80211_calc_tx_airtime); + + +u32 ieee80211_calc_expected_tx_airtime(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_sta *pubsta, + int len) +{ + struct ieee80211_supported_band *sband; + struct ieee80211_chanctx_conf *conf; + struct ieee80211_rate *rate; + struct sta_info *sta; + u8 band = 0; + + len += 38; /* Ethernet header length */ + + conf = rcu_dereference(vif->chanctx_conf); + if (conf) + band = conf->def.chan->band; + + if (pubsta) { + sta = container_of(pubsta, struct sta_info, sta); + return ieee80211_calc_tx_airtime_rate(hw, &sta->tx_stats.last_rate, + band, len); + } + + if (!conf) + return 0; + + /* No station to get latest rate from, so calculate the + * worst-case duration for the lowest rate. + */ + sband = hw->wiphy->bands[band]; + rate = &sband->bitrates[0]; + + return ieee80211_calc_legacy_rate_duration(rate, false, true, len); +} diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index 05406e9c05b3..225ea4e3cd76 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -2249,6 +2249,10 @@ const char *ieee80211_get_reason_code_string(u16 reason_code); extern const struct ethtool_ops ieee80211_ethtool_ops; +u32 ieee80211_calc_expected_tx_airtime(struct ieee80211_hw *hw, + struct ieee80211_vif *vif, + struct ieee80211_sta *pubsta, + int len); #ifdef CONFIG_MAC80211_NOINLINE #define debug_noinline noinline #else From patchwork Fri Oct 18 14:40:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 11198603 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9AF4112C for ; Fri, 18 Oct 2019 14:40:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 81398222C5 for ; Fri, 18 Oct 2019 14:40:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="F92nNU/h" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2634149AbfJROkc (ORCPT ); Fri, 18 Oct 2019 10:40:32 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:30014 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2405743AbfJROkc (ORCPT ); Fri, 18 Oct 2019 10:40:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1571409630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=flcWROVFTs9aet+BgqVvtVvkPpReNrRKNM+zaQRYbCk=; b=F92nNU/hSkGpMTprdsinSEO8sJ/Xm/UvqlRgsOLVeXcy3pn5glOLQwIimxE0gHGfucLkYl I43k0V1UpfVrE1MHEw0y/t+QdO8x+8Pb+c++3/IzfZgo8Dn3n+TRPi5RNBtgAvabFYRE2v YNr7eofJalak0K5XKI5Ztg1todEQPqc= Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com [209.85.167.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-10-2r4LAsybNi2VvMwIaUg3MQ-1; Fri, 18 Oct 2019 10:40:28 -0400 Received: by mail-lf1-f72.google.com with SMTP id p15so1317911lfc.20 for ; Fri, 18 Oct 2019 07:40:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=u20rgbyeO2XBQ3nCAkT/seqHpMxo6ynx78rwo05RXGA=; b=p8J7ydG0rtYYPYWauShia91AGeUF5GpeZEMMszJUhqR2miy0AaUAZ536NbsjzJ4IRA rSo/xbK5CuH/AmYdyb0sGUQVYUAe4hHDkl/pckWxRbJcPZJ2Lj451AfxH7Ro5IQn/7DL Q48ZJPry6+mld0F6Sov3zWB54X38+E3ezk8cX4MJnQa4kyRUXY9QxgaCzJRmMsil+CQ3 eXNeYBTtBeRRWcpgp+L9c7DSXO4J05yJuc04UmpVi5Th/hgiQJ/pvp6eFfIxVZBjqkbs LILOCysVGcA2ot+V0rqwP0494c4J+ixgwNnbxkoMajCWSBMgmwIKDNsFnOYLcOTcOEh4 aEAA== X-Gm-Message-State: APjAAAXyLmKq6AXAgt/AYGs4TUAw5urtXLdDF2s41lzQgfIhabwNxunQ HKHYmdiHT3syzPPgzoOMK9aT3UEbphNPtMPFOKVNczwBJ8PSCc55CY4+zoMzWfNqxJuT4hCkKUF tTZfKMZuHISl6y9DiIcbAhOikb8g= X-Received: by 2002:a19:f013:: with SMTP id p19mr6173593lfc.98.1571409626503; Fri, 18 Oct 2019 07:40:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqyNJERsPojvVwFKlcLVLfEAlJmm7Y+70D/Lr9lkHjn/7T3FPmPv4e+AfgtG/dp+A/IwqEAIoQ== X-Received: by 2002:a19:f013:: with SMTP id p19mr6173559lfc.98.1571409625940; Fri, 18 Oct 2019 07:40:25 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a00:7660:6da:443::2]) by smtp.gmail.com with ESMTPSA id l7sm2399962lji.46.2019.10.18.07.40.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2019 07:40:25 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 5360D1804C9; Fri, 18 Oct 2019 16:40:24 +0200 (CEST) Subject: [PATCH v3 3/4] mac80211: Implement Airtime-based Queue Limit (AQL) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: Johannes Berg Cc: linux-wireless@vger.kernel.org, make-wifi-fast@lists.bufferbloat.net, ath10k@lists.infradead.org, John Crispin , Lorenzo Bianconi , Felix Fietkau , Kan Yan , Rajkumar Manoharan , Kevin Hayes Date: Fri, 18 Oct 2019 16:40:24 +0200 Message-ID: <157140962422.2866668.6996249266501389333.stgit@toke.dk> In-Reply-To: <157140962094.2866668.3383184317264069493.stgit@toke.dk> References: <157140962094.2866668.3383184317264069493.stgit@toke.dk> User-Agent: StGit/0.20 MIME-Version: 1.0 X-MC-Unique: 2r4LAsybNi2VvMwIaUg3MQ-1 X-Mimecast-Spam-Score: 0 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Kan Yan In order for the Fq_CoDel integrated in mac80211 layer operates effectively to control excessive queueing latency, the CoDel algorithm requires an accurate measure of how long the packets stays in the queue, aka sojourn time. The sojourn time measured at mac80211 layer doesn't include queueing latency in lower layer (firmware/hardware) and CoDel expects lower layer to have a short queue. However, most 802.11ac chipsets offload tasks such TX aggregation to firmware or hardware, thus have a deep lower layer queue. Without a mechanism to control the lower layer queue size, packets only stays in mac80211 layer transiently before being sent to firmware queue. As a result, the sojourn time measured by CoDel in the mac80211 layer is almost always lower than the CoDel latency target, hence CoDel does little to control the latency, even when the lower layer queue causes excessive latency. Byte Queue limits (BQL) is commonly used to address the similar issue with wired network interface. However, this method cannot be applied directly to the wireless network interface. Byte is not a suitable measure of queue depth in the wireless network, as the data rate can vary dramatically from station to station in the same network, from a few Mbps to over Gbps. This patch implemented an Airtime-based Queue Limit (AQL) to make CoDel works effectively with wireless drivers that utilized firmware/hardware offloading. AQL only allows each txq to release just enough packets to the lower layer to form 1-2 large aggregations to keep hardware fully utilized and keep the rest of frames in mac80211 layer to be controlled by the CoDel algorithm. Signed-off-by: Kan Yan [ Toke: Get rid of the driver API to set pending airtime ] Signed-off-by: Toke Høiland-Jørgensen --- include/net/cfg80211.h | 7 ++++ include/net/mac80211.h | 12 +++++++ net/mac80211/debugfs.c | 78 ++++++++++++++++++++++++++++++++++++++++++++ net/mac80211/debugfs_sta.c | 43 +++++++++++++++++++----- net/mac80211/ieee80211_i.h | 4 ++ net/mac80211/main.c | 9 +++++ net/mac80211/sta_info.c | 32 ++++++++++++++++++ net/mac80211/sta_info.h | 8 +++++ net/mac80211/tx.c | 46 ++++++++++++++++++++++++-- 9 files changed, 225 insertions(+), 14 deletions(-) diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h index ff45c3e1abff..8d50c0a60dbd 100644 --- a/include/net/cfg80211.h +++ b/include/net/cfg80211.h @@ -2602,6 +2602,13 @@ enum wiphy_params_flags { #define IEEE80211_DEFAULT_AIRTIME_WEIGHT 256 +/* The per TXQ device queue limit in airtime */ +#define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_L 4000 +#define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_H 8000 + +/* The per interface airtime threshold to switch to lower queue limit */ +#define IEEE80211_AQL_THRESHOLD 24000 + /** * struct cfg80211_pmksa - PMK Security Association * diff --git a/include/net/mac80211.h b/include/net/mac80211.h index f058386e3fef..fdfa5805e1cf 100644 --- a/include/net/mac80211.h +++ b/include/net/mac80211.h @@ -5565,6 +5565,18 @@ void ieee80211_send_eosp_nullfunc(struct ieee80211_sta *pubsta, int tid); void ieee80211_sta_register_airtime(struct ieee80211_sta *pubsta, u8 tid, u32 tx_airtime, u32 rx_airtime); +/** + * ieee80211_txq_airtime_check - check if a txq can send frame to device + * + * @hw: pointer obtained from ieee80211_alloc_hw() + * @txq: pointer obtained from station or virtual interface + * + * Return true if the AQL's airtime limit has not been reached and the txq can + * continue to send more packets to the device. Otherwise return false. + */ +bool +ieee80211_txq_airtime_check(struct ieee80211_hw *hw, struct ieee80211_txq *txq); + /** * ieee80211_iter_keys - iterate keys programmed into the device * @hw: pointer obtained from ieee80211_alloc_hw() diff --git a/net/mac80211/debugfs.c b/net/mac80211/debugfs.c index 568b3b276931..d77ea0e51c1d 100644 --- a/net/mac80211/debugfs.c +++ b/net/mac80211/debugfs.c @@ -148,6 +148,80 @@ static const struct file_operations aqm_ops = { .llseek = default_llseek, }; +static ssize_t aql_txq_limit_read(struct file *file, + char __user *user_buf, + size_t count, + loff_t *ppos) +{ + struct ieee80211_local *local = file->private_data; + char buf[400]; + int len = 0; + + len = scnprintf(buf, sizeof(buf), + "AC AQL limit low AQL limit high\n" + "VO %u %u\n" + "VI %u %u\n" + "BE %u %u\n" + "BK %u %u\n", + local->aql_txq_limit_low[IEEE80211_AC_VO], + local->aql_txq_limit_high[IEEE80211_AC_VO], + local->aql_txq_limit_low[IEEE80211_AC_VI], + local->aql_txq_limit_high[IEEE80211_AC_VI], + local->aql_txq_limit_low[IEEE80211_AC_BE], + local->aql_txq_limit_high[IEEE80211_AC_BE], + local->aql_txq_limit_low[IEEE80211_AC_BK], + local->aql_txq_limit_high[IEEE80211_AC_BK]); + return simple_read_from_buffer(user_buf, count, ppos, + buf, len); +} + +static ssize_t aql_txq_limit_write(struct file *file, + const char __user *user_buf, + size_t count, + loff_t *ppos) +{ + struct ieee80211_local *local = file->private_data; + char buf[100]; + size_t len; + u32 ac, q_limit_low, q_limit_high; + struct sta_info *sta; + + if (count > sizeof(buf)) + return -EINVAL; + + if (copy_from_user(buf, user_buf, count)) + return -EFAULT; + + buf[sizeof(buf) - 1] = 0; + len = strlen(buf); + if (len > 0 && buf[len - 1] == '\n') + buf[len - 1] = 0; + + if (sscanf(buf, "%u %u %u", &ac, &q_limit_low, &q_limit_high) != 3) + return -EINVAL; + + if (ac >= IEEE80211_NUM_ACS) + return -EINVAL; + + local->aql_txq_limit_low[ac] = q_limit_low; + local->aql_txq_limit_high[ac] = q_limit_high; + + mutex_lock(&local->sta_mtx); + list_for_each_entry(sta, &local->sta_list, list) { + sta->airtime[ac].aql_limit_low = q_limit_low; + sta->airtime[ac].aql_limit_high = q_limit_high; + } + mutex_unlock(&local->sta_mtx); + return count; +} + +static const struct file_operations aql_txq_limit_ops = { + .write = aql_txq_limit_write, + .read = aql_txq_limit_read, + .open = simple_open, + .llseek = default_llseek, +}; + static ssize_t force_tx_status_read(struct file *file, char __user *user_buf, size_t count, @@ -441,6 +515,10 @@ void debugfs_hw_add(struct ieee80211_local *local) debugfs_create_u16("airtime_flags", 0600, phyd, &local->airtime_flags); + DEBUGFS_ADD(aql_txq_limit); + debugfs_create_u32("aql_threshold", 0600, + phyd, &local->aql_threshold); + statsd = debugfs_create_dir("statistics", phyd); /* if the dir failed, don't put all the other things into the root! */ diff --git a/net/mac80211/debugfs_sta.c b/net/mac80211/debugfs_sta.c index c8ad20c28c43..9f9b8f5ed86a 100644 --- a/net/mac80211/debugfs_sta.c +++ b/net/mac80211/debugfs_sta.c @@ -197,10 +197,12 @@ static ssize_t sta_airtime_read(struct file *file, char __user *userbuf, { struct sta_info *sta = file->private_data; struct ieee80211_local *local = sta->sdata->local; - size_t bufsz = 200; + size_t bufsz = 400; char *buf = kzalloc(bufsz, GFP_KERNEL), *p = buf; u64 rx_airtime = 0, tx_airtime = 0; s64 deficit[IEEE80211_NUM_ACS]; + u32 q_depth[IEEE80211_NUM_ACS]; + u32 q_limit_l[IEEE80211_NUM_ACS], q_limit_h[IEEE80211_NUM_ACS]; ssize_t rv; int ac; @@ -212,19 +214,22 @@ static ssize_t sta_airtime_read(struct file *file, char __user *userbuf, rx_airtime += sta->airtime[ac].rx_airtime; tx_airtime += sta->airtime[ac].tx_airtime; deficit[ac] = sta->airtime[ac].deficit; + q_limit_l[ac] = sta->airtime[ac].aql_limit_low; + q_limit_h[ac] = sta->airtime[ac].aql_limit_high; + q_depth[ac] = sta->airtime[ac].aql_tx_pending; spin_unlock_bh(&local->active_txq_lock[ac]); } p += scnprintf(p, bufsz + buf - p, "RX: %llu us\nTX: %llu us\nWeight: %u\n" - "Deficit: VO: %lld us VI: %lld us BE: %lld us BK: %lld us\n", - rx_airtime, - tx_airtime, - sta->airtime_weight, - deficit[0], - deficit[1], - deficit[2], - deficit[3]); + "Deficit: VO: %lld us VI: %lld us BE: %lld us BK: %lld us\n" + "Q depth: VO: %u us VI: %u us BE: %u us BK: %u us\n" + "Q limit[low/high]: VO: %u/%u VI: %u/%u BE: %u/%u BK: %u/%u\n", + rx_airtime, tx_airtime, sta->airtime_weight, + deficit[0], deficit[1], deficit[2], deficit[3], + q_depth[0], q_depth[1], q_depth[2], q_depth[3], + q_limit_l[0], q_limit_h[0], q_limit_l[1], q_limit_h[1], + q_limit_l[2], q_limit_h[2], q_limit_l[3], q_limit_h[3]), rv = simple_read_from_buffer(userbuf, count, ppos, buf, p - buf); kfree(buf); @@ -236,7 +241,25 @@ static ssize_t sta_airtime_write(struct file *file, const char __user *userbuf, { struct sta_info *sta = file->private_data; struct ieee80211_local *local = sta->sdata->local; - int ac; + u32 ac, q_limit_l, q_limit_h; + char _buf[100] = {}, *buf = _buf; + + if (count > sizeof(_buf)) + return -EINVAL; + + if (copy_from_user(buf, userbuf, count)) + return -EFAULT; + + buf[sizeof(_buf) - 1] = '\0'; + if (sscanf(buf, "queue limit %u %u %u", &ac, &q_limit_l, &q_limit_h) + != 3) + return -EINVAL; + + if (ac >= IEEE80211_NUM_ACS) + return -EINVAL; + + sta->airtime[ac].aql_limit_low = q_limit_l; + sta->airtime[ac].aql_limit_high = q_limit_h; for (ac = 0; ac < IEEE80211_NUM_ACS; ac++) { spin_lock_bh(&local->active_txq_lock[ac]); diff --git a/net/mac80211/ieee80211_i.h b/net/mac80211/ieee80211_i.h index 225ea4e3cd76..6fa690757388 100644 --- a/net/mac80211/ieee80211_i.h +++ b/net/mac80211/ieee80211_i.h @@ -1142,6 +1142,10 @@ struct ieee80211_local { u16 schedule_round[IEEE80211_NUM_ACS]; u16 airtime_flags; + u32 aql_txq_limit_low[IEEE80211_NUM_ACS]; + u32 aql_txq_limit_high[IEEE80211_NUM_ACS]; + u32 aql_threshold; + u32 aql_total_pending_airtime; const struct ieee80211_ops *ops; diff --git a/net/mac80211/main.c b/net/mac80211/main.c index aba094b4ccfc..0792c9b9c850 100644 --- a/net/mac80211/main.c +++ b/net/mac80211/main.c @@ -667,8 +667,15 @@ struct ieee80211_hw *ieee80211_alloc_hw_nm(size_t priv_data_len, for (i = 0; i < IEEE80211_NUM_ACS; i++) { INIT_LIST_HEAD(&local->active_txqs[i]); spin_lock_init(&local->active_txq_lock[i]); + local->aql_txq_limit_low[i] = IEEE80211_DEFAULT_AQL_TXQ_LIMIT_L; + local->aql_txq_limit_high[i] = + IEEE80211_DEFAULT_AQL_TXQ_LIMIT_H; } - local->airtime_flags = AIRTIME_USE_TX | AIRTIME_USE_RX; + + local->airtime_flags = AIRTIME_USE_TX | + AIRTIME_USE_RX | + AIRTIME_USE_AQL; + local->aql_threshold = IEEE80211_AQL_THRESHOLD; INIT_LIST_HEAD(&local->chanctx_list); mutex_init(&local->chanctx_mtx); diff --git a/net/mac80211/sta_info.c b/net/mac80211/sta_info.c index bd11fef2139f..64bacf4f068c 100644 --- a/net/mac80211/sta_info.c +++ b/net/mac80211/sta_info.c @@ -396,6 +396,9 @@ struct sta_info *sta_info_alloc(struct ieee80211_sub_if_data *sdata, skb_queue_head_init(&sta->ps_tx_buf[i]); skb_queue_head_init(&sta->tx_filtered[i]); sta->airtime[i].deficit = sta->airtime_weight; + sta->airtime[i].aql_tx_pending = 0; + sta->airtime[i].aql_limit_low = local->aql_txq_limit_low[i]; + sta->airtime[i].aql_limit_high = local->aql_txq_limit_high[i]; } for (i = 0; i < IEEE80211_NUM_TIDS; i++) @@ -1893,6 +1896,35 @@ void ieee80211_sta_register_airtime(struct ieee80211_sta *pubsta, u8 tid, } EXPORT_SYMBOL(ieee80211_sta_register_airtime); +void ieee80211_sta_update_pending_airtime(struct ieee80211_local *local, + struct sta_info *sta, u8 ac, + u16 tx_airtime, bool tx_completed) +{ + spin_lock_bh(&local->active_txq_lock[ac]); + if (tx_completed) { + if (sta) { + if (WARN_ONCE(sta->airtime[ac].aql_tx_pending < tx_airtime, + "TXQ pending airtime underflow: %u, %u", + sta->airtime[ac].aql_tx_pending, tx_airtime)) + sta->airtime[ac].aql_tx_pending = 0; + else + sta->airtime[ac].aql_tx_pending -= tx_airtime; + } + + if (WARN_ONCE(local->aql_total_pending_airtime < tx_airtime, + "Device pending airtime underflow: %u, %u", + local->aql_total_pending_airtime, tx_airtime)) + local->aql_total_pending_airtime = 0; + else + local->aql_total_pending_airtime -= tx_airtime; + } else { + if (sta) + sta->airtime[ac].aql_tx_pending += tx_airtime; + local->aql_total_pending_airtime += tx_airtime; + } + spin_unlock_bh(&local->active_txq_lock[ac]); +} + int sta_info_move_state(struct sta_info *sta, enum ieee80211_sta_state new_state) { diff --git a/net/mac80211/sta_info.h b/net/mac80211/sta_info.h index 369c2dddce52..4e4d76e81b0f 100644 --- a/net/mac80211/sta_info.h +++ b/net/mac80211/sta_info.h @@ -127,13 +127,21 @@ enum ieee80211_agg_stop_reason { /* Debugfs flags to enable/disable use of RX/TX airtime in scheduler */ #define AIRTIME_USE_TX BIT(0) #define AIRTIME_USE_RX BIT(1) +#define AIRTIME_USE_AQL BIT(2) struct airtime_info { u64 rx_airtime; u64 tx_airtime; s64 deficit; + u32 aql_tx_pending; /* Estimated airtime for frames pending in queue */ + u32 aql_limit_low; + u32 aql_limit_high; }; +void ieee80211_sta_update_pending_airtime(struct ieee80211_local *local, + struct sta_info *sta, u8 ac, + u16 tx_airtime, bool tx_completed); + struct sta_info; /** diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index a16c2f863702..12653d873b8c 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -3665,7 +3665,8 @@ struct ieee80211_txq *ieee80211_next_txq(struct ieee80211_hw *hw, u8 ac) { struct ieee80211_local *local = hw_to_local(hw); struct ieee80211_txq *ret = NULL; - struct txq_info *txqi = NULL; + struct txq_info *txqi = NULL, *head = NULL; + bool found_eligible_txq = false; spin_lock_bh(&local->active_txq_lock[ac]); @@ -3676,13 +3677,26 @@ struct ieee80211_txq *ieee80211_next_txq(struct ieee80211_hw *hw, u8 ac) if (!txqi) goto out; + if (txqi == head && !found_eligible_txq) + goto out; + + if (!head) + head = txqi; + if (txqi->txq.sta) { struct sta_info *sta = container_of(txqi->txq.sta, - struct sta_info, sta); + struct sta_info, sta); + bool aql_check = ieee80211_txq_airtime_check(hw, &txqi->txq); + s64 deficit = sta->airtime[txqi->txq.ac].deficit; + + if (aql_check) + found_eligible_txq = true; - if (sta->airtime[txqi->txq.ac].deficit < 0) { + if (deficit < 0) sta->airtime[txqi->txq.ac].deficit += sta->airtime_weight; + + if (deficit < 0 || !aql_check) { list_move_tail(&txqi->schedule_order, &local->active_txqs[txqi->txq.ac]); goto begin; @@ -3736,6 +3750,32 @@ void __ieee80211_schedule_txq(struct ieee80211_hw *hw, } EXPORT_SYMBOL(__ieee80211_schedule_txq); +bool ieee80211_txq_airtime_check(struct ieee80211_hw *hw, + struct ieee80211_txq *txq) +{ + struct sta_info *sta; + struct ieee80211_local *local = hw_to_local(hw); + + if (!(local->airtime_flags & AIRTIME_USE_AQL)) + return true; + + if (!txq->sta) + return true; + + sta = container_of(txq->sta, struct sta_info, sta); + if (sta->airtime[txq->ac].aql_tx_pending < + sta->airtime[txq->ac].aql_limit_low) + return true; + + if (local->aql_total_pending_airtime < local->aql_threshold && + sta->airtime[txq->ac].aql_tx_pending < + sta->airtime[txq->ac].aql_limit_high) + return true; + + return false; +} +EXPORT_SYMBOL(ieee80211_txq_airtime_check); + bool ieee80211_txq_may_transmit(struct ieee80211_hw *hw, struct ieee80211_txq *txq) { From patchwork Fri Oct 18 14:40:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 11198605 X-Patchwork-Delegate: johannes@sipsolutions.net Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69EFE913 for ; Fri, 18 Oct 2019 14:40:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 432AE222C5 for ; Fri, 18 Oct 2019 14:40:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="jFSOqBde" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2634141AbfJROkc (ORCPT ); Fri, 18 Oct 2019 10:40:32 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:26970 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2410018AbfJROkc (ORCPT ); Fri, 18 Oct 2019 10:40:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1571409630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lqBYAyfvGqVBfvQQyb/0J8hd/48vyEbBcTT/es8g2a8=; b=jFSOqBdeeUyrapoZYGsbTVp11SJVvfBO8VNUYHi0z8AsGXOH6lnHRVag7veTte8IuRFhj2 sTTkySeSGH/174pwPIGqHmAjI9HNO01V5xEZzC906uQ05lbQaR8ZooZUxZPahCTRpnDjxc 05o3KNOMz790eJ21+KUQpHRKQrW1PCI= Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com [209.85.167.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-12-FShSDlISMPm3bVlutHFqSA-1; Fri, 18 Oct 2019 10:40:28 -0400 Received: by mail-lf1-f72.google.com with SMTP id c7so1327757lfh.9 for ; Fri, 18 Oct 2019 07:40:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=B/zTxrtPVbDPMk1jNIRjvApKrsIv0Uj3+z7s63i2jdw=; b=cWddvK9qx15r/f/3sliMVEOkgOQsP3kD6xxaakHm0hasy6B8/wAfRA3OKCwsOiOI8Y 0mcX70btzMTZSaP2VHXZvQj8OontRbjO2LKxlI+pXb2i/fNphfm/E2XhEBIu6ktBxMVT zkh/L6K0SmqGrzNiL332Q7BOGNQSnEBS1GgmJJH6zs9U67WXBt6N5cjnrvldwwQxzpuo 6/IpOjHIubXMEZnf/1ukrrelI+IRzAy25LPlJMk4mVTuE92nBuu6ap6+bwC8CukRzgQV 1yJsEjascZt1nRM31p6WodH9rvm1NJf4X9FHx1+xc5Tyc5FYUE9h6Iah8m8XjTJnzsvf MkDA== X-Gm-Message-State: APjAAAXeYDCY8mzCNeAALkUovI0eBIM7GlHEuT1CuhUJUfcQPOX6HByg mj0pMqLRAEVZnS4nCdMFdGqY3MF6oZLCeYNQVR8VnFgecQSDQIWU3Q3GQcSuB3uJ/xUSMhAkS3e IdYyB+7hk+WlW2J9xnAWQUqgWg/Y= X-Received: by 2002:a2e:5dd5:: with SMTP id v82mr6545681lje.54.1571409627129; Fri, 18 Oct 2019 07:40:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqxolQE2snCfRFv7NPp6VkDcPkPnM67ptbRrwbO/RDkqMvooRv8yklzlnAGDuCtp1Rd02Oj9BA== X-Received: by 2002:a2e:5dd5:: with SMTP id v82mr6545664lje.54.1571409626863; Fri, 18 Oct 2019 07:40:26 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a00:7660:6da:443::2]) by smtp.gmail.com with ESMTPSA id d25sm2572501lfj.15.2019.10.18.07.40.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2019 07:40:26 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 6A8AC1804B6; Fri, 18 Oct 2019 16:40:25 +0200 (CEST) Subject: [PATCH v3 4/4] mac80211: Use Airtime-based Queue Limits (AQL) on packet dequeue From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: Johannes Berg Cc: linux-wireless@vger.kernel.org, make-wifi-fast@lists.bufferbloat.net, ath10k@lists.infradead.org, John Crispin , Lorenzo Bianconi , Felix Fietkau , Kan Yan , Rajkumar Manoharan , Kevin Hayes Date: Fri, 18 Oct 2019 16:40:25 +0200 Message-ID: <157140962534.2866668.1182822483250825739.stgit@toke.dk> In-Reply-To: <157140962094.2866668.3383184317264069493.stgit@toke.dk> References: <157140962094.2866668.3383184317264069493.stgit@toke.dk> User-Agent: StGit/0.20 MIME-Version: 1.0 X-MC-Unique: FShSDlISMPm3bVlutHFqSA-1 X-Mimecast-Spam-Score: 0 Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org From: Toke Høiland-Jørgensen The previous commit added the ability to throttle stations when they queue too much airtime in the hardware. This commit enables the functionality by calculating the expected airtime usage of each packet that is dequeued from the TXQs in mac80211, and accounting that as pending airtime. The estimated airtime for each skb is stored in the tx_info, so we can subtract the same amount from the running total when the skb is freed or recycled. The throttling mechanism relies on this accounting to be accurate (i.e., that we are not freeing skbs without subtracting any airtime they were accounted for), so we put the subtraction into ieee80211_report_used_skb(). As an optimisation, we also subtract the airtime on regular TX completion, zeroing out the value stored in the packet afterwards, to avoid having to do an expensive lookup of the station from the packet data on every packet. This patch does *not* include any mechanism to wake a throttled TXQ again, on the assumption that this will happen anyway as a side effect of whatever freed the skb (most commonly a TX completion). Signed-off-by: Toke Høiland-Jørgensen --- net/mac80211/status.c | 38 ++++++++++++++++++++++++++++++++++++++ net/mac80211/tx.c | 19 +++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/net/mac80211/status.c b/net/mac80211/status.c index ab8ba5835ca0..905b4afd457d 100644 --- a/net/mac80211/status.c +++ b/net/mac80211/status.c @@ -676,6 +676,33 @@ static void ieee80211_report_used_skb(struct ieee80211_local *local, if (dropped) acked = false; + if (info->tx_time_est) { + struct ieee80211_sub_if_data *sdata; + struct sta_info *sta = NULL; + u8 *qc, ac; + int tid; + + rcu_read_lock(); + + sdata = ieee80211_sdata_from_skb(local, skb); + if (sdata) + sta = sta_info_get_bss(sdata, skb_mac_header(skb)); + + if (ieee80211_is_data_qos(hdr->frame_control)) { + qc = ieee80211_get_qos_ctl(hdr); + tid = qc[0] & 0xf; + ac = ieee80211_ac_from_tid(tid); + } else { + ac = IEEE80211_AC_BE; + } + + ieee80211_sta_update_pending_airtime(local, sta, ac, + info->tx_time_est << 2, + true); + rcu_read_unlock(); + + } + if (info->flags & IEEE80211_TX_INTFL_MLME_CONN_TX) { struct ieee80211_sub_if_data *sdata; @@ -930,6 +957,17 @@ static void __ieee80211_tx_status(struct ieee80211_hw *hw, tid = qc[0] & 0xf; } + if (info->tx_time_est) { + /* Do this here to avoid the expensive lookup of the sta + * in ieee80211_report_used_skb(). + */ + ieee80211_sta_update_pending_airtime(local, sta, + ieee80211_ac_from_tid(tid), + info->tx_time_est << 2, + true); + info->tx_time_est = 0; + } + if (!acked && ieee80211_is_back_req(fc)) { u16 control; diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index 12653d873b8c..bd8284f04361 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -3539,9 +3539,14 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw, struct ieee80211_tx_data tx; ieee80211_tx_result r; struct ieee80211_vif *vif = txq->vif; + u8 ac = txq->ac; + u32 airtime; WARN_ON_ONCE(softirq_count() == 0); + if (!ieee80211_txq_airtime_check(hw, txq)) + return NULL; + begin: spin_lock_bh(&fq->lock); @@ -3652,6 +3657,20 @@ struct sk_buff *ieee80211_tx_dequeue(struct ieee80211_hw *hw, } IEEE80211_SKB_CB(skb)->control.vif = vif; + + if (local->airtime_flags & AIRTIME_USE_AQL) { + airtime = ieee80211_calc_expected_tx_airtime(hw, vif, txq->sta, + skb->len); + if (airtime) { + /* We only have 10 bits in tx_time_est, so store airtime + * in increments of 4 us and clamp that to 2**10. + */ + info->tx_time_est = min_t(u32, airtime >> 2, 1 << 10); + ieee80211_sta_update_pending_airtime(local, tx.sta, ac, + airtime, false); + } + } + return skb; out: