From patchwork Thu Jan 9 09:43:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steffen Klassert X-Patchwork-Id: 13932250 X-Patchwork-Delegate: kuba@kernel.org Received: from a.mx.secunet.com (a.mx.secunet.com [62.96.220.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3CEA215798 for ; Thu, 9 Jan 2025 09:43:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=62.96.220.36 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736415818; cv=none; b=NDZZWEEFHCQwdXtRcLl3flmAmGt1V+0LC5MtKJ32kXrtf4b7QtIqv+s8CuFR35gVFW1hT5HZazJSt9drBIxRgm9QrdMicMquYcmOyWaJZ4309nPlWPJ0iGajywd3JjZPOEA1fLZpd2vXTR4ZWKvdb/6q9Jz4Ji36CUunJbfa7dM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736415818; c=relaxed/simple; bh=fDxicSUSyUJioRCvmYsQoqBbT2/7waKVFn4Uf8kn52Y=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DS3xE0ZksMCbqVg5DIMa+WL1e6sTqZZGxEhDIqiyi2+MOaDZJLHVoCUvDxQiHR3zaIjQ7uG6PAJSDq7NbzZE9RzZB1IhYTAB8rhJgDUwOq09x98qhkK8BRGMic/As11IacvASKVgk7tjmp+QEY6dre6dwOoYENnVPTVVdswdTHY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=secunet.com; spf=pass smtp.mailfrom=secunet.com; dkim=pass (2048-bit key) header.d=secunet.com header.i=@secunet.com header.b=rEj7/CsS; arc=none smtp.client-ip=62.96.220.36 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=secunet.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=secunet.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=secunet.com header.i=@secunet.com header.b="rEj7/CsS" Received: from localhost (localhost [127.0.0.1]) by a.mx.secunet.com (Postfix) with ESMTP id 0A0482088F; Thu, 9 Jan 2025 10:43:33 +0100 (CET) X-Virus-Scanned: by secunet Received: from a.mx.secunet.com ([127.0.0.1]) by localhost (a.mx.secunet.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n1fhS9TXTPfq; Thu, 9 Jan 2025 10:43:32 +0100 (CET) Received: from cas-essen-02.secunet.de (rl2.secunet.de [10.53.40.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by a.mx.secunet.com (Postfix) with ESMTPS id 1169C20854; Thu, 9 Jan 2025 10:43:31 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.11.0 a.mx.secunet.com 1169C20854 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=secunet.com; s=202301; t=1736415811; bh=yrVdlAg+4AtmABv+lQlxumWeNmkIEpBr/sE1ud4FbYA=; h=From:To:CC:Subject:Date:In-Reply-To:References:From; b=rEj7/CsS9o8+759iRJdC0tRjSOhCubPZpJzqqTa5VoNE9ytMKs0jrqGSgvyQpaNxf ZUK34Yg+m5MPj0OLidK0/bwu0Abrm/eiHH7Tb/o5A3tHS3cMOR640u9LCRLAf1NC/Z R4v9NT7pFpehcAgROW4haEseWB/HWMU7JQmxjJjsNEiQ0mVbnAJtxIvKx+QxihK4Wy XoFO1VODUMv7Xa08gLm9DZS2iRKkujRFHOtmGL0kMNE/uURVQGU81ttLrGtgymLzSx n+318S6P42yMXve3MM5gvmhbvmC+D+ZBr8YgYya0JHEFdSF/Fdlm95X2E1UnvXPLRm keq0tYv0b6UEA== Received: from mbx-essen-02.secunet.de (10.53.40.198) by cas-essen-02.secunet.de (10.53.40.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 9 Jan 2025 10:43:30 +0100 Received: from gauss2.secunet.de (10.182.7.193) by mbx-essen-02.secunet.de (10.53.40.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 9 Jan 2025 10:43:30 +0100 Received: by gauss2.secunet.de (Postfix, from userid 1000) id 9B8B331840D6; Thu, 9 Jan 2025 10:43:29 +0100 (CET) From: Steffen Klassert To: David Miller , Jakub Kicinski CC: Herbert Xu , Steffen Klassert , Subject: [PATCH 08/17] xfrm: iptfs: share page fragments of inner packets Date: Thu, 9 Jan 2025 10:43:12 +0100 Message-ID: <20250109094321.2268124-9-steffen.klassert@secunet.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250109094321.2268124-1-steffen.klassert@secunet.com> References: <20250109094321.2268124-1-steffen.klassert@secunet.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: cas-essen-02.secunet.de (10.53.40.202) To mbx-essen-02.secunet.de (10.53.40.198) X-EXCLAIMER-MD-CONFIG: 2c86f778-e09b-4440-8b15-867914633a10 X-Patchwork-Delegate: kuba@kernel.org From: Christian Hopps When possible rather than appending secondary (aggregated) inner packets to the fragment list, share their page fragments with the outer IPTFS packet. This allows for more efficient packet transmission. Signed-off-by: Christian Hopps Tested-by: Antony Antony Signed-off-by: Steffen Klassert --- net/xfrm/xfrm_iptfs.c | 85 +++++++++++++++++++++++++++++++++++++++---- 1 file changed, 77 insertions(+), 8 deletions(-) diff --git a/net/xfrm/xfrm_iptfs.c b/net/xfrm/xfrm_iptfs.c index c4cff005ea9a..7bf18f472fed 100644 --- a/net/xfrm/xfrm_iptfs.c +++ b/net/xfrm/xfrm_iptfs.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -90,6 +91,23 @@ struct xfrm_iptfs_data { static u32 iptfs_get_inner_mtu(struct xfrm_state *x, int outer_mtu); static enum hrtimer_restart iptfs_delay_timer(struct hrtimer *me); +/* ======================= */ +/* IPTFS SK_BUFF Functions */ +/* ======================= */ + +/** + * iptfs_skb_head_to_frag() - initialize a skb_frag_t based on skb head data + * @skb: skb with the head data + * @frag: frag to initialize + */ +static void iptfs_skb_head_to_frag(const struct sk_buff *skb, skb_frag_t *frag) +{ + struct page *page = virt_to_head_page(skb->data); + unsigned char *addr = (unsigned char *)page_address(page); + + skb_frag_fill_page_desc(frag, page, skb->data - addr, skb_headlen(skb)); +} + /* ================================= */ /* IPTFS Sending (ingress) Functions */ /* ================================= */ @@ -297,14 +315,44 @@ static struct sk_buff **iptfs_rehome_fraglist(struct sk_buff **nextp, struct sk_ return nextp; } +static void iptfs_consume_frags(struct sk_buff *to, struct sk_buff *from) +{ + struct skb_shared_info *fromi = skb_shinfo(from); + struct skb_shared_info *toi = skb_shinfo(to); + unsigned int new_truesize; + + /* If we have data in a head page, grab it */ + if (!skb_headlen(from)) { + new_truesize = SKB_TRUESIZE(skb_end_offset(from)); + } else { + iptfs_skb_head_to_frag(from, &toi->frags[toi->nr_frags]); + skb_frag_ref(to, toi->nr_frags++); + new_truesize = SKB_DATA_ALIGN(sizeof(struct sk_buff)); + } + + /* Move any other page fragments rather than copy */ + memcpy(&toi->frags[toi->nr_frags], fromi->frags, + sizeof(fromi->frags[0]) * fromi->nr_frags); + toi->nr_frags += fromi->nr_frags; + fromi->nr_frags = 0; + from->data_len = 0; + from->len = 0; + to->truesize += from->truesize - new_truesize; + from->truesize = new_truesize; + + /* We are done with this SKB */ + consume_skb(from); +} + static void iptfs_output_queued(struct xfrm_state *x, struct sk_buff_head *list) { struct xfrm_iptfs_data *xtfs = x->mode_data; struct sk_buff *skb, *skb2, **nextp; - struct skb_shared_info *shi; + struct skb_shared_info *shi, *shi2; while ((skb = __skb_dequeue(list))) { u32 mtu = iptfs_get_cur_pmtu(x, xtfs, skb); + bool share_ok = true; int remaining; /* protocol comes to us cleared sometimes */ @@ -349,7 +397,7 @@ static void iptfs_output_queued(struct xfrm_state *x, struct sk_buff_head *list) /* Re-home (un-nest) nested fragment lists. We need to do this * b/c we will simply be appending any following aggregated - * inner packets to the frag list. + * inner packets using the frag list. */ shi = skb_shinfo(skb); nextp = &shi->frag_list; @@ -360,6 +408,9 @@ static void iptfs_output_queued(struct xfrm_state *x, struct sk_buff_head *list) nextp = &(*nextp)->next; } + if (shi->frag_list || skb_cloned(skb) || skb_shared(skb)) + share_ok = false; + /* See if we have enough space to simply append. * * NOTE: Maybe do not append if we will be mis-aligned, @@ -386,17 +437,35 @@ static void iptfs_output_queued(struct xfrm_state *x, struct sk_buff_head *list) } } + /* skb->pp_recycle is passed to __skb_flag_unref for all + * frag pages so we can only share pages with skb's who + * match ourselves. + */ + shi2 = skb_shinfo(skb2); + if (share_ok && + (shi2->frag_list || + (!skb2->head_frag && skb_headlen(skb)) || + skb->pp_recycle != skb2->pp_recycle || + skb_zcopy(skb2) || + (shi->nr_frags + shi2->nr_frags + 1 > MAX_SKB_FRAGS))) + share_ok = false; + /* Do accounting */ skb->data_len += skb2->len; skb->len += skb2->len; remaining -= skb2->len; - /* Append to the frag_list */ - *nextp = skb2; - nextp = &skb2->next; - if (skb_has_frag_list(skb2)) - nextp = iptfs_rehome_fraglist(nextp, skb2); - skb->truesize += skb2->truesize; + if (share_ok) { + iptfs_consume_frags(skb, skb2); + } else { + /* Append to the frag_list */ + *nextp = skb2; + nextp = &skb2->next; + if (skb_has_frag_list(skb2)) + nextp = iptfs_rehome_fraglist(nextp, + skb2); + skb->truesize += skb2->truesize; + } } xfrm_output(NULL, skb);