From patchwork Mon Apr 1 21:50:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 13613046 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F93955C0A for ; Mon, 1 Apr 2024 21:50:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712008250; cv=none; b=JaU72s7dfLHoa35LshOeXZQpuv0MpRztGzJ7dK62IsQ5OrbAwOBU174Yu1wkTZMMfsFvsoSLnIU8XO/LQvNYxhgBIFM7zNmCH8QsxAgSjDXzLqG4vYWVYOY52q6lm4q98sAjsG/QYILlHf2J1FihbG7WdTLJUkEW3I1qRwtLwGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712008250; c=relaxed/simple; bh=6iZBHoWjr0SW3V6W6W27wiyE26vfswz1Bipp3C2ZJ1U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=m+58AJDvwF2+vGdLeNE3xthuwqk8sS9dhX3eXkt94PNCyrrL9zsyR1BVtz14IS5llmrT37bGgt5poBr4BUt3dDKOmXgvPhkCH65yRXPp+YJVQuIagJJSN7iG/aUZkkxL+5xvBMD5Qbp3Rycyf8pyKPksQd3FX59cwwYevuns8IY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=u3MjqiD3; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="u3MjqiD3" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dcc4563611cso6982416276.3 for ; Mon, 01 Apr 2024 14:50:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712008248; x=1712613048; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=vXRjUgtATZ0E/Y+63Z5tthtMhFwiRvlicbw1JhZ19s0=; b=u3MjqiD3cgp+NoDLJCk0nQj5dCYJggnfsnjIhVgpAoN1vhtcD+3ZlnfibfUoxvg911 KZhW1I97X3JJUQ/AcoRFlH7Z6iIjOdJALdLOg/jHQm7kqCiM9MYafdyxj+b974P8/Esh jH86/2ktJ0VAK4MjkEpoy+QeOZY258HejBFJibEHmN+uIDr0nUbJl3i6Denal+td+qEU ShJZkBcuvoN+G++MkIcKATCNm0qNYyYH3ki+xTnxLMjukV23N/EVx1TLVgzn+7FyKwbs G4U1mvpIkqB147LmGbPYuPnMqwIf1B5ZRr2JcOX+MWQCmYIXWrR3wcLpgfLlWhYZpIIJ MFYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712008248; x=1712613048; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vXRjUgtATZ0E/Y+63Z5tthtMhFwiRvlicbw1JhZ19s0=; b=VhrMOH/Y99c9/2FioI2brBpoPDLg1D94QM71oJsUsDGm+Zhe0ZByLxcuLXl22shIQu TK3cJEJULgHN92tcdRmsysarpo+Nf2qqy930tdvIJrtfKKtJRKbJoiBe0qpJD3T9BKLe M2I4s/wlqQBjd6GS6eBSIfuTFpQtoNZUCN2z5/jv01psbyAQk1vv7S7aorGlsKhRnAAF hkwuTzmrSEq+fDiyWAq4JnU9ZVqiGwkUtaoszJkj4kATSgfueQx/NcjAn3LN5sWnLMWP Fvot6pLFzdPmpFlpP5al8bYu7PEMioGN3dawTtuRQIW6UnvHQjjSZoUDLtVCQ0ZDB/uq K0wA== X-Forwarded-Encrypted: i=1; AJvYcCVocdwwaNwFwz7o+nSTV9NtNdKqQhr0+96aVnDX5ebvCwsJM2ryrZ4LPZkotOzEumfE/VGPqlQxLQ/Z8uzLbj6Q7t4TFiuOZXaH/g== X-Gm-Message-State: AOJu0YzNIuKf6VOhA/cnSrGBYuTbjQo5u8OCJG9pt5vqkY8X9v13SIVG IcqKVlmLQznY83iAvz1V6x2j13z/bf6SSaKZDFSmGLfsORCjEMIChJA1ipRwhKiP1PiQ+6lduR4 cAJHdxpD8Uq8sMGTDrmpcTg== X-Google-Smtp-Source: AGHT+IFGadfogXIS8kOngK5G9vWIutCMBx8L4gKWUrm8XeH+h8lfoSVo1ljXz4C/IkewoV3oHHDRSuJKCxCHbdkprQ== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2c4:200:b337:405b:46e7:9bd9]) (user=almasrymina job=sendgmr) by 2002:a05:6902:72d:b0:dc8:5e26:f4d7 with SMTP id l13-20020a056902072d00b00dc85e26f4d7mr3443884ybt.13.1712008248349; Mon, 01 Apr 2024 14:50:48 -0700 (PDT) Date: Mon, 1 Apr 2024 14:50:37 -0700 In-Reply-To: <20240401215042.1877541-1-almasrymina@google.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240401215042.1877541-1-almasrymina@google.com> X-Mailer: git-send-email 2.44.0.478.gd926399ef9-goog Message-ID: <20240401215042.1877541-2-almasrymina@google.com> Subject: [PATCH net-next v3 1/3] net: make napi_frag_unref reuse skb_page_unref From: Mina Almasry To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: Mina Almasry , Ayush Sawal , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Steffen Klassert , Herbert Xu , David Ahern , Boris Pismenny , John Fastabend , Dragos Tatulea , Maxim Mikityanskiy , Sabrina Dubroca , Simon Horman , Yunsheng Lin , " =?utf-8?q?Ahelenia_Ziemia=C5=84s?= =?utf-8?q?ka?= " , Christophe JAILLET , David Howells , Florian Westphal , Aleksander Lobakin , Lorenzo Bianconi , Johannes Berg , Liang Chen The implementations of these 2 functions are almost identical. Remove the implementation of napi_frag_unref, and make it a call into skb_page_unref so we don't duplicate the implementation. Signed-off-by: Mina Almasry --- include/linux/skbuff.h | 12 +++--------- net/ipv4/esp4.c | 2 +- net/ipv6/esp6.c | 2 +- 3 files changed, 5 insertions(+), 11 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index b7f1ecdaec38..a6b5596dc0cb 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3513,10 +3513,10 @@ int skb_cow_data_for_xdp(struct page_pool *pool, struct sk_buff **pskb, bool napi_pp_put_page(struct page *page, bool napi_safe); static inline void -skb_page_unref(const struct sk_buff *skb, struct page *page, bool napi_safe) +skb_page_unref(struct page *page, bool recycle, bool napi_safe) { #ifdef CONFIG_PAGE_POOL - if (skb->pp_recycle && napi_pp_put_page(page, napi_safe)) + if (recycle && napi_pp_put_page(page, napi_safe)) return; #endif put_page(page); @@ -3525,13 +3525,7 @@ skb_page_unref(const struct sk_buff *skb, struct page *page, bool napi_safe) static inline void napi_frag_unref(skb_frag_t *frag, bool recycle, bool napi_safe) { - struct page *page = skb_frag_page(frag); - -#ifdef CONFIG_PAGE_POOL - if (recycle && napi_pp_put_page(page, napi_safe)) - return; -#endif - put_page(page); + skb_page_unref(skb_frag_page(frag), recycle, napi_safe); } /** diff --git a/net/ipv4/esp4.c b/net/ipv4/esp4.c index d33d12421814..3d2c252c5570 100644 --- a/net/ipv4/esp4.c +++ b/net/ipv4/esp4.c @@ -114,7 +114,7 @@ static void esp_ssg_unref(struct xfrm_state *x, void *tmp, struct sk_buff *skb) */ if (req->src != req->dst) for (sg = sg_next(req->src); sg; sg = sg_next(sg)) - skb_page_unref(skb, sg_page(sg), false); + skb_page_unref(sg_page(sg), skb->pp_recycle, false); } #ifdef CONFIG_INET_ESPINTCP diff --git a/net/ipv6/esp6.c b/net/ipv6/esp6.c index 7371886d4f9f..4fe4f97f5420 100644 --- a/net/ipv6/esp6.c +++ b/net/ipv6/esp6.c @@ -131,7 +131,7 @@ static void esp_ssg_unref(struct xfrm_state *x, void *tmp, struct sk_buff *skb) */ if (req->src != req->dst) for (sg = sg_next(req->src); sg; sg = sg_next(sg)) - skb_page_unref(skb, sg_page(sg), false); + skb_page_unref(sg_page(sg), skb->pp_recycle, false); } #ifdef CONFIG_INET6_ESPINTCP From patchwork Mon Apr 1 21:50:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 13613047 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA6DD54FAD for ; Mon, 1 Apr 2024 21:50:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712008256; cv=none; b=qIjvtl/gEPliRgVoQkLSY0FpLj8uYh5lVfzVY5Omk1hW4MwLiX4IqiiRjBdgunfO7jKmyMUoHrI1DL5+rRNvu1a1f1CPlqv7zLwRodHcGXFCBDpsC9WcDcJPzvpfMP3VqhnaqiObR5BCVCQygTOQR0p53zVxlGzzdnGzh/pWLrU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712008256; c=relaxed/simple; bh=Al2L+U/fB7j6TwXX2+zf6Bdg1+YylbkEbJBCbRWdeqE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=USm9h+MIIXiUeBPIryaUFcLckY8816yiZXn61IF8vLEdxBJ/DoBwnqQH1UbuxcTDREieqJwusQiQ/NDQlqwcU/iAb9fkdlUX59vMW58hYbci0wgTegpBWAuFlB7ms3Yry5Jv1WK/u16NK1WuHih6KqOYazXClHDB0N9cIw6Xwb4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WuNm0p3b; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WuNm0p3b" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-60cd073522cso73351647b3.1 for ; Mon, 01 Apr 2024 14:50:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712008253; x=1712613053; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=HQgeDPgrSEDZD6GhBq5cXu5ulSSJGNPUjkrXoP9adkA=; b=WuNm0p3b/mlY4ONRslYGjy7KcdGt2h/Nb8H4dDUDHZID+GXzdxp60KSnotdKb+mj6P Z/36Qph6MhSbbQQEs5esQ6YcA8R6WbcPvlMYWrn/E1KE7JJyV9cN2KVSLdz2+71PToXg mGNsJNpXdxohfckXh+DidzBdWq0MpG9wjD8T/eGW0ZEayMSvS3iQMR69Pr0Q4WFMk+/g WxbRbn3hSCbDwJ80K40iLQsFLxj+ZMZzVpkz0Vq60WKFnwqDc06ity9YW2FQHnhOGT9+ vtiffSQwBri8LPwdQjr8FpmMxzwx84mDbtNW+nteYvIu0ipjYhj10c4sD3Dt+/hUTIPI 0Zqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712008253; x=1712613053; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HQgeDPgrSEDZD6GhBq5cXu5ulSSJGNPUjkrXoP9adkA=; b=kNXoRaVnR5dslY5gs0tKUsaie08dFiiKbNTSl/eJEy9uDuD/c4yCtKFSAAvGRYCqZH 7x4oI+f30CqhnvlsKsVZFD7/M3u9eSTTbPvspmmmq9EAWeSYm9mtxzf/z0KAOpcTgUvG H7gADYmiUK5scFUfeU+SD1PyRLP2v+jKkCGgftI1ObGnsXLKEZMTQYbU5VPAlS8v0f8i qRogxI0onnCzoys5JIl6bi4N6kt+KsMtZfYe1kptHufEh5oj3LiK4G5RujAY7Yh5CCXE KjISJYv0KzBuJk1683HQA5scn9dZXg18AZOK9tO6tcfmcYagd0xzg3I4tJnmqFOr/Pg0 8lOg== X-Forwarded-Encrypted: i=1; AJvYcCXQIZvCnWX1Op42oKBG496Ho8OBpmJ3EopAodsR+nfJBLA56tQPdE8N+CvvXowV5O+n18p/45IOc8kjgqbyvop1VVvYVPFD0zL5kw== X-Gm-Message-State: AOJu0Ywh80cAj5svkq2IHWTnkslp6MXLUZVFbHM7emMKdVcTQ7sDT6QK PGth9gbYR8ZTurYJW8TTb1W2mfNPfxCjYBoCaSp0nNQ0DDY0CS3Yte2A3xYQ4NErUAaDmKIDfuC yOkg8F6gCKpYZzjS4+EjoXw== X-Google-Smtp-Source: AGHT+IG67OeeFsTFVWjEn9D72dtBvKlCeiL9nv9cFNxpPw5Urq1GS9sn+da9FbcvzwePSAOgsA9F3XSQ8fxqW+ZtiA== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2c4:200:b337:405b:46e7:9bd9]) (user=almasrymina job=sendgmr) by 2002:a81:f910:0:b0:611:7166:1a4d with SMTP id x16-20020a81f910000000b0061171661a4dmr2790727ywm.3.1712008253032; Mon, 01 Apr 2024 14:50:53 -0700 (PDT) Date: Mon, 1 Apr 2024 14:50:38 -0700 In-Reply-To: <20240401215042.1877541-1-almasrymina@google.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240401215042.1877541-1-almasrymina@google.com> X-Mailer: git-send-email 2.44.0.478.gd926399ef9-goog Message-ID: <20240401215042.1877541-3-almasrymina@google.com> Subject: [PATCH net-next v3 2/3] net: mirror skb frag ref/unref helpers From: Mina Almasry To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: Mina Almasry , Ayush Sawal , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Steffen Klassert , Herbert Xu , David Ahern , Boris Pismenny , John Fastabend , Dragos Tatulea , Maxim Mikityanskiy , Sabrina Dubroca , Simon Horman , Yunsheng Lin , " =?utf-8?q?Ahelenia_Ziemia=C5=84s?= =?utf-8?q?ka?= " , Christophe JAILLET , David Howells , Florian Westphal , Aleksander Lobakin , Lorenzo Bianconi , Johannes Berg , Liang Chen Refactor some of the skb frag ref/unref helpers for improved clarity. Implement napi_pp_get_page() to be the mirror counterpart of napi_pp_put_page(). Implement skb_page_ref() to be the mirror of skb_page_unref(). Improve __skb_frag_ref() to become a mirror counterpart of __skb_frag_unref(). Previously unref could handle pp & non-pp pages, while the ref could only handle non-pp pages. Now both the ref & unref helpers can correctly handle both pp & non-pp pages. Now that __skb_frag_ref() can handle both pp & non-pp pages, remove skb_pp_frag_ref(), and use __skb_frag_ref() instead. This lets us remove pp specific handling from skb_try_coalesce. Signed-off-by: Mina Almasry Reviewed-by: Dragos Tatulea --- v3: - Fix build errors reported by patchwork. - Fix drivers/net/veth.c & tls_device_fallback.c callsite I missed to update. - Fix page_pool_ref_page(head_page) -> page_pool_ref_page(page) --- .../chelsio/inline_crypto/ch_ktls/chcr_ktls.c | 2 +- drivers/net/ethernet/sun/cassini.c | 4 +- drivers/net/veth.c | 2 +- include/linux/skbuff.h | 22 ++++++-- net/core/skbuff.c | 54 ++++++------------- net/tls/tls_device_fallback.c | 2 +- 6 files changed, 40 insertions(+), 46 deletions(-) diff --git a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c index 6482728794dd..f9b0a9533985 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c @@ -1658,7 +1658,7 @@ static void chcr_ktls_copy_record_in_skb(struct sk_buff *nskb, for (i = 0; i < record->num_frags; i++) { skb_shinfo(nskb)->frags[i] = record->frags[i]; /* increase the frag ref count */ - __skb_frag_ref(&skb_shinfo(nskb)->frags[i]); + __skb_frag_ref(&skb_shinfo(nskb)->frags[i], false); } skb_shinfo(nskb)->nr_frags = record->num_frags; diff --git a/drivers/net/ethernet/sun/cassini.c b/drivers/net/ethernet/sun/cassini.c index bfb903506367..fabba729e1b8 100644 --- a/drivers/net/ethernet/sun/cassini.c +++ b/drivers/net/ethernet/sun/cassini.c @@ -1999,7 +1999,7 @@ static int cas_rx_process_pkt(struct cas *cp, struct cas_rx_comp *rxc, skb->len += hlen - swivel; skb_frag_fill_page_desc(frag, page->buffer, off, hlen - swivel); - __skb_frag_ref(frag); + __skb_frag_ref(frag, false); /* any more data? */ if ((words[0] & RX_COMP1_SPLIT_PKT) && ((dlen -= hlen) > 0)) { @@ -2023,7 +2023,7 @@ static int cas_rx_process_pkt(struct cas *cp, struct cas_rx_comp *rxc, frag++; skb_frag_fill_page_desc(frag, page->buffer, 0, hlen); - __skb_frag_ref(frag); + __skb_frag_ref(frag, false); RX_USED_ADD(page, hlen + cp->crc_size); } diff --git a/drivers/net/veth.c b/drivers/net/veth.c index bcdfbf61eb66..6160a3e8d341 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -716,7 +716,7 @@ static void veth_xdp_get(struct xdp_buff *xdp) return; for (i = 0; i < sinfo->nr_frags; i++) - __skb_frag_ref(&sinfo->frags[i]); + __skb_frag_ref(&sinfo->frags[i], false); } static int veth_convert_skb_to_xdp_buff(struct veth_rq *rq, diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index a6b5596dc0cb..849d78554b50 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3483,15 +3483,29 @@ static inline struct page *skb_frag_page(const skb_frag_t *frag) return netmem_to_page(frag->netmem); } +bool napi_pp_get_page(struct page *page); + +static inline void skb_page_ref(struct page *page, bool recycle) +{ +#ifdef CONFIG_PAGE_POOL + if (recycle && napi_pp_get_page(page)) + return; +#endif + get_page(page); +} + /** * __skb_frag_ref - take an addition reference on a paged fragment. * @frag: the paged fragment + * @recycle: skb->pp_recycle param of the parent skb. False if no parent skb. * - * Takes an additional reference on the paged fragment @frag. + * Takes an additional reference on the paged fragment @frag. Obtains the + * correct reference count depending on whether skb->pp_recycle is set and + * whether the frag is a page pool frag. */ -static inline void __skb_frag_ref(skb_frag_t *frag) +static inline void __skb_frag_ref(skb_frag_t *frag, bool recycle) { - get_page(skb_frag_page(frag)); + skb_page_ref(skb_frag_page(frag), recycle); } /** @@ -3503,7 +3517,7 @@ static inline void __skb_frag_ref(skb_frag_t *frag) */ static inline void skb_frag_ref(struct sk_buff *skb, int f) { - __skb_frag_ref(&skb_shinfo(skb)->frags[f]); + __skb_frag_ref(&skb_shinfo(skb)->frags[f], skb->pp_recycle); } int skb_pp_cow_data(struct page_pool *pool, struct sk_buff **pskb, diff --git a/net/core/skbuff.c b/net/core/skbuff.c index a1be84be5d35..d878f2e67567 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -1004,6 +1004,19 @@ int skb_cow_data_for_xdp(struct page_pool *pool, struct sk_buff **pskb, EXPORT_SYMBOL(skb_cow_data_for_xdp); #if IS_ENABLED(CONFIG_PAGE_POOL) +bool napi_pp_get_page(struct page *page) +{ + + page = compound_head(page); + + if (!is_pp_page(page)) + return false; + + page_pool_ref_page(page); + return true; +} +EXPORT_SYMBOL(napi_pp_get_page); + bool napi_pp_put_page(struct page *page, bool napi_safe) { bool allow_direct = false; @@ -1056,37 +1069,6 @@ static bool skb_pp_recycle(struct sk_buff *skb, void *data, bool napi_safe) return napi_pp_put_page(virt_to_page(data), napi_safe); } -/** - * skb_pp_frag_ref() - Increase fragment references of a page pool aware skb - * @skb: page pool aware skb - * - * Increase the fragment reference count (pp_ref_count) of a skb. This is - * intended to gain fragment references only for page pool aware skbs, - * i.e. when skb->pp_recycle is true, and not for fragments in a - * non-pp-recycling skb. It has a fallback to increase references on normal - * pages, as page pool aware skbs may also have normal page fragments. - */ -static int skb_pp_frag_ref(struct sk_buff *skb) -{ - struct skb_shared_info *shinfo; - struct page *head_page; - int i; - - if (!skb->pp_recycle) - return -EINVAL; - - shinfo = skb_shinfo(skb); - - for (i = 0; i < shinfo->nr_frags; i++) { - head_page = compound_head(skb_frag_page(&shinfo->frags[i])); - if (likely(is_pp_page(head_page))) - page_pool_ref_page(head_page); - else - page_ref_inc(head_page); - } - return 0; -} - static void skb_kfree_head(void *head, unsigned int end_offset) { if (end_offset == SKB_SMALL_HEAD_HEADROOM) @@ -4195,7 +4177,7 @@ int skb_shift(struct sk_buff *tgt, struct sk_buff *skb, int shiftlen) to++; } else { - __skb_frag_ref(fragfrom); + __skb_frag_ref(fragfrom, skb->pp_recycle); skb_frag_page_copy(fragto, fragfrom); skb_frag_off_copy(fragto, fragfrom); skb_frag_size_set(fragto, todo); @@ -4845,7 +4827,7 @@ struct sk_buff *skb_segment(struct sk_buff *head_skb, } *nskb_frag = (i < 0) ? skb_head_frag_to_page_desc(frag_skb) : *frag; - __skb_frag_ref(nskb_frag); + __skb_frag_ref(nskb_frag, nskb->pp_recycle); size = skb_frag_size(nskb_frag); if (pos < offset) { @@ -5976,10 +5958,8 @@ bool skb_try_coalesce(struct sk_buff *to, struct sk_buff *from, /* if the skb is not cloned this does nothing * since we set nr_frags to 0. */ - if (skb_pp_frag_ref(from)) { - for (i = 0; i < from_shinfo->nr_frags; i++) - __skb_frag_ref(&from_shinfo->frags[i]); - } + for (i = 0; i < from_shinfo->nr_frags; i++) + __skb_frag_ref(&from_shinfo->frags[i], from->pp_recycle); to->truesize += delta; to->len += len; diff --git a/net/tls/tls_device_fallback.c b/net/tls/tls_device_fallback.c index 4e7228f275fa..d4000b4a1f7d 100644 --- a/net/tls/tls_device_fallback.c +++ b/net/tls/tls_device_fallback.c @@ -277,7 +277,7 @@ static int fill_sg_in(struct scatterlist *sg_in, for (i = 0; remaining > 0; i++) { skb_frag_t *frag = &record->frags[i]; - __skb_frag_ref(frag); + __skb_frag_ref(frag, false); sg_set_page(sg_in + i, skb_frag_page(frag), skb_frag_size(frag), skb_frag_off(frag)); From patchwork Mon Apr 1 21:50:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mina Almasry X-Patchwork-Id: 13613057 Received: from mail-io1-f73.google.com (mail-io1-f73.google.com [209.85.166.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C6094D9EF for ; Mon, 1 Apr 2024 21:57:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712008657; cv=none; b=ZNhInqaNE/CP4t0j82W5TYBqfTyobjQxwaJREFYTFwlxCTRriFzSwzOpk+qYqpSOvda4KXifKn2IGL2XBMqgWyLRLqlNYn2sZnf8HmTaNZtTamOOPTn1t8TUmDyC6k8yY1K13xjAGEPh/Tqf536zd2pR7PzQisnoHQtSJZS0ZbI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712008657; c=relaxed/simple; bh=eKbx3OApRaXqcFV5vBkjDuv3x59L9Vh/SY2/lJJ5uf4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=JUQPKFmOOtbcu0+JoJgLsKTz3/qckvF/ySBXuVpwxM0BKQTg9veACZtIuzemrn2/MZnWi574kGHDwUR3SZLDKZ/GN7Aj86SY+3MUBN5WI3vMTwP3gvUt3NNhu17RWyrgj83ydJnX4djhQX1njroyJTzzDhohQMA9DEcoD1Rh0FE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oaI434o9; arc=none smtp.client-ip=209.85.166.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oaI434o9" Received: by mail-io1-f73.google.com with SMTP id ca18e2360f4ac-7cc61b1d690so357294039f.3 for ; Mon, 01 Apr 2024 14:57:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1712008654; x=1712613454; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Zqw+BRTwGBvSeZ9P47W/5lq4WU5+34JtOjPY9Qvl6y0=; b=oaI434o9eTRcZXUGo6APreZjqeCmy0tny53qzJl/EylZIUZj9IOlREpppShPitcDeJ W4XwKGmAyHrzINUTCIrvuV6FCwBQGnxePXDkJ+2NsZRQ5VcrtVg1MLHhThflNP1gdDfO pwfELLkT43HAFfu4GD6F3eTP7Dz0H68oY3nf7GFGaIh0vcg7foV+PNxiiHy/FuKEEBCB kPylc/CSlfjHB5KvD+vGemtkAiDQdOVYrJAU9jWR6hXousAqe0uteRm821LR22XZzSzv wmBbt5DbFqD9j9gOKJgW8+XTN6lbU4FMIXXbr8zrA5PEJpCwyY4CIyOHFEegb2OPUl+Z pzDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712008654; x=1712613454; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Zqw+BRTwGBvSeZ9P47W/5lq4WU5+34JtOjPY9Qvl6y0=; b=wjKmsbx7exvF/wXPV0C5KS2cmNF3FDTlq8D5CsoXNvPosb2+tTTA9Te+C+FyEStB1C ql3u6u9D/Gi6LGN5OM2ENSd3GQycLvfgBhnG7s1BMsyJIjWY15siqGrvGeGodf/JKk3G Foxws8DOXAWhF48CmWVHZuvtjAJGvsdndyaMx1MNwXHFdcE0oZZK/XRPoNE4F0ACT/IV Uj9mhjN9ALlJyxBcMDtT0cJwtRSbb5O7h773QNaLgHIrqXR/qvrlkcxfr6+uGy6ySNa0 nMEddunwFlnv3qpFIv0/FSFxlIrd/slNXmo4Uw5S3kuCQUm+56Mxnrc6/WTkMP8noPlG o9ZQ== X-Forwarded-Encrypted: i=1; AJvYcCUUeuQLCt3rGoDatlxRASLCOYVWAhTTEleotMalTTYXqTabFRYBzMNLlxbieFEyv/gd6YtkRiWLTlBWywXXWzN3hBhZgXRfQJmn/g== X-Gm-Message-State: AOJu0Yz/A2QUMmcWe/4lDrWnm4GDKBbMS2y931oUFL/QxAF+Ueo6K59M 7PcO7oytlcRwJvIgUXICX9/S3y9kJ8S02jEyU9AXolaeXrU6eLZE9SgR0C6k++3KS1mnWElc+JY uqqeUvXNxGQpHqKnxc4sv4g== X-Google-Smtp-Source: AGHT+IHiSO1MuKLG2PKVQbdCWuwFdZapbLt2UKR3INhEcUZ7SVwCPFzN3L+SYnZHvf01xLFIZrXx+L/wRv6bW4/Bww== X-Received: from almasrymina.svl.corp.google.com ([2620:15c:2c4:200:b337:405b:46e7:9bd9]) (user=almasrymina job=sendgmr) by 2002:a05:690c:d8b:b0:614:a42b:1528 with SMTP id da11-20020a05690c0d8b00b00614a42b1528mr1480060ywb.10.1712008255028; Mon, 01 Apr 2024 14:50:55 -0700 (PDT) Date: Mon, 1 Apr 2024 14:50:39 -0700 In-Reply-To: <20240401215042.1877541-1-almasrymina@google.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240401215042.1877541-1-almasrymina@google.com> X-Mailer: git-send-email 2.44.0.478.gd926399ef9-goog Message-ID: <20240401215042.1877541-4-almasrymina@google.com> Subject: [PATCH net-next v3 3/3] net: remove napi_frag_unref From: Mina Almasry To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Cc: Mina Almasry , Ayush Sawal , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Steffen Klassert , Herbert Xu , David Ahern , Boris Pismenny , John Fastabend , Dragos Tatulea , Maxim Mikityanskiy , Sabrina Dubroca , Simon Horman , Yunsheng Lin , " =?utf-8?q?Ahelenia_Ziemia=C5=84s?= =?utf-8?q?ka?= " , Christophe JAILLET , David Howells , Florian Westphal , Aleksander Lobakin , Lorenzo Bianconi , Johannes Berg , Liang Chen With the changes in the last patches, napi_frag_unref() is now reduandant. Remove it and use skb_page_unref directly. Signed-off-by: Mina Almasry Reviewed-by: Dragos Tatulea --- drivers/net/ethernet/marvell/sky2.c | 2 +- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 2 +- include/linux/skbuff.h | 14 +++++--------- net/core/skbuff.c | 4 ++-- net/tls/tls_device.c | 2 +- net/tls/tls_strp.c | 2 +- 6 files changed, 11 insertions(+), 15 deletions(-) diff --git a/drivers/net/ethernet/marvell/sky2.c b/drivers/net/ethernet/marvell/sky2.c index 07720841a8d7..8e00a5856856 100644 --- a/drivers/net/ethernet/marvell/sky2.c +++ b/drivers/net/ethernet/marvell/sky2.c @@ -2501,7 +2501,7 @@ static void skb_put_frags(struct sk_buff *skb, unsigned int hdr_space, if (length == 0) { /* don't need this page */ - __skb_frag_unref(frag, false); + __skb_frag_unref(frag, false, false); --skb_shinfo(skb)->nr_frags; } else { size = min(length, (unsigned) PAGE_SIZE); diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index eac49657bd07..4dbf29b46979 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -526,7 +526,7 @@ static int mlx4_en_complete_rx_desc(struct mlx4_en_priv *priv, fail: while (nr > 0) { nr--; - __skb_frag_unref(skb_shinfo(skb)->frags + nr, false); + __skb_frag_unref(skb_shinfo(skb)->frags + nr, false, false); } return 0; } diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 849d78554b50..e3f40f89e15a 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3536,23 +3536,19 @@ skb_page_unref(struct page *page, bool recycle, bool napi_safe) put_page(page); } -static inline void -napi_frag_unref(skb_frag_t *frag, bool recycle, bool napi_safe) -{ - skb_page_unref(skb_frag_page(frag), recycle, napi_safe); -} - /** * __skb_frag_unref - release a reference on a paged fragment. * @frag: the paged fragment * @recycle: recycle the page if allocated via page_pool + * @napi_safe: set to true if running in the same napi context as where the + * consumer would run. * * Releases a reference on the paged fragment @frag * or recycles the page via the page_pool API. */ -static inline void __skb_frag_unref(skb_frag_t *frag, bool recycle) +static inline void __skb_frag_unref(skb_frag_t *frag, bool recycle, bool napi_safe) { - napi_frag_unref(frag, recycle, false); + skb_page_unref(skb_frag_page(frag), recycle, napi_safe); } /** @@ -3567,7 +3563,7 @@ static inline void skb_frag_unref(struct sk_buff *skb, int f) struct skb_shared_info *shinfo = skb_shinfo(skb); if (!skb_zcopy_managed(skb)) - __skb_frag_unref(&shinfo->frags[f], skb->pp_recycle); + __skb_frag_unref(&shinfo->frags[f], skb->pp_recycle, false); } /** diff --git a/net/core/skbuff.c b/net/core/skbuff.c index d878f2e67567..c4460408467e 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -1108,7 +1108,7 @@ static void skb_release_data(struct sk_buff *skb, enum skb_drop_reason reason, } for (i = 0; i < shinfo->nr_frags; i++) - napi_frag_unref(&shinfo->frags[i], skb->pp_recycle, napi_safe); + __skb_frag_unref(&shinfo->frags[i], skb->pp_recycle, napi_safe); free_head: if (shinfo->frag_list) @@ -4199,7 +4199,7 @@ int skb_shift(struct sk_buff *tgt, struct sk_buff *skb, int shiftlen) fragto = &skb_shinfo(tgt)->frags[merge]; skb_frag_size_add(fragto, skb_frag_size(fragfrom)); - __skb_frag_unref(fragfrom, skb->pp_recycle); + __skb_frag_unref(fragfrom, skb->pp_recycle, false); } /* Reposition in the original skb */ diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index bf8ed36b1ad6..5dc6381f34fb 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -140,7 +140,7 @@ static void destroy_record(struct tls_record_info *record) int i; for (i = 0; i < record->num_frags; i++) - __skb_frag_unref(&record->frags[i], false); + __skb_frag_unref(&record->frags[i], false, false); kfree(record); } diff --git a/net/tls/tls_strp.c b/net/tls/tls_strp.c index ca1e0e198ceb..85b41f226978 100644 --- a/net/tls/tls_strp.c +++ b/net/tls/tls_strp.c @@ -196,7 +196,7 @@ static void tls_strp_flush_anchor_copy(struct tls_strparser *strp) DEBUG_NET_WARN_ON_ONCE(atomic_read(&shinfo->dataref) != 1); for (i = 0; i < shinfo->nr_frags; i++) - __skb_frag_unref(&shinfo->frags[i], false); + __skb_frag_unref(&shinfo->frags[i], false, false); shinfo->nr_frags = 0; if (strp->copy_mode) { kfree_skb_list(shinfo->frag_list);