From patchwork Tue Feb 18 05:01:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979003 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F05019F104; Tue, 18 Feb 2025 05:02:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854931; cv=none; b=LXzOEbFllqhIiV4xPZjT5aMoyQW0787TQ0Wjmtw39h3ONRuIIWfujqjBaj+BD5bf9xAKe2hZ58T3FHCnqn7X+Q/BEq+xCI1PJLjc0x4vd6nBpmM8cVKR4Gp0sq6yCZCp2u66YElnF0jFbGEZCTgRIVQvetLOf300LRJ0G0ExBYI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854931; c=relaxed/simple; bh=DDtM+vZerkMryfJ9Kp2UIWhKVpj3cuPoaYpevaSK4/E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YhNoYPY9yQI5FXwFcg+dvP5EwpAWMxbC8aerkeNJXQSn+U2vKPozedUIaKJ915ZPDTps6gKRxFincbCNGF5T2+Ylxh5MH5EzVeeWDhBAbtAQwH2Ly4Gkl5evYWcBaPuy5D7Ujwo6t33rowYxMLyfwczrutT1DFRreOuDIPyQZYs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bn6Q2nJ5; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bn6Q2nJ5" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-220ca204d04so66637115ad.0; Mon, 17 Feb 2025 21:02:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854929; x=1740459729; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Kfb9GI6feOAbgYzfRSzf+rdXBFIMELjuaFO2C7wE51E=; b=bn6Q2nJ5pNnwsp2fELaLM1p2dni2vTUwwbQf+xpbFoHGvxKPOI1wI0q+IFSceznGBW IBbL3p6rrNrwP1p0Fasr5dU9TKfylHSYMHI2OQzUOgJdvlPH+txMYSkKl32v+QgrR6tA LIueepxvQ2L3Xdj8lBu92d/LaHHdm5QNiH01iBmbJjN4BV5MEsdDnsg0rVzr8YbTUzAN 15ROBCkpLDh60ev7NaoHXQtxd5QI5vF2Kd6LkS+nfATbZ48j9sx5ImNvdHg+PrU0yEIH cu6Qs9QpGikNNHXTizU4tKmgf5HQUXkYa4spb/lL+QUyNESZyBCGcb3VjvEmVtKKnzNw 9IwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854929; x=1740459729; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Kfb9GI6feOAbgYzfRSzf+rdXBFIMELjuaFO2C7wE51E=; b=ZInfXf29HwLSHQREEArEiMsEsi4yBwAgRWfbT6+7SBTVXX+EWGJRgwRBA0NgFHga47 2cp1LbZaVPqlE2TBScfUM6jyia08QgTBoyf2wgxTqInyfdm4fnqL0DEL36Mhuu9ik8eB zWBB200019Gi01p1RaVe9WMt4HIRU5vPLx2h0jbOY+NOKBM0ut1s7fPYfmI7OBw/fp1p 8RFxFy5Zre3azwZfevBm774J+9dJL0tHLA9+5w7p56283ZWPovXBAVajrhulAawqmXkd h2zFXKGWV1dwjk0wST2Y+C34C0+JnR9WEVHbITgwgqptej64WnC8Yj6VSfZgYjE97guL K64w== X-Forwarded-Encrypted: i=1; AJvYcCXCe5avnOIRSooooy0hnDSo8v8vk4OApJqIP89NItgI5B5ixzOOCMjO0kNCC/Gn0RbR3l4qxGw=@vger.kernel.org X-Gm-Message-State: AOJu0Yz0IGOdhWrFv1W6x7D9796IaOioHT6rx1x1IK4Ns/RxsX7w7MPI OiS8HAdvNxVqKh9on+wqtIeUwkpAyY48+F9p9jxLKgC7sB84hxbJ X-Gm-Gg: ASbGncuNwY+XhIX516Bf7NBwGg1BWCbZ+/zsUA5vk2A+4xAwa8FN1NNIhLokGYoyeVK qcM8LcfQeGeU46g7lN1vl65XLrrCMm1D37fjokik5kphgjlKWb6gkwCTHfZodzWSUfQJnZ4wu2y wny3cac86K6QXTMtsMjc3s29pioQP9P9P/Z8u5d3kDk7uQI6FjE9bnO9LpcPIYt4m361QkoNpxg +HZheYZ6N2sq9xnqMclgfhZDBiCgZBLJ6Y2r5PMtG5aiD4RjyS3o71FR7AjhJx7vQzLfAr4SvG3 4YrrrkKIB/fXZCnm4Zz69JEqMSJYZIW+9nLQx3vSFTM10kLKmpazFCQTldEdVnI= X-Google-Smtp-Source: AGHT+IH8asg/MVoVMADaOjKCy5UOrL/BXtdMPTP4xOEZEniFcwiAze36Tys5tzX6YtzxY5REtK4lbg== X-Received: by 2002:a05:6a20:a120:b0:1ee:7fa1:9156 with SMTP id adf61e73a8af0-1ee8cb0e995mr23992033637.3.1739854929546; Mon, 17 Feb 2025 21:02:09 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:09 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 01/12] bpf: add networking timestamping support to bpf_get/setsockopt() Date: Tue, 18 Feb 2025 13:01:14 +0800 Message-Id: <20250218050125.73676-2-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The new SK_BPF_CB_FLAGS and new SK_BPF_CB_TX_TIMESTAMPING are added to bpf_get/setsockopt. The later patches will implement the BPF networking timestamping. The BPF program will use bpf_setsockopt(SK_BPF_CB_FLAGS, SK_BPF_CB_TX_TIMESTAMPING) to enable the BPF networking timestamping on a socket. Signed-off-by: Jason Xing --- include/net/sock.h | 3 +++ include/uapi/linux/bpf.h | 8 ++++++++ net/core/filter.c | 23 +++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 1 + 4 files changed, 35 insertions(+) diff --git a/include/net/sock.h b/include/net/sock.h index 8036b3b79cd8..7916982343c6 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -303,6 +303,7 @@ struct sk_filter; * @sk_stamp: time stamp of last packet received * @sk_stamp_seq: lock for accessing sk_stamp on 32 bit architectures only * @sk_tsflags: SO_TIMESTAMPING flags + * @sk_bpf_cb_flags: used in bpf_setsockopt() * @sk_use_task_frag: allow sk_page_frag() to use current->task_frag. * Sockets that can be used under memory reclaim should * set this to false. @@ -445,6 +446,8 @@ struct sock { u32 sk_reserved_mem; int sk_forward_alloc; u32 sk_tsflags; +#define SK_BPF_CB_FLAG_TEST(SK, FLAG) ((SK)->sk_bpf_cb_flags & (FLAG)) + u32 sk_bpf_cb_flags; __cacheline_group_end(sock_write_rxtx); __cacheline_group_begin(sock_write_tx); diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index fff6cdb8d11a..fa666d51dffe 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -6916,6 +6916,13 @@ enum { BPF_SOCK_OPS_ALL_CB_FLAGS = 0x7F, }; +/* Definitions for bpf_sk_cb_flags */ +enum { + SK_BPF_CB_TX_TIMESTAMPING = 1<<0, + SK_BPF_CB_MASK = (SK_BPF_CB_TX_TIMESTAMPING - 1) | + SK_BPF_CB_TX_TIMESTAMPING +}; + /* List of known BPF sock_ops operators. * New entries can only be added at the end */ @@ -7094,6 +7101,7 @@ enum { TCP_BPF_SYN_IP = 1006, /* Copy the IP[46] and TCP header */ TCP_BPF_SYN_MAC = 1007, /* Copy the MAC, IP[46], and TCP header */ TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */ + SK_BPF_CB_FLAGS = 1009, /* Used to set socket bpf flags */ }; enum { diff --git a/net/core/filter.c b/net/core/filter.c index 2ec162dd83c4..1c6c07507a78 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5222,6 +5222,25 @@ static const struct bpf_func_proto bpf_get_socket_uid_proto = { .arg1_type = ARG_PTR_TO_CTX, }; +static int sk_bpf_set_get_cb_flags(struct sock *sk, char *optval, bool getopt) +{ + u32 sk_bpf_cb_flags; + + if (getopt) { + *(u32 *)optval = sk->sk_bpf_cb_flags; + return 0; + } + + sk_bpf_cb_flags = *(u32 *)optval; + + if (sk_bpf_cb_flags & ~SK_BPF_CB_MASK) + return -EINVAL; + + sk->sk_bpf_cb_flags = sk_bpf_cb_flags; + + return 0; +} + static int sol_socket_sockopt(struct sock *sk, int optname, char *optval, int *optlen, bool getopt) @@ -5238,6 +5257,7 @@ static int sol_socket_sockopt(struct sock *sk, int optname, case SO_MAX_PACING_RATE: case SO_BINDTOIFINDEX: case SO_TXREHASH: + case SK_BPF_CB_FLAGS: if (*optlen != sizeof(int)) return -EINVAL; break; @@ -5247,6 +5267,9 @@ static int sol_socket_sockopt(struct sock *sk, int optname, return -EINVAL; } + if (optname == SK_BPF_CB_FLAGS) + return sk_bpf_set_get_cb_flags(sk, optval, getopt); + if (getopt) { if (optname == SO_BINDTODEVICE) return -EINVAL; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 2acf9b336371..70366f74ef4e 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7091,6 +7091,7 @@ enum { TCP_BPF_SYN_IP = 1006, /* Copy the IP[46] and TCP header */ TCP_BPF_SYN_MAC = 1007, /* Copy the MAC, IP[46], and TCP header */ TCP_BPF_SOCK_OPS_CB_FLAGS = 1008, /* Get or Set TCP sock ops flags */ + SK_BPF_CB_FLAGS = 1009, /* Used to set socket bpf flags */ }; enum { From patchwork Tue Feb 18 05:01:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979004 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4418C1AF0B5; Tue, 18 Feb 2025 05:02:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854937; cv=none; b=oztRBTzVTjQIzanGoUBygB9TqO0+ZtKciihcHbblGyrxnqkr7QxVayp6I29YrzreIT1+qPTJog3kvZhYXi72Dy8o+OKS6slZxzFUb0zshKur/cqoLQlSeKlCr5KdAcbHSaChs5cZjaMan4SGiECJVFKdX2mPGERNd491HBPfAIA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854937; c=relaxed/simple; bh=xKbudb0FUrth0r9fVulR0YQSW9YlrDYUxAjIB9W3r94=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oBRXg+JHPwlenbd2o/1dE6YH1TQAxu2NIscqkq8iF8kH5k4eH2JhF6WlArctMcJZK4pQKbjecKVeyEmgSZAvlvz//0B+RYX3f8lLiT2TK+N5WTFvfgZ3FtWiFrrtury4taGnetvfFuIfaB9v0sHOSRR254lK4xb3rBhCFoI6gIE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GDLEyWJv; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GDLEyWJv" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-220c8cf98bbso101135775ad.1; Mon, 17 Feb 2025 21:02:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854935; x=1740459735; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nCClYBX4dv1juFzLnShvLFoS82zgt+5gs+BbDuAzfbU=; b=GDLEyWJvLOEbe3puWmfBBA1hCImQeI3tUiJGy1ReVIu1qiv6LUbM2YjTrdMhQYtB43 34Hz9hWAqHHXiAv9oUbfAb4DIbUu08wMxiirf33Y6KYbsujpCNRWVOa6B4SIc24EdLR+ ea4kbKOp8UKO6XMGlYmW72LVxC97N96EYrSb8mdgIA0s930L4dy4YfHUrH3O69mw+RF4 hgTG2Yxj/+XqeLvZtwGoS8wdiJ58Qo5XMzk/PT3PZgEFHSP9GZ3Y7SNVjOi3JBTVTYNk eh2MqSBVz+yiqAUCZzN8sgBe/N6p3pLagj8l8R1Mx1rp5vhulS/+zTZ6J0cSHbnBLTRX hdcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854935; x=1740459735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nCClYBX4dv1juFzLnShvLFoS82zgt+5gs+BbDuAzfbU=; b=xCy/NsE+lxSKy50YfvnMCNFnLw5hAuAC8yrZsKeuTMzYo3VecnoSgs70FgFnXw6+0X FsmOO0vleh08DN/643NFg3EQWaz0RF1hcZpETU3KJUcUtbJ830iBnGwJRZPOMnozcKc5 eTpZELttxAmxTiEv1YeUIiruQoUA4hZXarCAiM0Qm+1BbpzXX4a6L6Ey8BYlOPeeXocW UDFo92x2wtkGhoJHUxpN7ojaaF7V5DWu0WIwVUpXhDFv2kp5q7cTJzcQJP1RfQpFkqJ6 nCzsYSEvdNe//+lNk50/U+lx8OrfROger8Q5VjeIJSIJ9bd9I/rOXIRXg5Tz7ckxsYlA mv5w== X-Forwarded-Encrypted: i=1; AJvYcCVcZOFJ5fB3qWKcCq4Nv2VLuqe6Qabx6SiEd8VG09RhKoeoiV0n5BzrYQYCuEZlhAssgt9Vkn8=@vger.kernel.org X-Gm-Message-State: AOJu0YxvGPNo6TdS2Es84zt6jRdm9DVFHqk6Rgrdas8h2dhQVGOWpQcu QR6rsk55p93Za0m7c8ioBhEj5DTVeiR419iYlEp6qtErKR8fzYF+R47d/mwBi+Ewcw== X-Gm-Gg: ASbGncucY2rBg8knDspNP9jpShpvRG11Y705jFgmb78H/HfvMrF7CQHjxgu5GmGscor BZNQ6hjAtXV9xy9X1ZPUA7La1XN56rYEHufodMZFQ3v2NfTmHXBZAIbhXjPznt1vX2A7/diEbiN sRmeCLYF5eJAcs3I0zhxcOhU57A71Sk1cSiOCR1P3PwBngXSkosgVufaytWGDUEnOaatIDy9ZP4 htklOPpL9FprTiPfxjF8ErByDYCMwITSaZyW1qw1cr7/nS44S+6C+YOD09eJ3I0Sc/RxizY0lXK 0W8G46GRIfJe0VwQ4QEKQDHuwRoaqAUGqzu7/M6LU3/fqT8DDoxEY7IlWIVDtQ4= X-Google-Smtp-Source: AGHT+IH/L1/jtW7zZyUJGIuQ9OaKIw5soUDOH3dF6adpivdAR4s+e8QQHBDG80QcRAsDWNgb/IigSQ== X-Received: by 2002:a05:6a00:4b16:b0:732:1eb2:7bf3 with SMTP id d2e1a72fcca58-7326190da79mr19491632b3a.21.1739854935456; Mon, 17 Feb 2025 21:02:15 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:15 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 02/12] bpf: prepare the sock_ops ctx and call bpf prog for TX timestamping Date: Tue, 18 Feb 2025 13:01:15 +0800 Message-Id: <20250218050125.73676-3-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This patch introduces a new bpf_skops_tx_timestamping() function that prepares the "struct bpf_sock_ops" ctx and then executes the sockops BPF program. The subsequent patch will utilize bpf_skops_tx_timestamping() at the existing TX timestamping kernel callbacks (__sk_tstamp_tx specifically) to call the sockops BPF program. Later, four callback points to report information to user space based on this patch will be introduced. Signed-off-by: Jason Xing --- include/net/sock.h | 7 +++++++ net/core/sock.c | 14 ++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/include/net/sock.h b/include/net/sock.h index 7916982343c6..6f4d54faba92 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -2923,6 +2923,13 @@ int sock_set_timestamping(struct sock *sk, int optname, struct so_timestamping timestamping); void sock_enable_timestamps(struct sock *sk); +#if defined(CONFIG_CGROUP_BPF) +void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op); +#else +static inline void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op) +{ +} +#endif void sock_no_linger(struct sock *sk); void sock_set_keepalive(struct sock *sk); void sock_set_priority(struct sock *sk, u32 priority); diff --git a/net/core/sock.c b/net/core/sock.c index eae2ae70a2e0..bde45569d4da 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -948,6 +948,20 @@ int sock_set_timestamping(struct sock *sk, int optname, return 0; } +#if defined(CONFIG_CGROUP_BPF) +void bpf_skops_tx_timestamping(struct sock *sk, struct sk_buff *skb, int op) +{ + struct bpf_sock_ops_kern sock_ops; + + memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); + sock_ops.op = op; + sock_ops.is_fullsock = 1; + sock_ops.sk = sk; + bpf_skops_init_skb(&sock_ops, skb, 0); + __cgroup_bpf_run_filter_sock_ops(sk, &sock_ops, CGROUP_SOCK_OPS); +} +#endif + void sock_set_keepalive(struct sock *sk) { lock_sock(sk); From patchwork Tue Feb 18 05:01:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979005 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 478831AE01C; Tue, 18 Feb 2025 05:02:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854944; cv=none; b=Cgo1yPBxONL1bym5Imh12YR6Sz2uGeNyJMzqYDSOpf7xCiRJARAqsrxj7H4Ni8ETgcyFb4MvK93KLYtjTvCBtBSSm5JRjl457LDwsSVWzvB3rsmI+wFgl0HOg8y6hvzfJudJooSCI13MSpl/j9g5SssHs7+aVQ+bC9ehbJ7SygE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854944; c=relaxed/simple; bh=bKrm1hiTC/nINgvY+o1Fo1gLAQbsMuznKHX0Uw3gAzY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eWgfAfALewEZphw89kV4wSmmqtcsGPE1UBhXhTQOPuHAwzO6xs5/BIBwir+1fy60iODXUfWuKj/xyBFcQIjh5kJ/M2FQpQsHP88tcmgdy9tNNdW7kMAFa8ivPdE/l51i+1i4xLcT8tnYOnUbcw2bKEkpYVnaQc3jOQZKGY/TUpI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OfFjySqH; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OfFjySqH" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-220c665ef4cso73637725ad.3; Mon, 17 Feb 2025 21:02:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854941; x=1740459741; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7Alr0Cy5SCB+CgKRZoduXfthZLgwnrJQ/3rmN5eLpq4=; b=OfFjySqHf6cMXtVeCBJBSt5E7JrhhGjhIr2iqrJm+9asTxO68ETHHjXGNSWxpkZFMI NbbCdGuKGXRL8YbqpIwD4MKz0YyjzU4KS9GkJkDb2BM6SH1wu5ovzPXWW9LB8X284YWs 8zWz8OR0Kc4iNjo5JHKDAYODukWUYaYp6XxvJA1FAPSUxCKHzNdSWX7Zy1b50TnLF53E LeNn3X/yhENpgFfgWYOx8MvkkkZMGux29ddWf056wsS81Q2oDDNWWLaZONSDPdDM73lI NT1fKW8ohaB7nFAtZ8+c4z6dJOO5rQOCcrKAVM8ql+NyKvO7yrdH+puOn35whN+RPAhE ZMNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854941; x=1740459741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7Alr0Cy5SCB+CgKRZoduXfthZLgwnrJQ/3rmN5eLpq4=; b=Z+uzjpyVVtcEwXFdd153Y44DMaGFYrBkKhB6FhH8Vz3ACgdN3kkWr9nPLRYIxl9F85 n7+ydY74UnM5DWLwYlrCA0TGnXxGE7tVB725kIRW12Fnrmsa5ZQS3OCdIDZRRGaOgjiS RnR7oMoTh+chSVpMgpc4bHTPIGuyPUZLBeEG/H7HEFheTUFD1ISPNB0y9Yqy1HAunpKO 4mrh64Pgi1/x0i94wvXKRnJZ8lVFZ8l/VehmKMW3EkcM85iJpRmMHHadbOGdwfwpbnYu oB4G6b+OSPBDFRNbB2vTudox9Ck4dPUyMqZP5CiDraw3PYW0aJJQ787NVbvCCF7wnlfq eDzg== X-Forwarded-Encrypted: i=1; AJvYcCWVrZbzqUKRyWmNQEiMVGU9zk1TfSvcUpnOdCgVeldgcJsg5j/Ua869hsxagn54EohsiWwurl0=@vger.kernel.org X-Gm-Message-State: AOJu0YyyDZeylWDbDREuPSOeGM2pb8wrBn+ix85pE+S7njqYrZNODZkH mP6esCXWbD/i/vpms3wE+wXFa/87WFgaDTKUKKPnxxIWK92CNXyu X-Gm-Gg: ASbGncvsIgWMCAbk+86pSxtabx09zV7sL+yv+aDIQEZbdgzn/fSxiLa5WJVwFAyEDCt iXN6YeHeuSagTNtpIz0yis6YemX5o0plEIXigUuQdH+NyIYO/N1wVHdACPqsqjrfS7SYjac07E6 dv6OnOvpfw9+Pu8eqOBYJ6ybXfmqS9Yw2fsPYo6Q5lohwK+Gn9qS211nzoCadA7C80mRRrliZVJ znce8nk4x82FbQL9jbesS4N0HgfBjOSZh7dIndhX80Xa/aLDkS/unVMW0/Nz/eiI4hM0kOqQdWq tiRQpVlfppCjQ4AvYxtDjVayY3mRIlKNzJ7L+BF7ECr3+gk9S3S9hj2u/W45mNk= X-Google-Smtp-Source: AGHT+IGaWyF5GNI0ZDtgXzpTHIDHF5EAZDNjVL9/zkUW/fyN6Nq8v8afcyhoXBQ2Evf1my7QRCfsBA== X-Received: by 2002:a05:6a20:2d23:b0:1e1:ce4d:a144 with SMTP id adf61e73a8af0-1ee8c98f0b1mr22599545637.0.1739854941433; Mon, 17 Feb 2025 21:02:21 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:21 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 03/12] bpf: prevent unsafe access to the sock fields in the BPF timestamping callback Date: Tue, 18 Feb 2025 13:01:16 +0800 Message-Id: <20250218050125.73676-4-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The subsequent patch will implement BPF TX timestamping. It will call the sockops BPF program without holding the sock lock. This breaks the current assumption that all sock ops programs will hold the sock lock. The sock's fields of the uapi's bpf_sock_ops requires this assumption. To address this, a new "u8 is_locked_tcp_sock;" field is added. This patch sets it in the current sock_ops callbacks. The "is_fullsock" test is then replaced by the "is_locked_tcp_sock" test during sock_ops_convert_ctx_access(). The new TX timestamping callbacks added in the subsequent patch will not have this set. This will prevent unsafe access from the new timestamping callbacks. Potentially, we could allow read-only access. However, this would require identifying which callback is read-safe-only and also requires additional BPF instruction rewrites in the covert_ctx. Since the BPF program can always read everything from a socket (e.g., by using bpf_core_cast), this patch keeps it simple and disables all read and write access to any socket fields through the bpf_sock_ops UAPI from the new TX timestamping callback. Moreover, note that some of the fields in bpf_sock_ops are specific to tcp_sock, and sock_ops currently only supports tcp_sock. In the future, UDP timestamping will be added, which will also break this assumption. The same idea used in this patch will be reused. Considering that the current sock_ops only supports tcp_sock, the variable is named is_locked_"tcp"_sock. Signed-off-by: Jason Xing --- include/linux/filter.h | 1 + include/net/tcp.h | 1 + net/core/filter.c | 8 ++++---- net/ipv4/tcp_input.c | 2 ++ net/ipv4/tcp_output.c | 2 ++ 5 files changed, 10 insertions(+), 4 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index a3ea46281595..d36d5d5180b1 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -1508,6 +1508,7 @@ struct bpf_sock_ops_kern { void *skb_data_end; u8 op; u8 is_fullsock; + u8 is_locked_tcp_sock; u8 remaining_opt_len; u64 temp; /* temp and everything after is not * initialized to 0 before calling diff --git a/include/net/tcp.h b/include/net/tcp.h index 5b2b04835688..4c4dca59352b 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -2649,6 +2649,7 @@ static inline int tcp_call_bpf(struct sock *sk, int op, u32 nargs, u32 *args) memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); if (sk_fullsock(sk)) { sock_ops.is_fullsock = 1; + sock_ops.is_locked_tcp_sock = 1; sock_owned_by_me(sk); } diff --git a/net/core/filter.c b/net/core/filter.c index 1c6c07507a78..8631036f6b64 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -10381,10 +10381,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, } \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, \ - is_fullsock), \ + is_locked_tcp_sock), \ fullsock_reg, si->src_reg, \ offsetof(struct bpf_sock_ops_kern, \ - is_fullsock)); \ + is_locked_tcp_sock)); \ *insn++ = BPF_JMP_IMM(BPF_JEQ, fullsock_reg, 0, jmp); \ if (si->dst_reg == si->src_reg) \ *insn++ = BPF_LDX_MEM(BPF_DW, reg, si->src_reg, \ @@ -10469,10 +10469,10 @@ static u32 sock_ops_convert_ctx_access(enum bpf_access_type type, temp)); \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, \ - is_fullsock), \ + is_locked_tcp_sock), \ reg, si->dst_reg, \ offsetof(struct bpf_sock_ops_kern, \ - is_fullsock)); \ + is_locked_tcp_sock)); \ *insn++ = BPF_JMP_IMM(BPF_JEQ, reg, 0, 2); \ *insn++ = BPF_LDX_MEM(BPF_FIELD_SIZEOF( \ struct bpf_sock_ops_kern, sk),\ diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index eb82e01da911..95733dcdfb4b 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -169,6 +169,7 @@ static void bpf_skops_parse_hdr(struct sock *sk, struct sk_buff *skb) memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); sock_ops.op = BPF_SOCK_OPS_PARSE_HDR_OPT_CB; sock_ops.is_fullsock = 1; + sock_ops.is_locked_tcp_sock = 1; sock_ops.sk = sk; bpf_skops_init_skb(&sock_ops, skb, tcp_hdrlen(skb)); @@ -185,6 +186,7 @@ static void bpf_skops_established(struct sock *sk, int bpf_op, memset(&sock_ops, 0, offsetof(struct bpf_sock_ops_kern, temp)); sock_ops.op = bpf_op; sock_ops.is_fullsock = 1; + sock_ops.is_locked_tcp_sock = 1; sock_ops.sk = sk; /* sk with TCP_REPAIR_ON does not have skb in tcp_finish_connect */ if (skb) diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c index bc95d2a5924f..a0e779bdbc6b 100644 --- a/net/ipv4/tcp_output.c +++ b/net/ipv4/tcp_output.c @@ -525,6 +525,7 @@ static void bpf_skops_hdr_opt_len(struct sock *sk, struct sk_buff *skb, sock_owned_by_me(sk); sock_ops.is_fullsock = 1; + sock_ops.is_locked_tcp_sock = 1; sock_ops.sk = sk; } @@ -570,6 +571,7 @@ static void bpf_skops_write_hdr_opt(struct sock *sk, struct sk_buff *skb, sock_owned_by_me(sk); sock_ops.is_fullsock = 1; + sock_ops.is_locked_tcp_sock = 1; sock_ops.sk = sk; } From patchwork Tue Feb 18 05:01:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979006 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2ED6AECF; Tue, 18 Feb 2025 05:02:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854949; cv=none; b=ib9lQZqt2hiAPYRAGlDfvHk9bhTYnFPp62u7KwMvT61jzG7FQlNm/cx/RNmSpj3HkjupL2dri1LIntMcoPk5b6GOKZjq9dK4VyFlOC5Q93vstJ79N8aAPJupHAF/iWZk6UkEDBWrfU0LsXYQ0hLcXOnXA38QZuDHUwC2WgwJNqM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854949; c=relaxed/simple; bh=3f4fr3wEVqOkIsOc+1STd0Ns5iBgLGFkFca2uJbMweM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ff0hGcYOavJUQifRWv0RqKGL9N6kii3SPGfKS/yP6KuG1FbYxi0beOJiEN/NKGdAEINIaGbXPPp0flQntDDtpWGH6rZ8o8vL+LM2ngZuU2/czk11G+C8U30Mmxt9/8bWxpTGnr52skuFNwVDFYBkdFjciAMLATB4NhPllMM29pY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kET4Ngg6; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kET4Ngg6" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-22101839807so60018265ad.3; Mon, 17 Feb 2025 21:02:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854947; x=1740459747; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/jcgdad4TkftRTh1q65WudnE3mEROlme3+Pzdel+ZVs=; b=kET4Ngg6A00p2M0ibwTCCWGHPHWqzQwrySLgJ+oXH+hJdsGMxUu1lMtznP6+jvP4d4 We4d/fbPaMy2W7nStqCpJltvLGmW894n7osSStuqV4lpOxD9MilE72+ShwGDPvs0Dj1e VpgO4/xI5oV9ItLpkHQWDCbzCOGFd5XZKrfsyFR9zh/qvvvdiodjG3TjRxPKh/T1PaHd OiY3ruCX5C4whmWtOOs244UVf7YZzWJBnyUi5t+z7AWDWuCFg/tADHd01ok2l4mdCVnl HreFCqyByz3HWgcVPpTR0GlDVA6aL9X2sYGt8S/y55XR5vOWR6VpGXPIyNLpFS949X64 7MYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854947; x=1740459747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/jcgdad4TkftRTh1q65WudnE3mEROlme3+Pzdel+ZVs=; b=dmxNJ9ZksVI9v3EOov3fhbbo/qjP3iB6S5yEYB+dS/z7kvp3RSirqRpH5NqTNpDTtj oTgwMufpwYyk7ZiasrjirMQOmmuveTPnVztQN/TD/fN/uWLuTC7f28dzKEpyA3FcTfv+ 5sztn8B7pT6ITX0fc5yOiBgZcQkoQ26cFPTvwDN5o+Rxc6/wp4D40rbD08W6te0MonpC Ox2+toAG8oZI1OongjS2An3TfZkUcdKeczpQ/fHdcJG1T/cGwUqbMa8DdBe4lBQWKnWX jOL1tmqCg2rhwsLCvVKp2wscxdyv8BZvAuEObf1VNwcUwo8RFJWU5SnehVYjBMQPzrpf gFNg== X-Forwarded-Encrypted: i=1; AJvYcCWtEaJmhdQcKrix4DTdm/MOfd3dx9MpioSn1enSx0uvYgeItq/VANVk3p1yjrgpQxzTc0J76xk=@vger.kernel.org X-Gm-Message-State: AOJu0YzVEPXna3EWqmK10mokG8rRHrWEcIt8r3ITqb0yr+9xqGtdFzAH bgXPqYXArkRfDzITvrOmJwQYpGbTXrGGUmqYCtcOLYXemu26viSVde91no9TS4ZCcg== X-Gm-Gg: ASbGncsv9VLlSGYk4pOd7Gp//9l3EouhHjlTKEpvr93OnCUs6QJJJn9OuHDBPpwqcnZ 2uIrbK8gULUWBbyiFlsIgv3LttNYx1NwQARQcG0cP4LuhA/KGAiHQxIFocVZqEaqZCtFu+dFWS7 2ORMj83iQQwo7CLXQyfjdcED41iuFCA1/M2I2eq+oR145TflhWBPIzyJomK6r+cBtioiVCTz9T1 xhHUjl/qDyn8kgGzqezTXpAJtfQaX5DHVD9ZLnZ+OETExz/07bCq9EfI76TvuoJBfiu6e5lOppz wGdpZVEwZkyIiMDYKaTi87CyWpjSk9EtvR91Vzf8OczmbXmTfybA38cU+/ErfFo= X-Google-Smtp-Source: AGHT+IGDlDQtZ2/iCjhv8oUTNh8upPD6clzP6SkxIJRB0Kv9p9JbKOPeLrkNgWpE8H3l9IrFrx8inw== X-Received: by 2002:a05:6a21:3994:b0:1ee:6e65:6335 with SMTP id adf61e73a8af0-1ee8cbbd682mr25774380637.29.1739854947482; Mon, 17 Feb 2025 21:02:27 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:27 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 04/12] bpf: disable unsafe helpers in TX timestamping callbacks Date: Tue, 18 Feb 2025 13:01:17 +0800 Message-Id: <20250218050125.73676-5-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net New TX timestamping sock_ops callbacks will be added in the subsequent patch. Some of the existing BPF helpers will not be safe to be used in the TX timestamping callbacks. The bpf_sock_ops_setsockopt, bpf_sock_ops_getsockopt, and bpf_sock_ops_cb_flags_set require owning the sock lock. TX timestamping callbacks will not own the lock. The bpf_sock_ops_load_hdr_opt needs the skb->data pointing to the TCP header. This will not be true in the TX timestamping callbacks. At the beginning of these helpers, this patch checks the bpf_sock->op to ensure these helpers are used by the existing sock_ops callbacks only. Signed-off-by: Jason Xing --- net/core/filter.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/core/filter.c b/net/core/filter.c index 8631036f6b64..7f56d0bbeb00 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5523,6 +5523,11 @@ static int __bpf_setsockopt(struct sock *sk, int level, int optname, return -EINVAL; } +static bool is_locked_tcp_sock_ops(struct bpf_sock_ops_kern *bpf_sock) +{ + return bpf_sock->op <= BPF_SOCK_OPS_WRITE_HDR_OPT_CB; +} + static int _bpf_setsockopt(struct sock *sk, int level, int optname, char *optval, int optlen) { @@ -5673,6 +5678,9 @@ static const struct bpf_func_proto bpf_sock_addr_getsockopt_proto = { BPF_CALL_5(bpf_sock_ops_setsockopt, struct bpf_sock_ops_kern *, bpf_sock, int, level, int, optname, char *, optval, int, optlen) { + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + return _bpf_setsockopt(bpf_sock->sk, level, optname, optval, optlen); } @@ -5758,6 +5766,9 @@ static int bpf_sock_ops_get_syn(struct bpf_sock_ops_kern *bpf_sock, BPF_CALL_5(bpf_sock_ops_getsockopt, struct bpf_sock_ops_kern *, bpf_sock, int, level, int, optname, char *, optval, int, optlen) { + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + if (IS_ENABLED(CONFIG_INET) && level == SOL_TCP && optname >= TCP_BPF_SYN && optname <= TCP_BPF_SYN_MAC) { int ret, copy_len = 0; @@ -5800,6 +5811,9 @@ BPF_CALL_2(bpf_sock_ops_cb_flags_set, struct bpf_sock_ops_kern *, bpf_sock, struct sock *sk = bpf_sock->sk; int val = argval & BPF_SOCK_OPS_ALL_CB_FLAGS; + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + if (!IS_ENABLED(CONFIG_INET) || !sk_fullsock(sk)) return -EINVAL; @@ -7609,6 +7623,9 @@ BPF_CALL_4(bpf_sock_ops_load_hdr_opt, struct bpf_sock_ops_kern *, bpf_sock, u8 search_kind, search_len, copy_len, magic_len; int ret; + if (!is_locked_tcp_sock_ops(bpf_sock)) + return -EOPNOTSUPP; + /* 2 byte is the minimal option len except TCPOPT_NOP and * TCPOPT_EOL which are useless for the bpf prog to learn * and this helper disallow loading them also. From patchwork Tue Feb 18 05:01:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979007 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AA83ECF; Tue, 18 Feb 2025 05:02:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854955; cv=none; b=p8gniiyt+/ES7Hv0e7yAD/uCsZHL9w1+k29MHaLjfS6fKqMTUkcsFIuZWOrQnbx+WXhZpYy+GwxfyxcvH0Qupi6x4LyHLj8b1r0s5f9LE3lfzlK6ToVYB/TNzm9NI8gTyyCV0OrXA3vWBf+QC0lmmgM+0MowT/m/QkYPRWbmz3c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854955; c=relaxed/simple; bh=eTHtJrLTgKPapbvhe7qwd0kLWB4F9Df0+/dv2qnCE3o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QQkaLdkzM7Dbej0REY+kOnruDQdQDAn/vd4lnmia8HTerRY8jZezX+aAh1nqhhkDxu4X1lKa++HF7FYkWIJoiL9NZs96lqd5rc4nxcEUnI6hb/ojfGLz6ATElUVZoyIgQunXqSj/SHfJgS3gsFy3BrXSJyuOpBqlu4tvMFBczMw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aMwiUcYA; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aMwiUcYA" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-22104c4de96so41232565ad.3; Mon, 17 Feb 2025 21:02:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854953; x=1740459753; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MJ9F212PtsSHKIVUZOrPA2mk84naWjg27j0jUX3VDVs=; b=aMwiUcYAbUcy5ARrj4oXyeg4tLU979DmpiWnpLt/wiFUbDjn3EAGYvvxacLx81tGEs TTv7LU6xkuTx8NWFwcOoFs88LmLdS813WvNLMyngSPeqK9vrC04kAYz12G1WoMYYS8BH 302+4Tu/F3lU10oPxVhMll7RKsqiDWaPkGeKx27HciKrW6BUZPlldz6DRxruZq7QdkNT VWB8gaM/uBiDbKoz3K0/qJUkogYe8hOREF2wslD7HoshCavOqO6RkbmAfP5EngkAWJ5L SMykXDj+VLFcJCcoxFsNuF2nCLojlvffRk1ZeR8TVNlhG1pfN1DUXoXIuUev3NmpZ6hf PEfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854953; x=1740459753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MJ9F212PtsSHKIVUZOrPA2mk84naWjg27j0jUX3VDVs=; b=B2Uqv/mT9/B8W+6pris1pBZud/MQAqXzFtbVE1oenQpXMr9ysr5TH6UNoOxy8HGpQg p7cDz5iOQxrva2CAGZnG4j/hnXbsW4nbEc5waIDv15Sio4p8C3cSckxIYVmOFqDroHsz G54J+7VzLz6xNVhuTF1TSsPz67X4J7E5ZJ7lpN5t8Sqi8hl+K8OJYfISF4zjOkbna+05 ir6Cc2l14v9ti4x+dm3Q7ic7pZRIsoqt2ONdEEyrBZORu16x35/1YTYLtPzGVp+1tQsB TuZ+5Qp5ZfO79uG0wzEVizr6eWezxf4a+okwki1z0iJm0zHiszBLt2RaxdkdhflrHTNp 3Rtw== X-Forwarded-Encrypted: i=1; AJvYcCVsQunptakvm5HsCcUvnu6+LnhAMxNQKEgYMVRNcr1oQbSMxEx03BOrJ2dQJkoXtpGIAIau0XM=@vger.kernel.org X-Gm-Message-State: AOJu0YzV5NPeXbnt72kg5+D0X5DX9l+zyvB8luCha18IqLqv0HnDZrG9 M1g0wH78uIngRsgTT2VvTApQHD3v6pbwNr297Li1nXjcuBBoFA4v X-Gm-Gg: ASbGncvXuLNZeD3rpSRa4oleAKN5coTpr1roR4UPRiaLMnh7y+M+m4WIk0GVxZcku3L 7JAIhjiKyeQylzrHdyhgcFce72pzcWYi49giRZLc23KffQU0aC+gGVr3bxSinT0zzK950lBAJym EqjROYRs8Il9zrSY7KoYmuSTXVg0WtrhtOOiORUEEApGpwu7oz7Y8MRn0zvtz9zN8WYqqHvBqqe ACPIbQgbkG6kQ/zZ0MnaipRyCIxWMBpcBd1pm4BJicWKe6wlVkWk3PNO0wVopsdrOSbj2Ha5w1S d6V47dkXVgDNQDT8X9bF7vwXy4QydEot3EfYBeCfR046oB0z8/BVJ22k9umMclo= X-Google-Smtp-Source: AGHT+IF5jNJ1zaRLc4VHBX0frpikS9WNXmW8Uck/woIYq2085fB3pqq5bGx8lkSyYr9T4ObYzUp9Eg== X-Received: by 2002:a05:6a20:72a6:b0:1ee:615c:6c8e with SMTP id adf61e73a8af0-1ee8cacbf27mr18660115637.9.1739854953526; Mon, 17 Feb 2025 21:02:33 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:33 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 05/12] net-timestamp: prepare for isolating two modes of SO_TIMESTAMPING Date: Tue, 18 Feb 2025 13:01:18 +0800 Message-Id: <20250218050125.73676-6-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net No functional changes here. Only add test to see if the orig_skb matches the usage of application SO_TIMESTAMPING. In this series, bpf timestamping and previous socket timestamping are implemented in the same function __skb_tstamp_tx(). To test the socket enables socket timestamping feature, this function skb_tstamp_tx_report_so_timestamping() is added. In the next patch, another check for bpf timestamping feature will be introduced just like the above report function, namely, skb_tstamp_tx_report_bpf_timestamping(). Then users will be able to know the socket enables either or both of features. Signed-off-by: Jason Xing Reviewed-by: Willem de Bruijn --- net/core/skbuff.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/net/core/skbuff.c b/net/core/skbuff.c index a441613a1e6c..341a3290e898 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5539,6 +5539,23 @@ void skb_complete_tx_timestamp(struct sk_buff *skb, } EXPORT_SYMBOL_GPL(skb_complete_tx_timestamp); +static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb, + struct skb_shared_hwtstamps *hwtstamps, + int tstype) +{ + switch (tstype) { + case SCM_TSTAMP_SCHED: + return skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP; + case SCM_TSTAMP_SND: + return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP : + SKBTX_SW_TSTAMP); + case SCM_TSTAMP_ACK: + return TCP_SKB_CB(skb)->txstamp_ack; + } + + return false; +} + void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb, struct skb_shared_hwtstamps *hwtstamps, @@ -5551,6 +5568,9 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb, if (!sk) return; + if (!skb_tstamp_tx_report_so_timestamping(orig_skb, hwtstamps, tstype)) + return; + tsflags = READ_ONCE(sk->sk_tsflags); if (!hwtstamps && !(tsflags & SOF_TIMESTAMPING_OPT_TX_SWHW) && skb_shinfo(orig_skb)->tx_flags & SKBTX_IN_PROGRESS) From patchwork Tue Feb 18 05:01:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979008 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5617719D89D; Tue, 18 Feb 2025 05:02:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854962; cv=none; b=Fpjtj0LWZeVlrXZSP+uopo7H+pftgIyzEbAVQP6LRqCVTfYuue6VPLTlVcZ8yTCCXY6zYbJYUiTOTmKyfzUnM1SfUeCmrCJoidi9XYomz/v8H+kjgGMvVnzRTX3mSV8KySOP//tN4AED6MhFAxq7OfLvc7JQUqzHqIKFY4whn70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854962; c=relaxed/simple; bh=MwnhAchlDNKve5J+/cAJpVlmx6+Xk3aqDL+WS1Pv+Mk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=isMP7BuNVotO50rD/+BhK8NknA+4h9EZkNwvdrUSP7LfnElyemfAfqF7CJ/oaTYuS7aNDOIykqQb6aG18bwDp4Qtw+ZghOunQ1WfwstO+qDTLRvoCyBc1mhBP3j/5GdFSTuLN/8Pmy8f04xezyJtjJB7bCT2DEwM8oXq8dbF+DI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IyLZNlYn; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IyLZNlYn" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-220c2a87378so69262035ad.1; Mon, 17 Feb 2025 21:02:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854960; x=1740459760; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FeqW566o5arTPjVwkcb4h3wmjpR1zsB/qZsvzlvgmI4=; b=IyLZNlYnSst5/pwyQhTbMIZxSfodlt4xAnTh51++e2fFA3fLecMYv+uCe5528YtwAJ z5S0jLvs5217Ltn1bowcx6HSTWWr5NIMsAlAICFFqowJ7swLG2pj7R11b2HgWVa4lklm 7Kr17CJrcDB899iLm9ERyU12ytZLZbIjriD0JUYxXiIttEtu5Bb/iFFLJmJwl4941nLA aS/WoYItffnQXReppvhJ3rmjazBea86owrViIx4ptk2hYr17RO8RSJ9U7ETCCG+xlXIT KlT3S71rZFrrnbdmAsf8YY4hYK9M5Q94krKN+C//itiGopi1P8FSNP05ix4CNqyqGNq0 WVLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854960; x=1740459760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FeqW566o5arTPjVwkcb4h3wmjpR1zsB/qZsvzlvgmI4=; b=UCWMDq6J/+f0WNNd05sUnVGHVDh3nyGaLaci/KwmmEPh0YlfyyS4HT+vX0Cp4Vrg8G 93HMKVMXeFtrtPt/cQygcQ1H/no8BTgh2SWa+449YyC6LUgpMO6UIH9Q7CQKb0v63Nc8 A3FW2QWeo1TpBPjav2plQKEYMP4ePFLeK3nNFcK2WKzk+Y9tCu3+/ugbjYg4QNl+EgWa B9nvm5WHVtfm1jCfjysEoIHgOmL4nQMxydkejrTWvNsTrKDzEGHODnFQtJOU4JDD0sT9 ssNclnZUGpUiWQFBUviGI5O7ML57EHuZ3Tgt7tE2U/IgCOZ0irGinG91wlN24OUg6tAy hccw== X-Forwarded-Encrypted: i=1; AJvYcCVoUZ5YvGj8ZUHTj74CHbxnQGFLRfs91s9BCs+z5awnuVnR7UwuPpvmxDHBZpIEbfmiBQTQjM0=@vger.kernel.org X-Gm-Message-State: AOJu0YwgCv/5rS9pWZOr+ZsrCiAwxaaiYHHh1pGd4caDAow3wRlHjBC8 fKTUpPAeG8ZBS2kvICi/HKTOE84TPBKSkL7LI0aLBGd1e8TxI31Z X-Gm-Gg: ASbGnctzG86FN18Rao7/AY+Ud9Jl5xxIO1Y5UMSxCtqL7tWmeRAlaehdP/UmWw0EAh+ mS3p5fmuMuVH35dLyK9nbyYSeN8ynP1+W4SnMPjgRFJlNEPJb0aGigQ+Y3J2Ufu9Wf+spe5Np1d kbhl+ZjelP6PnksYUmm3jJ06tYV2gHTUnZRccODGnLO2o589WQ0LyfDA0CasPNRFI8TACXOTbsH GuYgsxFtktFIahYGbzjkECR1wUnhnpuzhHp+d3aAvY4Emck2v2Bolt2q+bQICAnGdtwFPinhSRT 6s+p/LiKp5iSJQN8bSrz5bTQ+TDRoj8zwh9LZHU8g4ixQsf3JhLThguDOJM2qi4= X-Google-Smtp-Source: AGHT+IHD2o7+vVGjFxLyBdympf/mXmQa90Uu3NaJ8fllrALp6+mnFBukciz3QXbsbo3eCzUfEZ+yhQ== X-Received: by 2002:a05:6a00:2351:b0:730:f1b7:9bd2 with SMTP id d2e1a72fcca58-73261799a56mr19945923b3a.6.1739854960568; Mon, 17 Feb 2025 21:02:40 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:39 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 06/12] bpf: add BPF_SOCK_OPS_TS_SCHED_OPT_CB callback Date: Tue, 18 Feb 2025 13:01:19 +0800 Message-Id: <20250218050125.73676-7-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Support SCM_TSTAMP_SCHED case for bpf timestamping. Add a new sock_ops callback, BPF_SOCK_OPS_TS_SCHED_OPT_CB. This callback will occur at the same timestamping point as the user space's SCM_TSTAMP_SCHED. The BPF program can use it to get the same SCM_TSTAMP_SCHED timestamp without modifying the user-space application. A new SKBTX_BPF flag is added to mark skb_shinfo(skb)->tx_flags, ensuring that the new BPF timestamping and the current user space's SO_TIMESTAMPING do not interfere with each other. Signed-off-by: Jason Xing Reviewed-by: Willem de Bruijn --- include/linux/skbuff.h | 6 +++++- include/uapi/linux/bpf.h | 4 ++++ net/core/dev.c | 3 ++- net/core/skbuff.c | 20 ++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 ++++ 5 files changed, 35 insertions(+), 2 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index bb2b751d274a..52f6e033e704 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -489,10 +489,14 @@ enum { /* generate software time stamp when entering packet scheduling */ SKBTX_SCHED_TSTAMP = 1 << 6, + + /* used for bpf extension when a bpf program is loaded */ + SKBTX_BPF = 1 << 7, }; #define SKBTX_ANY_SW_TSTAMP (SKBTX_SW_TSTAMP | \ - SKBTX_SCHED_TSTAMP) + SKBTX_SCHED_TSTAMP | \ + SKBTX_BPF) #define SKBTX_ANY_TSTAMP (SKBTX_HW_TSTAMP | \ SKBTX_HW_TSTAMP_USE_CYCLES | \ SKBTX_ANY_SW_TSTAMP) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index fa666d51dffe..68664ececdc0 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7035,6 +7035,10 @@ enum { * by the kernel or the * earlier bpf-progs. */ + BPF_SOCK_OPS_TS_SCHED_OPT_CB, /* Called when skb is passing through + * dev layer when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/dev.c b/net/core/dev.c index c0021cbd28fc..cbbde68c17cb 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4500,7 +4500,8 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) skb_reset_mac_header(skb); skb_assert_len(skb); - if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP)) + if (unlikely(skb_shinfo(skb)->tx_flags & + (SKBTX_SCHED_TSTAMP | SKBTX_BPF))) __skb_tstamp_tx(skb, NULL, NULL, skb->sk, SCM_TSTAMP_SCHED); /* Disable soft irqs for various locks below. Also diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 341a3290e898..0aa54d102624 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5556,6 +5556,23 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb, return false; } +static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb, + struct sock *sk, + int tstype) +{ + int op; + + switch (tstype) { + case SCM_TSTAMP_SCHED: + op = BPF_SOCK_OPS_TS_SCHED_OPT_CB; + break; + default: + return; + } + + bpf_skops_tx_timestamping(sk, skb, op); +} + void __skb_tstamp_tx(struct sk_buff *orig_skb, const struct sk_buff *ack_skb, struct skb_shared_hwtstamps *hwtstamps, @@ -5568,6 +5585,9 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb, if (!sk) return; + if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF) + skb_tstamp_tx_report_bpf_timestamping(orig_skb, sk, tstype); + if (!skb_tstamp_tx_report_so_timestamping(orig_skb, hwtstamps, tstype)) return; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 70366f74ef4e..eed91b7296b7 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7025,6 +7025,10 @@ enum { * by the kernel or the * earlier bpf-progs. */ + BPF_SOCK_OPS_TS_SCHED_OPT_CB, /* Called when skb is passing through + * dev layer when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Feb 18 05:01:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979009 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 547991B0409; Tue, 18 Feb 2025 05:02:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854968; cv=none; b=ArBysinNXqfT/vD9G8O0JVd5RxdPlm10AQYvSYHcFKm4a0YEumH/U72SBenbUYWNJF2DgV1lv3yJLAES92CfcN5Sqiol8u8I8WE7d8tOFRNiX269GN8SgyN9vU6Bi9fi/lTK1LcLukZ05+NFtqBKGXCp3zqvCJaf8QniZCMj2Is= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854968; c=relaxed/simple; bh=IXLUCYhCNHZgztCruzDU0ZGKFctaGuPZNEaKlMrXO4I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DeDxwu68OrUasmg/Ig8dnOfekHM9noBcFJzLxQpCvwJeZEMqSOCDMEuB9bT03cC4ZD0TdUl4h+ac7GbusdjciepKDLzdnhR6/WCL+KU4DXBr3+FmK6a0MPwcSfkHoqwChbBlaSqYqy8FPHm3WaR0zfJWakYDtnkCB9U7SNwG4lk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bDDjggxm; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bDDjggxm" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-220dc3831e3so72062845ad.0; Mon, 17 Feb 2025 21:02:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854967; x=1740459767; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mVlgz+q5Bkt8Btds59rDLuHiYriySzR73dW98PkMWw0=; b=bDDjggxm6gxwsJsnmAr/QhMrD6vkwwAhKqENNpECF9gbrpU+0xQkCrUDAhtOqQRqDk 4DS9FcAszT/fdaDxGsandC4HjacI5S91J4JQz+cVK8GiPvFENEfb8n5QLm6QGVpogcMd EZVpubrAWZqNthjnUjbM1yO86nnfeL/QPllZRRW+xdaDOOLo80TBqqAmYwalaQFtIf5m t8m1dE5pfV75PyJ6rClntVkzCqXF3DOe3FlJpFHoW2Scsv4vZwgc8Q6De27NYnQK+Ds8 dDO+UUXhmlYawGSK6Y+0mibFP4vJkfSTsQPsc4bjz9hgXAY3/ABv6kTiAvQSl6p88yzP MFoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854967; x=1740459767; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mVlgz+q5Bkt8Btds59rDLuHiYriySzR73dW98PkMWw0=; b=UXzDfo8tWLsHXu/JB3A05a0K0Fs4ha/B7Q/q7wwoytjKYKmeKIU2TVhx0UKK1azbdP RrmplPf70Bm0GDfgAx6SVb4VfA46KujqP87vD3pqSYYIzWb0+ui8HjNUdeWc3MBqgG33 HxsMnqtQXo7YjTlF6fbB46PSY8dNvPTmu/SLCoZq4b53ERe2WylUfi4OGpn5vVMX2LIi f2ctE71iMl7l51eSQWs22c091YQMwiF6vpjMSnttAp0cbOVTP6HiKkqxcecrQofXaG4D HSuzJ5SSDVfANr0XnRs1x81XbQ9N4zKQ7D2+DN+eFOMYIsv5/C2aDdNUbCkIht0JLzwn MDGw== X-Forwarded-Encrypted: i=1; AJvYcCWR/nsjvLG3sfBZPabYcc6M1HEsCOCU8WoYlSGfOuGEVc75SW/w/3sYEz1XT92HmI/QzPxPMk0=@vger.kernel.org X-Gm-Message-State: AOJu0Yxvk2zfGkKQZkLhm4YGHv2HEYJmWy/F1IZAbjdpej/eSH/gCzUp C5izE3cfUDiKtM0mS3Wn2cSNSJFCTUkFyenaf2AlDDPewseivFfz X-Gm-Gg: ASbGncs3MhGAcv0LNBYdiun6BZBMCpdtpyZMtRsJx4Ox8R2mV0+CXa7gwUjBxp9fYLt gzM3OBbtOBYCGYkVerWq1H943i1n9rcnbFqLZzcNg6G3aGM3wEANI+Gjs4Vh4cOVrdcG6vvl1c3 uHGSDK00nkmxa4p0GEXdM1sFQHqzVf0bdREyOBUWC1xFB5fcNv7dSw/BHHYr49hmwh2jLWbd0DI XH33r3mWdUYQpZTgGGOiWOY5SNkfL/llwF44F0GyCIV/PGilwUhdiT0frejeXzdMJ/11ZRWRpXT RlAeCAyGEGcveUd1zTsoM0GqbwRnfmPSxJxyBIRYNrFXw8yVXGLw7LeeCzECrQU= X-Google-Smtp-Source: AGHT+IE4gN00x+MBMbm+YHE83mUYVOOcg+U3qdaIY+3bCk3PcDqtRpXCAoNzPuL7ZZNeYnhkFnYjpQ== X-Received: by 2002:a05:6a21:6da7:b0:1ee:47e7:7e00 with SMTP id adf61e73a8af0-1ee6c6ad940mr32536063637.13.1739854966683; Mon, 17 Feb 2025 21:02:46 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:46 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 07/12] bpf: add BPF_SOCK_OPS_TS_SW_OPT_CB callback Date: Tue, 18 Feb 2025 13:01:20 +0800 Message-Id: <20250218050125.73676-8-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Support sw SCM_TSTAMP_SND case for bpf timestamping. Add a new sock_ops callback, BPF_SOCK_OPS_TS_SW_OPT_CB. This callback will occur at the same timestamping point as the user space's software SCM_TSTAMP_SND. The BPF program can use it to get the same SCM_TSTAMP_SND timestamp without modifying the user-space application. Based on this patch, BPF program will get the software timestamp when the driver is ready to send the skb. In the sebsequent patch, the hardware timestamp will be supported. Signed-off-by: Jason Xing Reviewed-by: Willem de Bruijn --- include/linux/skbuff.h | 2 +- include/uapi/linux/bpf.h | 4 ++++ net/core/skbuff.c | 9 ++++++++- tools/include/uapi/linux/bpf.h | 4 ++++ 4 files changed, 17 insertions(+), 2 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 52f6e033e704..76582500c5ea 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -4568,7 +4568,7 @@ void skb_tstamp_tx(struct sk_buff *orig_skb, static inline void skb_tx_timestamp(struct sk_buff *skb) { skb_clone_tx_timestamp(skb); - if (skb_shinfo(skb)->tx_flags & SKBTX_SW_TSTAMP) + if (skb_shinfo(skb)->tx_flags & (SKBTX_SW_TSTAMP | SKBTX_BPF)) skb_tstamp_tx(skb, NULL); } diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 68664ececdc0..b3bd92281084 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7039,6 +7039,10 @@ enum { * dev layer when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SW_OPT_CB, /* Called when skb is about to send + * to the nic when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 0aa54d102624..03b90f04d0b0 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5557,6 +5557,7 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb, } static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb, + struct skb_shared_hwtstamps *hwtstamps, struct sock *sk, int tstype) { @@ -5566,6 +5567,11 @@ static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb, case SCM_TSTAMP_SCHED: op = BPF_SOCK_OPS_TS_SCHED_OPT_CB; break; + case SCM_TSTAMP_SND: + if (hwtstamps) + return; + op = BPF_SOCK_OPS_TS_SW_OPT_CB; + break; default: return; } @@ -5586,7 +5592,8 @@ void __skb_tstamp_tx(struct sk_buff *orig_skb, return; if (skb_shinfo(orig_skb)->tx_flags & SKBTX_BPF) - skb_tstamp_tx_report_bpf_timestamping(orig_skb, sk, tstype); + skb_tstamp_tx_report_bpf_timestamping(orig_skb, hwtstamps, + sk, tstype); if (!skb_tstamp_tx_report_so_timestamping(orig_skb, hwtstamps, tstype)) return; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index eed91b7296b7..9bd1c7c77b17 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7029,6 +7029,10 @@ enum { * dev layer when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SW_OPT_CB, /* Called when skb is about to send + * to the nic when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Feb 18 05:01:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979010 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5986A1ACECF; Tue, 18 Feb 2025 05:02:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854974; cv=none; b=LfXmZvDwJ+c9qrk0s9+qFg6gMGjjPa3eMoYj4JCWZ61Q6M3+Z2O5cHU3MD5FEpGvcXUA+5B+ohxi5vbYyTcyH98sm+U7pbN1Aydma7WYST2r0D2XdnGAZY/YpmoW2XdwVBGI0PKMDUK0xzQLdCZi6JXar/fGFyxI/3eknyB0cC4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854974; c=relaxed/simple; bh=KmC/1wQSvEc5h2YZU90pwy7s8cYyQeNpKxYvTEl8WO8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gfuz7I0dMZQh/+QSDYqnYbwu1V/M1y/sNHug99v7LeH0ibf67mom0FVbZdlQpP0zO0vE8oN5piUlUuUJ89HPnQLraDdoOrPAZqQFfdTqbA18lNZ7ALhMriQKDM628/yJlsktBdpBfhV5axJLN15x4aUxc+sQlefCyXGFRXdgQV8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KjsLPMdb; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KjsLPMdb" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-22104c4de96so41236415ad.3; Mon, 17 Feb 2025 21:02:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854973; x=1740459773; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/l2sahMfyVzXLK9+XiQ8AOEzAxEpWK00yzRyIH205+Y=; b=KjsLPMdbM+jPPbQLqTTN61QFQGxPVth+Q9gT1KHbY5lq6rihk6hd4ngCQTgWEutEHC AOEfViYZDzZHXoPFhDDzraQYBgHLbIKE5PnNbc/u3JBX8bWV8HzaZXeqgFkh/mioIHH1 OuwKc90gPnEeJ3prFHChvkNB0f3RO2A2gbBPznc9wjI1hZ+/S1I+xHuiHq8GBqGcO3QQ MwO+DCWsrmjieq9L6XKVG9i02ULMiMuJfzY/FLCXCWU8m2++nqFx9haU4JohFZWzlPmz NrTqqkwuyiRbuKF0U3H6bNBxiIBR77qHLI07SVMzIKHXEa5Prhs7bFk0xd+o4LLpwk7b MfIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854973; x=1740459773; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/l2sahMfyVzXLK9+XiQ8AOEzAxEpWK00yzRyIH205+Y=; b=SOTWIyCBX1U27BJucUdWWXkEAXwk5cOZad2bc/DlEzYxhcYTElm0e5f9FU8MWiCZt1 F2Eo7UHyGZqzLCGH4E4iI6LnDFIyFUpbHJEIV3IZ53TbGttJz7/RmBjj4u6ySTH4cmzf 8ASa3HLDkc7Ja9JmM/vk9cbQdJipI1TgFx7bAX1xajChpHWRORHOndp54ubk24SPTuOT Q4a2EGnMCYPNlw4Bm2ZSZRuXTcd+B9ezPNFgrncASbpifXG6aOWcJS2bzmuEhE13vi5b pTvwxJOIMFglgYEaz8/Z1lBfkjRP1Vtvq9l51OqATQExum8wFEuYGRYTr7jZsavo0hY/ N/wQ== X-Forwarded-Encrypted: i=1; AJvYcCXZ03a2ii3NdWqq4R0FPuQn3czCU8bCCRet+Fa9yYkTg3YcCACClhaXP5VldJPCusUxn9pZEHI=@vger.kernel.org X-Gm-Message-State: AOJu0YwzzKoo9v9ydW3CSbHqTVHZSnWXfhyWMUFD9bwqJ3lwzstEQD/U YMGgoFjLU1fLM8ZcKH66R1UMbR0sRJ0NEeu4eWCs+RZwmcm922hLAyWs0lZ7Zo4k1g== X-Gm-Gg: ASbGnctmYN6gHKtC6T0F8V3a40Ot/faFbeTig0eUnF7UJMnUY0RyJtVAI/QuPRolVka 75IOFjX+NLvqoBXzaCkMKZMUCbP9BEO0zGhENblwkARpY2xwMxEA45L9p7KeaQdj+iuELJvddqc r7gYwLyG/4Jlk/Ll85nebCK4XO5wbrhyG1JjMx7QR5uANq7qnZFHjzjurgPqi944m83w48RJUxd k8ntrxh6zMpjzJpgGe2dufPw0vb7oHyes5vk+tzqTkGI6fectZZL/INbqTynAWrqA/8n80IzuuA d0uFDd8ji7M8r44NULAAQLsZ6asZdaqbC/4rMbvVPFAT7tstcAiXj1FVQGw65Ds= X-Google-Smtp-Source: AGHT+IFcyWnuPtK4F2+F/fF2LMRqZphdqRNBz7UzSwzTOpFSmMS39Aa3GMCu2w4ECNaz3NMozG0M9g== X-Received: by 2002:a17:903:3d0c:b0:21f:7964:e989 with SMTP id d9443c01a7336-221040d84ccmr190990025ad.52.1739854972729; Mon, 17 Feb 2025 21:02:52 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:52 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 08/12] bpf: add BPF_SOCK_OPS_TS_HW_OPT_CB callback Date: Tue, 18 Feb 2025 13:01:21 +0800 Message-Id: <20250218050125.73676-9-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Support hw SCM_TSTAMP_SND case for bpf timestamping. Add a new sock_ops callback, BPF_SOCK_OPS_TS_HW_OPT_CB. This callback will occur at the same timestamping point as the user space's hardware SCM_TSTAMP_SND. The BPF program can use it to get the same SCM_TSTAMP_SND timestamp without modifying the user-space application. To avoid increasing the code complexity, replace SKBTX_HW_TSTAMP with SKBTX_HW_TSTAMP_NOBPF instead of changing numerous callers from driver side using SKBTX_HW_TSTAMP. The new definition of SKBTX_HW_TSTAMP means the combination tests of socket timestamping and bpf timestamping. After this patch, drivers can work under the bpf timestamping. Considering some drivers don't assign the skb with hardware timestamp, this patch does the assignment and then BPF program can acquire the hwstamp from skb directly. Signed-off-by: Jason Xing Reviewed-by: Willem de Bruijn --- include/linux/skbuff.h | 4 +++- include/uapi/linux/bpf.h | 4 ++++ net/core/skbuff.c | 11 +++++++---- net/dsa/user.c | 2 +- net/socket.c | 2 +- tools/include/uapi/linux/bpf.h | 4 ++++ 6 files changed, 20 insertions(+), 7 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 76582500c5ea..0b4f1889500d 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -470,7 +470,7 @@ struct skb_shared_hwtstamps { /* Definitions for tx_flags in struct skb_shared_info */ enum { /* generate hardware time stamp */ - SKBTX_HW_TSTAMP = 1 << 0, + SKBTX_HW_TSTAMP_NOBPF = 1 << 0, /* generate software time stamp when queueing packet to NIC */ SKBTX_SW_TSTAMP = 1 << 1, @@ -494,6 +494,8 @@ enum { SKBTX_BPF = 1 << 7, }; +#define SKBTX_HW_TSTAMP (SKBTX_HW_TSTAMP_NOBPF | SKBTX_BPF) + #define SKBTX_ANY_SW_TSTAMP (SKBTX_SW_TSTAMP | \ SKBTX_SCHED_TSTAMP | \ SKBTX_BPF) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index b3bd92281084..f70edd067edf 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7043,6 +7043,10 @@ enum { * to the nic when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_HW_OPT_CB, /* Called in hardware phase when + * SK_BPF_CB_TX_TIMESTAMPING feature + * is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 03b90f04d0b0..acafa05f7f58 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5547,7 +5547,7 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb, case SCM_TSTAMP_SCHED: return skb_shinfo(skb)->tx_flags & SKBTX_SCHED_TSTAMP; case SCM_TSTAMP_SND: - return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP : + return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP_NOBPF : SKBTX_SW_TSTAMP); case SCM_TSTAMP_ACK: return TCP_SKB_CB(skb)->txstamp_ack; @@ -5568,9 +5568,12 @@ static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb, op = BPF_SOCK_OPS_TS_SCHED_OPT_CB; break; case SCM_TSTAMP_SND: - if (hwtstamps) - return; - op = BPF_SOCK_OPS_TS_SW_OPT_CB; + if (hwtstamps) { + op = BPF_SOCK_OPS_TS_HW_OPT_CB; + *skb_hwtstamps(skb) = *hwtstamps; + } else { + op = BPF_SOCK_OPS_TS_SW_OPT_CB; + } break; default: return; diff --git a/net/dsa/user.c b/net/dsa/user.c index 291ab1b4acc4..794fe553dd77 100644 --- a/net/dsa/user.c +++ b/net/dsa/user.c @@ -897,7 +897,7 @@ static void dsa_skb_tx_timestamp(struct dsa_user_priv *p, { struct dsa_switch *ds = p->dp->ds; - if (!(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)) + if (!(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP_NOBPF)) return; if (!ds->ops->port_txtstamp) diff --git a/net/socket.c b/net/socket.c index 262a28b59c7f..517de433d4bb 100644 --- a/net/socket.c +++ b/net/socket.c @@ -676,7 +676,7 @@ void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags) u8 flags = *tx_flags; if (tsflags & SOF_TIMESTAMPING_TX_HARDWARE) { - flags |= SKBTX_HW_TSTAMP; + flags |= SKBTX_HW_TSTAMP_NOBPF; /* PTP hardware clocks can provide a free running cycle counter * as a time base for virtual clocks. Tell driver to use the diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 9bd1c7c77b17..7b9652ce7e3c 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7033,6 +7033,10 @@ enum { * to the nic when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_HW_OPT_CB, /* Called in hardware phase when + * SK_BPF_CB_TX_TIMESTAMPING feature + * is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Feb 18 05:01:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979011 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 737271AF0C9; Tue, 18 Feb 2025 05:02:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854980; cv=none; b=twIyzSP2HlpTn0nANpG74KRIOoWT79ENIwcRtt0bVRRjDp7MFgY6D6tF/ob/vf/3It1Zj8s9j6kcfvwSnB5qBRTsSg9FgWhtcadXO9Njiqy4XwMavi/KV+FjVAzNLBfIOULvHjkwyg0g+li5M7padFBhidkvJvasQzMOsDHzcf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854980; c=relaxed/simple; bh=f6UPX8nqSzhpR5jLn4sqYvOct90ttv2Ux+uobqmyXyI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tdVYt4kR3qnuuPNB28DYrz+XTcdg/z5ZQfo0zL5GuQiyjuEEsFTIusT1Av66bYEPklpKMWPl1rdaMM1fDk1RPoxdu7+iCdbYaIf7RAnE2Z6NZMG953Wwdw0ILvIDp5YnrOpzbL4GLRINtbtzK40BfXorxSnk+myaW5+DE/nxDwQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=D5N1pixO; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="D5N1pixO" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-220c665ef4cso73645025ad.3; Mon, 17 Feb 2025 21:02:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854979; x=1740459779; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OwKWPUXCleIrg1+YnGzh7tU7O5+lCSDVQCdHHSHmWTs=; b=D5N1pixO+DGwOQAYJJScXyMSyBIvHDdhuKPuZ88FtNDpDeUQzqXYO3b7b1BPgvlJAh youk4IgT+FNSFFJ/OFnt8bK7pjKpGTiA8bY74hnIXJu5uKHjWgRzReZgIyRf6+kmpfUI y1m0FTV/v6hjwfcWEGRYxgVpLIFvZC76oE0A9/Rxc9+F98Q8A9AqQUkR59hovkBA/eAl QT2s3jqoSkBI8MgrX0gI1YdrIGO8IbOkBTxdTZXDW4dkIc8T7zNhrR4hg8JI/11vrwrG kqvFme+MY/fCvmlnqZZRFKUarZZdFZ8+jAau5Yrq6GIZrW406h74bLrQuy8G4+T2oTPo 6HwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854979; x=1740459779; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OwKWPUXCleIrg1+YnGzh7tU7O5+lCSDVQCdHHSHmWTs=; b=dJcdGgSGZh2NPDE6k7aIsOklVfLommtAN40qeIw+O4BbH5YiYiEIU9p4a78/1f/uyG M9m3B/hXM0WRUShqMVCGpFLyZQEAg1DGf75hMXsTPbJyguEOl1/+7GUw5pzuLFmEBhH+ 4zFr124sRBtN3fcVG1NiKhYDtQ9/zhbRRya6/v5AuMyoSSNHCo2CC/zkI0YloLNo7TS5 xLPS0W1bJal898VI5YCDn5Be/ek8/Dr86kdgI3wA7vYHbJs9Y1pGsZYBvnDPXL1bpmF3 AxaJMOidoQPukcVolkCFQDnF3cTWGIw1TZwqWe7Q0v3XYVI3J7WSp3ZX2UjigLpBZb9Q z9zg== X-Forwarded-Encrypted: i=1; AJvYcCUXCK/pwAQu3s1IraoW2YSg0iD2lRv8NouZgBdtLaNVY8x4NIcZ2TYO01Ie9rbtznaXrYjA/dk=@vger.kernel.org X-Gm-Message-State: AOJu0Yz1u8tr9OHHHL3TQOzgDAgH+qBkV6XPhnj9O3BsXzek9hiFbx18 Ud41wMnhIpqYgKkB6Fiyq3k7XcLhLA5nZSc+KsyhJ4XTUXB1RiMU X-Gm-Gg: ASbGncsjeF16GmhuGroP/CBcaEVF1JodZvAUUKTqncdBXubk/HEX3k4VaHFo1Zb4SF7 3PX7WIpQMihRcZDhuBepbA5aCslrFAi2iUrys3HxEVcmvEAUGJWfePgD3HU3ysbwcpOLBNZlgnY vTrKoXSLGnD4l71Z236P/P26XTwNyXr4YYqsxyvgWub1NQuPYXw/LlAqhe/LyOVilw0a0RDu7ut LZwIzjkaLBQ8v1Z5LlPqWT8CMId6owHotvAEe75DSAZdgS9Wy3Ks83x0r6o+3CPeF8POO+O3lEb th8ziXnsC460200yUX0buccbOBmi765Fl+rupmGO4aBD9WTkVDLgRyBXf2WRsKc= X-Google-Smtp-Source: AGHT+IHFIRwQHnq2ePwaF80rtxwcZ0D0O/3Vy0GD5n5EBqBnIxttX+8R2iarVx5rNFOgs/y8BKccBA== X-Received: by 2002:a05:6a21:7a4c:b0:1ee:c8e7:203c with SMTP id adf61e73a8af0-1eec8e7219amr1911588637.24.1739854978661; Mon, 17 Feb 2025 21:02:58 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:02:58 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 09/12] bpf: add BPF_SOCK_OPS_TS_ACK_OPT_CB callback Date: Tue, 18 Feb 2025 13:01:22 +0800 Message-Id: <20250218050125.73676-10-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Support the ACK case for bpf timestamping. Add a new sock_ops callback, BPF_SOCK_OPS_TS_ACK_OPT_CB. This callback will occur at the same timestamping point as the user space's SCM_TSTAMP_ACK. The BPF program can use it to get the same SCM_TSTAMP_ACK timestamp without modifying the user-space application. This patch extends txstamp_ack to two bits: 1 stands for SO_TIMESTAMPING mode, 2 bpf extension. Signed-off-by: Jason Xing Reviewed-by: Willem de Bruijn --- include/net/tcp.h | 6 ++++-- include/uapi/linux/bpf.h | 5 +++++ net/core/skbuff.c | 5 ++++- net/ipv4/tcp.c | 2 +- tools/include/uapi/linux/bpf.h | 5 +++++ 5 files changed, 19 insertions(+), 4 deletions(-) diff --git a/include/net/tcp.h b/include/net/tcp.h index 4c4dca59352b..2e2fc72e115b 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -958,10 +958,12 @@ struct tcp_skb_cb { __u8 sacked; /* State flags for SACK. */ __u8 ip_dsfield; /* IPv4 tos or IPv6 dsfield */ - __u8 txstamp_ack:1, /* Record TX timestamp for ack? */ +#define TSTAMP_ACK_SK 0x1 +#define TSTAMP_ACK_BPF 0x2 + __u8 txstamp_ack:2, /* Record TX timestamp for ack? */ eor:1, /* Is skb MSG_EOR marked? */ has_rxtstamp:1, /* SKB has a RX timestamp */ - unused:5; + unused:4; __u32 ack_seq; /* Sequence number ACK'd */ union { struct { diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index f70edd067edf..9355d617767f 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7047,6 +7047,11 @@ enum { * SK_BPF_CB_TX_TIMESTAMPING feature * is on. */ + BPF_SOCK_OPS_TS_ACK_OPT_CB, /* Called when all the skbs in the + * same sendmsg call are acked + * when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/core/skbuff.c b/net/core/skbuff.c index acafa05f7f58..f096ca6c2ced 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -5550,7 +5550,7 @@ static bool skb_tstamp_tx_report_so_timestamping(struct sk_buff *skb, return skb_shinfo(skb)->tx_flags & (hwtstamps ? SKBTX_HW_TSTAMP_NOBPF : SKBTX_SW_TSTAMP); case SCM_TSTAMP_ACK: - return TCP_SKB_CB(skb)->txstamp_ack; + return TCP_SKB_CB(skb)->txstamp_ack & TSTAMP_ACK_SK; } return false; @@ -5575,6 +5575,9 @@ static void skb_tstamp_tx_report_bpf_timestamping(struct sk_buff *skb, op = BPF_SOCK_OPS_TS_SW_OPT_CB; } break; + case SCM_TSTAMP_ACK: + op = BPF_SOCK_OPS_TS_ACK_OPT_CB; + break; default: return; } diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 0d704bda6c41..12b9c4f9c151 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -488,7 +488,7 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc) sock_tx_timestamp(sk, sockc, &shinfo->tx_flags); if (tsflags & SOF_TIMESTAMPING_TX_ACK) - tcb->txstamp_ack = 1; + tcb->txstamp_ack |= TSTAMP_ACK_SK; if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK) shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 7b9652ce7e3c..d3e2988b3b4c 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7037,6 +7037,11 @@ enum { * SK_BPF_CB_TX_TIMESTAMPING feature * is on. */ + BPF_SOCK_OPS_TS_ACK_OPT_CB, /* Called when all the skbs in the + * same sendmsg call are acked + * when SK_BPF_CB_TX_TIMESTAMPING + * feature is on. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Feb 18 05:01:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979012 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31D241AF0C2; Tue, 18 Feb 2025 05:03:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854986; cv=none; b=j81NVqytUNGRiLm/A/RqoVEN0GsRDn7iScumWRX0c6wTvdGW+Ql53fYvQJBzPmKR5mx7Y0iIUBABroDPZhxt09w1IHvsT0zqHJavkcJV9EmRvgDp9WAaSDw0ieJ+qZf/K/QyvNYYGCkRbuG0/BxDFq6oPWPSBJG4QK1BP07K6XE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854986; c=relaxed/simple; bh=JPoQgaS/t3HFTd09aBUhh/TgXSHh06+b57Va1cSIE4c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Bp1raMa573gGALF+JB8qWt+5hxC1FH3dlWSZIrsg65Nr+Dv4VgiEKAqXGpv6kAiRfOXEF0wPJpFV9La9eJfLCxB3/a42ogYVlsZhYQNiy4i1BopLfeNwUySAf3LmWnr7X3w/1qA01GNvI7GHNHYN9wlTWCYTTm1uZjgHoVxkHUA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FeTpup2X; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FeTpup2X" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-220e83d65e5so76535615ad.1; Mon, 17 Feb 2025 21:03:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854984; x=1740459784; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LS3SgII/6U7/5mkIGtNBNd4qXYAT6zWaOxbAl9i5SRo=; b=FeTpup2X00PiOhKy5rFgER1t5AA6XGANiLOUF0LyCXjSh2uBswzO64DQWyi6/BAymL IJI4ATMs6xm80tdDLPofR/U8qH3zl3wApTcgl3s9iXUo2+imHe/6NDKRQPJPvG+oq1oI gXLx9114cvS1eiFhX1dVk5K/7w6m+M10OGn451/bJSVY/ghEZ8A1DbKgP9KS9sjiDGAP U3peZywmBDxCy1NCmeK4r9U819hGi0hjM9w//Eb+aSRnNfHylN+DSw1mOEd16MhDNdUA +SmE7cfWVN9dhTuJbcEHHbTFdvp9S6AHRIRtJdxGHH0uLF1r9Rr++mSd+KM5wlxL3o5t Q9Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854984; x=1740459784; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LS3SgII/6U7/5mkIGtNBNd4qXYAT6zWaOxbAl9i5SRo=; b=hVcIWic8QEDLP7tBr38WlooJOPbi/SJju85LHgWZk6h4gjDfJ0bokiDQyinHV1t0Rd 1Pye4Cno4amPTWC59KPUjoFRrN6YYcWMyvHG7nbHyDrlvCCPLZgHK4Uta7IomDWilxof 5rKKJCHS+nefSLtyGFWAzxX3vh9bNPoVbqwM4mJmW5vD/l9AUPpdOucgMcPw4g1TV69d YlAKxrzjm0ny7Pku/ftRCqC4h+lakgaHDRJnjQTk0ZHapyv6O/Rjoz6HyNeUmnhJPO7f RaOgNIGzTi/czgf38zXBGvnPK3+WRH711iLOp20yJqkjPA5/Rrm4IxwTeytY6MWRCNN7 5Ymg== X-Forwarded-Encrypted: i=1; AJvYcCVScMNsnl/u5mRWVe6vFrG7sTLJVDnJK6hzZZlyMsRokSLo95Zh6IWEu4iXGFUKPmi3zAtRwKc=@vger.kernel.org X-Gm-Message-State: AOJu0YzOeLwBzEh2Z5jALRoHn6X8gv2Zz1ymH7VlG+/FOD2lUuIXDKyc GUMbcU3rnyRFciFk3RdzA5vuZHY190NhYQgeCSQ2cOfDcxV9QkH7 X-Gm-Gg: ASbGnct99Q2+dlbBh0MwBHs5zuTFbNJcyLIqXGMEhkGEqVtdCrzLMUbzHUexn9kC9FB +BOM4PANYImSZj4vapgADL15WAujKGgEMLOZp2RjLk6Rpc+dbcmikuq9CHCCt/E7pAVtskSsFQf SjZQDa//x7fxkqvcg8kA9j1Zb8iBoM6Gop8+Otfm7+ummdGn9oSZcbmUlqfRGtb7mdKDgCoDaEm 5BCIz9ijsh/6CwhI3WMGWi6G4ggVG2rBvb7u3fDEyJzaWGwiigWZXmmHgxLzIQognqQ99CIjyvv SdtOKQ1YaG9S3wIIDx97F0qCotSjjjsdqPwOPjXiWrbxi23HiRbFXaoX9YfA6iE= X-Google-Smtp-Source: AGHT+IEz6xMo41QHYsr+T/pmGF28Lz//2TqP2lecZxbeYMrtWlLLua55udRa0CBPOKX2Z5yPmRWEiw== X-Received: by 2002:a05:6a21:339b:b0:1ee:c7c8:c9d with SMTP id adf61e73a8af0-1eec7c80d8dmr2248683637.4.1739854984486; Mon, 17 Feb 2025 21:03:04 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.02.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:03:04 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 10/12] bpf: add BPF_SOCK_OPS_TS_SND_CB callback Date: Tue, 18 Feb 2025 13:01:23 +0800 Message-Id: <20250218050125.73676-11-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This patch introduces a new callback in tcp_tx_timestamp() to correlate tcp_sendmsg timestamp with timestamps from other tx timestamping callbacks (e.g., SND/SW/ACK). Without this patch, BPF program wouldn't know which timestamps belong to which flow because of no socket lock protection. This new callback is inserted in tcp_tx_timestamp() to address this issue because tcp_tx_timestamp() still owns the same socket lock with tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes the timestamping related fields for the skb, especially tskey. The tskey is the bridge to do the correlation. For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and then stores the sendmsg timestamp at the bpf_sk_storage, correlating this timestamp with its tskey that are later used in other sending timestamping callbacks. Signed-off-by: Jason Xing Reviewed-by: Willem de Bruijn --- include/uapi/linux/bpf.h | 5 +++++ net/ipv4/tcp.c | 4 ++++ tools/include/uapi/linux/bpf.h | 5 +++++ 3 files changed, 14 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 9355d617767f..86fca729fbd8 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7052,6 +7052,11 @@ enum { * when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SND_CB, /* Called when every sendmsg syscall + * is triggered. It's used to correlate + * sendmsg timestamp with corresponding + * tskey. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 12b9c4f9c151..4b9739cd3bc5 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -492,6 +492,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc) if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK) shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; } + + if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) && + SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) + bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TS_SND_CB); } static bool tcp_stream_is_readable(struct sock *sk, int target) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index d3e2988b3b4c..2739ee0154a0 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7042,6 +7042,11 @@ enum { * when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TS_SND_CB, /* Called when every sendmsg syscall + * is triggered. It's used to correlate + * sendmsg timestamp with corresponding + * tskey. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect From patchwork Tue Feb 18 05:01:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979013 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EAAA1AF0C2; Tue, 18 Feb 2025 05:03:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854992; cv=none; b=NS7zrcRnIgusGRPjQTjiOvR3BRJIsWlYW0WRZ/PIaTEmbwXwHEePuEkLryK6bo07zKq6Dl4wEYPDYPwxnEg3vKh0NFDv2D++peCFlhM4ZeZ9Zwt3CU2kyeakbyXdCNW3Eu1SymUF34ZBJ7TeHa2h06728oQgYzkwntrWmsnTrGc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854992; c=relaxed/simple; bh=96hvCuSxCs+mUJ6LJEGW3IEdDKTIaTJ+daWiYKxNdCg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T3UIkQ2h0txY8Szze76WLomU1aWJ1e+UuM++loB1MbVlDAqTBFhJnVHqwTycwlGsKE6nLNzr5rMRvdRXOUSSmRWCKtZOsBM3BhiqY75Zp/2ENr1Tcg9DoZsk8LeglSprJ1Ptt32PC/bbeHSc9AMIljnYdppGjUgQ9pH35EdmPRs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Z6pS3435; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Z6pS3435" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-21f2339dcfdso75647765ad.1; Mon, 17 Feb 2025 21:03:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854990; x=1740459790; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pw0RyHf+BlC6NWbRId5LTA+LQRScTEJXKQpbnXvfIO4=; b=Z6pS3435r000U8K9zBzpFlJ7895KDX3qu/sIA327Rp5FZKH9S2pfGa3eQ3jZaCEqTT iVYXo5BXQpl5y2b4Ld6HhvcIMS9LU8j2eshsrY6Yje3dwKkyCKLsHLkBzSvecEoRO89z 6u3DIhQn2U4wYh/JW58MlWDpj9DJc6YLt6CV6vGijxjFVw+eF9EVMcJECtgNKDVnMAFZ DOFx/KfGZox2fc7bvquT/i3mMoiS+f5J7T44jGrbmoa1zQ7a8LuEAzV9eFbNJ26xGiQv GPiWG6ctJ9b1nT0F8OAbeH9T1X0el5o5Mrc/BtsKn+iMPVOBOy5ZArvCW6NY9fSBeCJL 3cIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854990; x=1740459790; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pw0RyHf+BlC6NWbRId5LTA+LQRScTEJXKQpbnXvfIO4=; b=R/urew3x2NjvVUv7Qr8uVFGm9J8H3f8NigSQriAlYVkZHIagsYaParjw1vS45N3XwO TXnPK7mcltLJAr3s0bOMd37hEQ0HdvL6Zdnf37HI/HsRaNgmv1z0mgC2hJyM6bmi5wa/ a1NeASO50EHSR5gFDiwcobNiRJysjKcM/zFSeyKYA/IHsVa+ZRoSF8aBUtWtUgbq3N8S kUVt6tERUjWQyeitYRceXx6/yeFErqMQWwbfticK+oB3bD6c+KpVlG2U3ONXPWs/iAZx oWQmOfa0pWdSFYM0ztVP1Fz8B+7qOIJyLaOaoOhlLXU6K8dd1g4wavZlbgRQTKZWBBsy Y88Q== X-Forwarded-Encrypted: i=1; AJvYcCUsyf8/ktvy4QImCThVhJNBoOsYlfy8MEloxkTzw6DbP8P20ICiTFhOLSyAtX6MZLTBpuBnyGM=@vger.kernel.org X-Gm-Message-State: AOJu0Ywee8OjuBgY1gT7sqpgwFDLjnXnN8F9LMcY1VaYqHr/NKDEdPAD ng6yo87BJe3MTBcmeUKtjp2bQiITetglyZsKJF/fgFaNypIomqzU X-Gm-Gg: ASbGncvGm/bfzC7ScZxujECepfIkOz3BtuES2MDQAmgsF+9QkME/uEfpOnmiwAQrS/v cmOFuX0rHNFHNhhInj6ryp5mLuyHWeD65u1fXbcx8w7alteSmOvsYP4lMRBF644QWM0h/DZEcWh xzhRyCeW80sKcppcYhy4Z/pKm9QhsoQncIQbshdcph+DcRYdLr5GzIDkq4uM7OVXt5uDmriDlaF stNyjBqLKrt+B29cyRgVbp2/i+nW57bhOp1FaE+sMbLLMFmTKaGtt7yK7rwKfZOUx2Iq9VIjneJ zYAwk3vY085PCr5XUBc/n6hYLdi190RIFEUDJSdSgEcIaBa7/a/cnUN7HeatzX4= X-Google-Smtp-Source: AGHT+IFDE3F9vH43EFtyXKUM6dm00EYG1y0nglUMeLGbSxtgNZF4yysZw5kPrlomLcWYIjZ9K0sRcg== X-Received: by 2002:a17:902:da8f:b0:21d:dab4:b360 with SMTP id d9443c01a7336-220d32e4f9fmr314462275ad.0.1739854990379; Mon, 17 Feb 2025 21:03:10 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.03.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:03:10 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 11/12] bpf: support selective sampling for bpf timestamping Date: Tue, 18 Feb 2025 13:01:24 +0800 Message-Id: <20250218050125.73676-12-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add the bpf_sock_ops_enable_tx_tstamp kfunc to allow BPF programs to selectively enable TX timestamping on a skb during tcp_sendmsg(). For example, BPF program will limit tracking X numbers of packets and then will stop there instead of tracing all the sendmsgs of matched flow all along. It would be helpful for users who cannot afford to calculate latencies from every sendmsg call probably due to the performance or storage space consideration. Signed-off-by: Jason Xing --- kernel/bpf/btf.c | 1 + net/core/filter.c | 31 ++++++++++++++++++++++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 9433b6467bbe..740210f883dc 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -8522,6 +8522,7 @@ static int bpf_prog_type_to_kfunc_hook(enum bpf_prog_type prog_type) case BPF_PROG_TYPE_CGROUP_SOCK_ADDR: case BPF_PROG_TYPE_CGROUP_SOCKOPT: case BPF_PROG_TYPE_CGROUP_SYSCTL: + case BPF_PROG_TYPE_SOCK_OPS: return BTF_KFUNC_HOOK_CGROUP; case BPF_PROG_TYPE_SCHED_ACT: return BTF_KFUNC_HOOK_SCHED_ACT; diff --git a/net/core/filter.c b/net/core/filter.c index 7f56d0bbeb00..5611d4a7cf89 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -12102,6 +12102,25 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk, #endif } +__bpf_kfunc int bpf_sock_ops_enable_tx_tstamp(struct bpf_sock_ops_kern *skops, + u64 flags) +{ + struct sk_buff *skb; + + if (skops->op != BPF_SOCK_OPS_TS_SND_CB) + return -EOPNOTSUPP; + + if (flags) + return -EINVAL; + + skb = skops->skb; + skb_shinfo(skb)->tx_flags |= SKBTX_BPF; + TCP_SKB_CB(skb)->txstamp_ack |= TSTAMP_ACK_BPF; + skb_shinfo(skb)->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; + + return 0; +} + __bpf_kfunc_end_defs(); int bpf_dynptr_from_skb_rdonly(struct __sk_buff *skb, u64 flags, @@ -12135,6 +12154,10 @@ BTF_KFUNCS_START(bpf_kfunc_check_set_tcp_reqsk) BTF_ID_FLAGS(func, bpf_sk_assign_tcp_reqsk, KF_TRUSTED_ARGS) BTF_KFUNCS_END(bpf_kfunc_check_set_tcp_reqsk) +BTF_KFUNCS_START(bpf_kfunc_check_set_sock_ops) +BTF_ID_FLAGS(func, bpf_sock_ops_enable_tx_tstamp, KF_TRUSTED_ARGS) +BTF_KFUNCS_END(bpf_kfunc_check_set_sock_ops) + static const struct btf_kfunc_id_set bpf_kfunc_set_skb = { .owner = THIS_MODULE, .set = &bpf_kfunc_check_set_skb, @@ -12155,6 +12178,11 @@ static const struct btf_kfunc_id_set bpf_kfunc_set_tcp_reqsk = { .set = &bpf_kfunc_check_set_tcp_reqsk, }; +static const struct btf_kfunc_id_set bpf_kfunc_set_sock_ops = { + .owner = THIS_MODULE, + .set = &bpf_kfunc_check_set_sock_ops, +}; + static int __init bpf_kfunc_init(void) { int ret; @@ -12173,7 +12201,8 @@ static int __init bpf_kfunc_init(void) ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &bpf_kfunc_set_xdp); ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_CGROUP_SOCK_ADDR, &bpf_kfunc_set_sock_addr); - return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_kfunc_set_tcp_reqsk); + ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_kfunc_set_tcp_reqsk); + return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SOCK_OPS, &bpf_kfunc_set_sock_ops); } late_initcall(bpf_kfunc_init); From patchwork Tue Feb 18 05:01:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Xing X-Patchwork-Id: 13979014 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA2261AF0C2; Tue, 18 Feb 2025 05:03:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854999; cv=none; b=aOvUddYbx0ncJIP2UNu6GR9srYTEFHQGi43Uu5vCHtgKIuHSttz98XPivsOcyXlJOzgrcAtxRzLGk0OgHaWy1TdC7jskX+l4jSushEzBvdzcHpnwzZpxCo0DOsS+KewvFjo5ULDiimr0yfB+lkw4Koq6yZIAWd3lW5jakDLzjfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739854999; c=relaxed/simple; bh=8wVc8kQACvuH7dpMqmuA9pYOlwEvpcbGJRrrHijufWM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UIMp/i2DBV5PhonD19b0XpHEfDigmJ6tdV5OUX63OwFDERz8y3U6UoHrm2hU2Z9/81a9Xvx/5ctRz5oCp1yCUl9NHjVGSEWVSTFOeRP++kNg1/lSLP1Hq6XEzBYkZLzKBDKhqZwPaPvXDUd1SOAcA9y8/hifevK3gvH7+rF5KNY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UCJqxxg9; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UCJqxxg9" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-2fc737aeeb1so2328570a91.3; Mon, 17 Feb 2025 21:03:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739854997; x=1740459797; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UI4DNKaq4drIljj+cCqk5+/yFYwlzlF9id7t8XUb57Y=; b=UCJqxxg9w/DIyM3BMcoaGKKKzlg3Mb+E3d3MjDFCSm1/2fcOsiL67/uyeIahgAD9mf WqNiB+Ci31cm1BfKU3yCYphFkOKZ5BxM/6I5dyGuALUr/jX5uvImJZxm3z5x1En70xWG HXnEtCP8KJY3DJCQ7IJ4qjCnybrii4FqQf7R6fDK6ISJR7i/IOFNMPITxAfY3AHkgJEO yJeWIxAIsMAnE+z4EcOGi8vyxdvrEKm4AApVC2wznHwJCxK/89lXl34hynUnHK75DAyz 2gFU2LHebLv8FNY2pytvxaZq+qZ30S0CU2SV7dlqr71xou+zi4MoUCS+1UtvfZvdDtOL +ifg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739854997; x=1740459797; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UI4DNKaq4drIljj+cCqk5+/yFYwlzlF9id7t8XUb57Y=; b=hmN4hGygEU7DM+RY354DDwo148KNGBdHZ09h2RztDiMWoFqCoqMTfrGFylA4TZUmGE SfQnDDUWv80pXU8GVkhHZNlixs8pDUJqp1btxRCujivBG9EIcb4jm+cs6TPOM1fil3+S yvsWgpp5hhJmojghbFkXF3qTnJvu2nvs/MhUsvfs9oB0blLM2vNeoAeaGIa7r2lUDHcB l+5QpuEO+1UQSuGkIEA0/bNjEzwrlP4STBF2tIV9mFAgzbWINOtum3+Ts5wOe0UksUYn d59MVqFHnmaqagSpAq5daVZaSO/xRpbfFsm4nbOwIP17Zno1KCjCM9QvLnyC9bRy8+T9 Y5qA== X-Forwarded-Encrypted: i=1; AJvYcCVrd+uDamuKO2KZbree85woP8pQ6MTGOQPKKa9flx91lWmZcEiz+xTGp5BFVqJlkVhQ7L+HykI=@vger.kernel.org X-Gm-Message-State: AOJu0YxNbar6MC41h8M9pitAV8Q4YYd0bI6pUGhsY5E1a3VCx+D44Hba xYC0M7Mffj4C20GcOmi0oe+EjzWp0O4D1XTae0xz+0a3ysaFRMdY X-Gm-Gg: ASbGncv4Qo1SR4YOw/r9SusT+FUKG1EkqeK1q0r8W/O/ElwB3ok1ikdDqg7Q7LPYvB0 f7WLwtzlTV7zUa/JfXYTzbx6bf/pUz615MdWlZIwp7bW88N6u2Fps93Yt1pxnqWcEQAofclZ9it MopUqhUUULL5Vw3lDt/vkC+N5eNY7UPlAZ5FyJD9hpJm8PO/KijsEJFyUYmi5jMAZf9GetJdJNj ZMx5JxzP+hPmJ1wZvr1W3w6B2nrGVlmAWvHCWDgU2292IYc4VU9EfhKH28oWmYUcQs0wzE0DEQP fTaTXK6v54ywJ03sWWK+3SR1oVLEV6OmhyVe6gNUDtkNngvmYIDfjAxs6iI4W24= X-Google-Smtp-Source: AGHT+IFQijm+bJDFzTHPBzTifygwiuLWhiOSSoUjoYJ8hcSJekfCaLXDwJ1aNM0/6ydzixMPCgNMWw== X-Received: by 2002:a05:6a00:244e:b0:730:8d25:4c31 with SMTP id d2e1a72fcca58-7326179d09cmr23937779b3a.8.1739854996832; Mon, 17 Feb 2025 21:03:16 -0800 (PST) Received: from KERNELXING-MB0.tencent.com ([43.132.141.20]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adda5be52f1sm4337938a12.34.2025.02.17.21.03.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 21:03:16 -0800 (PST) From: Jason Xing To: davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, dsahern@kernel.org, willemdebruijn.kernel@gmail.com, willemb@google.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, eddyz87@gmail.com, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@fomichev.me, haoluo@google.com, jolsa@kernel.org, shuah@kernel.org, ykolal@fb.com Cc: bpf@vger.kernel.org, netdev@vger.kernel.org, Jason Xing Subject: [PATCH bpf-next v12 12/12] selftests/bpf: add simple bpf tests in the tx path for timestamping feature Date: Tue, 18 Feb 2025 13:01:25 +0800 Message-Id: <20250218050125.73676-13-kerneljasonxing@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20250218050125.73676-1-kerneljasonxing@gmail.com> References: <20250218050125.73676-1-kerneljasonxing@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net BPF program calculates a couple of latency deltas between each tx timestamping callbacks. It can be used in the real world to diagnose the kernel behaviour in the tx path. Check the safety issues by accessing a few bpf calls in bpf_test_access_bpf_calls() which are implemented in the patch 3 and 4. Check if the bpf timestamping can co-exist with socket timestamping. There remains a few realistic things[1][2] to highlight: 1. in general a packet may pass through multiple qdiscs. For instance with bonding or tunnel virtual devices in the egress path. 2. packets may be resent, in which case an ACK might precede a repeat SCHED and SND. 3. erroneous or malicious peers may also just never send an ACK. [1]: https://lore.kernel.org/all/67a389af981b0_14e0832949d@willemb.c.googlers.com.notmuch/ [2]: https://lore.kernel.org/all/c329a0c1-239b-4ca1-91f2-cb30b8dd2f6a@linux.dev/ Signed-off-by: Jason Xing Reviewed-by: Willem de Bruijn --- .../bpf/prog_tests/net_timestamping.c | 239 +++++++++++++++++ .../selftests/bpf/progs/net_timestamping.c | 248 ++++++++++++++++++ 2 files changed, 487 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/net_timestamping.c create mode 100644 tools/testing/selftests/bpf/progs/net_timestamping.c diff --git a/tools/testing/selftests/bpf/prog_tests/net_timestamping.c b/tools/testing/selftests/bpf/prog_tests/net_timestamping.c new file mode 100644 index 000000000000..dbfd87499b6b --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/net_timestamping.c @@ -0,0 +1,239 @@ +#include +#include +#include +#include "test_progs.h" +#include "network_helpers.h" +#include "net_timestamping.skel.h" + +#define CG_NAME "/net-timestamping-test" +#define NSEC_PER_SEC 1000000000LL + +static const char addr4_str[] = "127.0.0.1"; +static const char addr6_str[] = "::1"; +static struct net_timestamping *skel; +static const int cfg_payload_len = 30; +static struct timespec usr_ts; +static u64 delay_tolerance_nsec = 10000000000; /* 10 seconds */ +int SK_TS_SCHED; +int SK_TS_TXSW; +int SK_TS_ACK; + +static int64_t timespec_to_ns64(struct timespec *ts) +{ + return ts->tv_sec * NSEC_PER_SEC + ts->tv_nsec; +} + +static void validate_key(int tskey, int tstype) +{ + static int expected_tskey = -1; + + if (tstype == SCM_TSTAMP_SCHED) + expected_tskey = cfg_payload_len - 1; + + ASSERT_EQ(expected_tskey, tskey, "tskey mismatch"); + + expected_tskey = tskey; +} + +static void validate_timestamp(struct timespec *cur, struct timespec *prev) +{ + int64_t cur_ns, prev_ns; + + cur_ns = timespec_to_ns64(cur); + prev_ns = timespec_to_ns64(prev); + + ASSERT_LT(cur_ns - prev_ns, delay_tolerance_nsec, "latency"); +} + +static void test_socket_timestamp(struct scm_timestamping *tss, int tstype, + int tskey) +{ + static struct timespec prev_ts; + + validate_key(tskey, tstype); + + switch (tstype) { + case SCM_TSTAMP_SCHED: + validate_timestamp(&tss->ts[0], &usr_ts); + SK_TS_SCHED += 1; + break; + case SCM_TSTAMP_SND: + validate_timestamp(&tss->ts[0], &prev_ts); + SK_TS_TXSW += 1; + break; + case SCM_TSTAMP_ACK: + validate_timestamp(&tss->ts[0], &prev_ts); + SK_TS_ACK += 1; + break; + } + + prev_ts = tss->ts[0]; +} + +static void test_recv_errmsg_cmsg(struct msghdr *msg) +{ + struct sock_extended_err *serr = NULL; + struct scm_timestamping *tss = NULL; + struct cmsghdr *cm; + + for (cm = CMSG_FIRSTHDR(msg); + cm && cm->cmsg_len; + cm = CMSG_NXTHDR(msg, cm)) { + if (cm->cmsg_level == SOL_SOCKET && + cm->cmsg_type == SCM_TIMESTAMPING) { + tss = (void *)CMSG_DATA(cm); + } else if ((cm->cmsg_level == SOL_IP && + cm->cmsg_type == IP_RECVERR) || + (cm->cmsg_level == SOL_IPV6 && + cm->cmsg_type == IPV6_RECVERR) || + (cm->cmsg_level == SOL_PACKET && + cm->cmsg_type == PACKET_TX_TIMESTAMP)) { + serr = (void *)CMSG_DATA(cm); + ASSERT_EQ(serr->ee_origin, SO_EE_ORIGIN_TIMESTAMPING, + "cmsg type"); + } + + if (serr && tss) + test_socket_timestamp(tss, serr->ee_info, + serr->ee_data); + } +} + +static bool socket_recv_errmsg(int fd) +{ + static char ctrl[1024 /* overprovision*/]; + char data[cfg_payload_len]; + static struct msghdr msg; + struct iovec entry; + int n = 0; + + memset(&msg, 0, sizeof(msg)); + memset(&entry, 0, sizeof(entry)); + memset(ctrl, 0, sizeof(ctrl)); + + entry.iov_base = data; + entry.iov_len = cfg_payload_len; + msg.msg_iov = &entry; + msg.msg_iovlen = 1; + msg.msg_name = NULL; + msg.msg_namelen = 0; + msg.msg_control = ctrl; + msg.msg_controllen = sizeof(ctrl); + + n = recvmsg(fd, &msg, MSG_ERRQUEUE); + if (n == -1) + ASSERT_EQ(errno, EAGAIN, "recvmsg MSG_ERRQUEUE"); + + if (n >= 0) + test_recv_errmsg_cmsg(&msg); + + return n == -1; +} + +static void test_socket_timestamping(int fd) +{ + while (!socket_recv_errmsg(fd)); + + ASSERT_EQ(SK_TS_SCHED, 1, "SCM_TSTAMP_SCHED"); + ASSERT_EQ(SK_TS_TXSW, 1, "SCM_TSTAMP_SND"); + ASSERT_EQ(SK_TS_ACK, 1, "SCM_TSTAMP_ACK"); + + SK_TS_SCHED = 0; + SK_TS_TXSW = 0; + SK_TS_ACK = 0; +} + +static void test_tcp(int family, bool enable_socket_timestamping) +{ + struct net_timestamping__bss *bss; + char buf[cfg_payload_len]; + int sfd = -1, cfd = -1; + unsigned int sock_opt; + struct netns_obj *ns; + int cg_fd; + int ret; + + cg_fd = test__join_cgroup(CG_NAME); + if (!ASSERT_OK_FD(cg_fd, "join cgroup")) + return; + + ns = netns_new("net_timestamping_ns", true); + if (!ASSERT_OK_PTR(ns, "create ns")) + goto out; + + skel = net_timestamping__open_and_load(); + if (!ASSERT_OK_PTR(skel, "open and load skel")) + goto out; + + if (!ASSERT_OK(net_timestamping__attach(skel), "attach skel")) + goto out; + + skel->links.skops_sockopt = + bpf_program__attach_cgroup(skel->progs.skops_sockopt, cg_fd); + if (!ASSERT_OK_PTR(skel->links.skops_sockopt, "attach cgroup")) + goto out; + + bss = skel->bss; + memset(bss, 0, sizeof(*bss)); + + skel->bss->monitored_pid = getpid(); + + sfd = start_server(family, SOCK_STREAM, + family == AF_INET6 ? addr6_str : addr4_str, 0, 0); + if (!ASSERT_OK_FD(sfd, "start_server")) + goto out; + + cfd = connect_to_fd(sfd, 0); + if (!ASSERT_OK_FD(cfd, "connect_to_fd_server")) + goto out; + + if (enable_socket_timestamping) { + sock_opt = SOF_TIMESTAMPING_SOFTWARE | + SOF_TIMESTAMPING_OPT_ID | + SOF_TIMESTAMPING_TX_SCHED | + SOF_TIMESTAMPING_TX_SOFTWARE | + SOF_TIMESTAMPING_TX_ACK; + ret = setsockopt(cfd, SOL_SOCKET, SO_TIMESTAMPING, + (char *) &sock_opt, sizeof(sock_opt)); + if (!ASSERT_OK(ret, "setsockopt SO_TIMESTAMPING")) + goto out; + + ret = clock_gettime(CLOCK_REALTIME, &usr_ts); + if (!ASSERT_OK(ret, "get user time")) + goto out; + } + + ret = write(cfd, buf, sizeof(buf)); + if (!ASSERT_EQ(ret, sizeof(buf), "send to server")) + goto out; + + if (enable_socket_timestamping) + test_socket_timestamping(cfd); + + ASSERT_EQ(bss->nr_active, 1, "nr_active"); + ASSERT_EQ(bss->nr_snd, 2, "nr_snd"); + ASSERT_EQ(bss->nr_sched, 1, "nr_sched"); + ASSERT_EQ(bss->nr_txsw, 1, "nr_txsw"); + ASSERT_EQ(bss->nr_ack, 1, "nr_ack"); + +out: + if (sfd >= 0) + close(sfd); + if (cfd >= 0) + close(cfd); + net_timestamping__destroy(skel); + netns_free(ns); + close(cg_fd); +} + +void test_net_timestamping(void) +{ + if (test__start_subtest("INET4: bpf timestamping")) + test_tcp(AF_INET, false); + if (test__start_subtest("INET4: bpf and socket timestamping")) + test_tcp(AF_INET, true); + if (test__start_subtest("INET6: bpf timestamping")) + test_tcp(AF_INET6, false); + if (test__start_subtest("INET6: bpf and socket timestamping")) + test_tcp(AF_INET6, true); +} diff --git a/tools/testing/selftests/bpf/progs/net_timestamping.c b/tools/testing/selftests/bpf/progs/net_timestamping.c new file mode 100644 index 000000000000..ab2ed2ea5822 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/net_timestamping.c @@ -0,0 +1,248 @@ +#include "vmlinux.h" +#include "bpf_tracing_net.h" +#include +#include +#include "bpf_misc.h" +#include "bpf_kfuncs.h" +#include + +__u32 monitored_pid = 0; + +int nr_active; +int nr_snd; +int nr_passive; +int nr_sched; +int nr_txsw; +int nr_ack; + +struct sk_stg { + __u64 sendmsg_ns; /* record ts when sendmsg is called */ +}; + +struct sk_tskey { + u64 cookie; + u32 tskey; +}; + +struct delay_info { + u64 sendmsg_ns; /* record ts when sendmsg is called */ + u32 sched_delay; /* SCHED_OPT_CB - sendmsg_ns */ + u32 sw_snd_delay; /* SW_OPT_CB - SCHED_OPT_CB */ + u32 ack_delay; /* ACK_OPT_CB - SW_OPT_CB */ +}; + +struct { + __uint(type, BPF_MAP_TYPE_SK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, struct sk_stg); +} sk_stg_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __type(key, struct sk_tskey); + __type(value, struct delay_info); + __uint(max_entries, 1024); +} time_map SEC(".maps"); + +static u64 delay_tolerance_nsec = 10000000000; /* 10 second as an example */ + +extern int bpf_sock_ops_enable_tx_tstamp(struct bpf_sock_ops_kern *skops, u64 flags) __ksym; + +static int bpf_test_sockopt(void *ctx, const struct sock *sk, int expected) +{ + int tmp, new = SK_BPF_CB_TX_TIMESTAMPING; + int opt = SK_BPF_CB_FLAGS; + int level = SOL_SOCKET; + + if (bpf_setsockopt(ctx, level, opt, &new, sizeof(new)) != expected) + return 1; + + if (bpf_getsockopt(ctx, level, opt, &tmp, sizeof(tmp)) != expected || + (!expected && tmp != new)) + return 1; + + return 0; +} + +static bool bpf_test_access_sockopt(void *ctx, const struct sock *sk) +{ + if (bpf_test_sockopt(ctx, sk, -EOPNOTSUPP)) + return true; + return false; +} + +static bool bpf_test_access_load_hdr_opt(struct bpf_sock_ops *skops) +{ + u8 opt[3] = {0}; + int load_flags = 0; + int ret; + + ret = bpf_load_hdr_opt(skops, opt, sizeof(opt), load_flags); + if (ret != -EOPNOTSUPP) + return true; + + return false; +} + +static bool bpf_test_access_cb_flags_set(struct bpf_sock_ops *skops) +{ + int ret; + + ret = bpf_sock_ops_cb_flags_set(skops, 0); + if (ret != -EOPNOTSUPP) + return true; + + return false; +} + +/* In the timestamping callbacks, we're not allowed to call the following + * BPF CALLs for the safety concern. Return false if expected. + */ +static bool bpf_test_access_bpf_calls(struct bpf_sock_ops *skops, + const struct sock *sk) +{ + if (bpf_test_access_sockopt(skops, sk)) + return true; + + if (bpf_test_access_load_hdr_opt(skops)) + return true; + + if (bpf_test_access_cb_flags_set(skops)) + return true; + + return false; +} + +static bool bpf_test_delay(struct bpf_sock_ops *skops, const struct sock *sk) +{ + struct bpf_sock_ops_kern *skops_kern; + u64 timestamp = bpf_ktime_get_ns(); + struct skb_shared_info *shinfo; + struct delay_info dinfo = {0}; + struct sk_tskey key = {0}; + struct delay_info *val; + struct sk_buff *skb; + struct sk_stg *stg; + u64 prior_ts, delay; + + if (bpf_test_access_bpf_calls(skops, sk)) + return false; + + skops_kern = bpf_cast_to_kern_ctx(skops); + skb = skops_kern->skb; + shinfo = bpf_core_cast(skb->head + skb->end, struct skb_shared_info); + + key.cookie = bpf_get_socket_cookie(skops); + if (!key.cookie) + return false; + + if (skops->op == BPF_SOCK_OPS_TS_SND_CB) { + stg = bpf_sk_storage_get(&sk_stg_map, (void *)sk, 0, 0); + if (!stg) + return false; + dinfo.sendmsg_ns = stg->sendmsg_ns; + bpf_sock_ops_enable_tx_tstamp(skops_kern, 0); + key.tskey = shinfo->tskey; + if (!key.tskey) + return false; + bpf_map_update_elem(&time_map, &key, &dinfo, BPF_ANY); + return true; + } + + key.tskey = shinfo->tskey; + if (!key.tskey) + return false; + + val = bpf_map_lookup_elem(&time_map, &key); + if (!val) + return false; + + switch (skops->op) { + case BPF_SOCK_OPS_TS_SCHED_OPT_CB: + val->sched_delay = timestamp - val->sendmsg_ns; + delay = val->sched_delay; + break; + case BPF_SOCK_OPS_TS_SW_OPT_CB: + prior_ts = val->sched_delay + val->sendmsg_ns; + val->sw_snd_delay = timestamp - prior_ts; + delay = val->sw_snd_delay; + break; + case BPF_SOCK_OPS_TS_ACK_OPT_CB: + prior_ts = val->sw_snd_delay + val->sched_delay + val->sendmsg_ns; + val->ack_delay = timestamp - prior_ts; + delay = val->ack_delay; + break; + } + + if (delay >= delay_tolerance_nsec) + return false; + + /* Since it's the last one, remove from the map after latency check */ + if (skops->op == BPF_SOCK_OPS_TS_ACK_OPT_CB) + bpf_map_delete_elem(&time_map, &key); + + return true; +} + +SEC("fentry/tcp_sendmsg_locked") +int BPF_PROG(trace_tcp_sendmsg_locked, struct sock *sk, struct msghdr *msg, + size_t size) +{ + __u32 pid = bpf_get_current_pid_tgid() >> 32; + u64 timestamp = bpf_ktime_get_ns(); + u32 flag = sk->sk_bpf_cb_flags; + struct sk_stg *stg; + + if (pid != monitored_pid || !flag) + return 0; + + stg = bpf_sk_storage_get(&sk_stg_map, sk, 0, + BPF_SK_STORAGE_GET_F_CREATE); + if (!stg) + return 0; + + stg->sendmsg_ns = timestamp; + nr_snd += 1; + return 0; +} + +SEC("sockops") +int skops_sockopt(struct bpf_sock_ops *skops) +{ + struct bpf_sock *bpf_sk = skops->sk; + const struct sock *sk; + + if (!bpf_sk) + return 1; + + sk = (struct sock *)bpf_skc_to_tcp_sock(bpf_sk); + if (!sk) + return 1; + + switch (skops->op) { + case BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB: + nr_active += !bpf_test_sockopt(skops, sk, 0); + break; + case BPF_SOCK_OPS_TS_SND_CB: + if (bpf_test_delay(skops, sk)) + nr_snd += 1; + break; + case BPF_SOCK_OPS_TS_SCHED_OPT_CB: + if (bpf_test_delay(skops, sk)) + nr_sched += 1; + break; + case BPF_SOCK_OPS_TS_SW_OPT_CB: + if (bpf_test_delay(skops, sk)) + nr_txsw += 1; + break; + case BPF_SOCK_OPS_TS_ACK_OPT_CB: + if (bpf_test_delay(skops, sk)) + nr_ack += 1; + break; + } + + return 1; +} + +char _license[] SEC("license") = "GPL";