From patchwork Tue May 23 09:46:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13251996 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7EFC1DDE5 for ; Tue, 23 May 2023 09:47:14 +0000 (UTC) Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5570794 for ; Tue, 23 May 2023 02:47:13 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-64d426e63baso4138549b3a.0 for ; Tue, 23 May 2023 02:47:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1684835233; x=1687427233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zSeZy3dBgM/hAvW1l4S3j81M4ry+F3vLfbl4Wg72DeY=; b=O06lEBAs+5Eoxzd1wIs6DBaPLQaIKoySacNOnVoqMNyuuuH7QAO7G1QGk4c99I8+s/ ZGgXooronTdovPxYNwkii+X4BWjP+OkCBIQKjNd2rdLzhkzbEjBTRHnEYvW2t9LWHSCc epJfFOheoOPOBWJ2uYRHSVB+TRR7MnHFCHHipwrAZL3WwjLmvdaWxE/ufB9Ni23DX9tR aVC8Ed1tWF4kMycrLwek1nsWvnOWC2zkkZ8U3l7l2+j6/c8lOx8I+/0h+1U0uKlQTULU YRrQetIlPircJ5m0BuklZMOMwuPFbKEt7M3gFbVbueJosraYOXJxbjMfNCCRhuTkdtyr koTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684835233; x=1687427233; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zSeZy3dBgM/hAvW1l4S3j81M4ry+F3vLfbl4Wg72DeY=; b=IZHofvo4yH4boZs14eNYrReHGzXhruV4O6Iu8REmEaADr1get8V2wYlPqFY9u/JzIj j8UAJ3/aFaZEV7YBrA80A+qB6bSsyIv6C2Na6/f/JEnPXrwgYo9hYJR6DTWMWgv8ppwR rOggMsYViGaXPJ7BhlhVLZVVSJyySgQ/eHsonsTp3mi8BK6NUKDw+EGBWRTveKbpq/5z 4Cm2McfbnODMPGSbf6PvxlkATFKso6+yeEUTD7EeCGUmwdzoBQPi/hWFxe3+3CoUZUok FXBqEqnnSA8LhcwagHeWOeVKnc1DGyerGLIUbA5rmv/w10y0zLxUwTfPizdBJeCj/3dE Poyg== X-Gm-Message-State: AC+VfDzIiXxoaskwCOu8/bAinSLnCJrUl70rzJ/Fc3DJII/hUe/9RNV6 mkK0r2PuVDZkPhQ4zua8NPAx2g== X-Google-Smtp-Source: ACHHUZ7pooz0B7VbymfYjthlSe/h/bL0s1OnpZY/LiIbUylPNxh9mStfsdIABB88QcmT4d2NFpElYA== X-Received: by 2002:a05:6a00:98f:b0:645:cfb0:2779 with SMTP id u15-20020a056a00098f00b00645cfb02779mr19397362pfg.26.1684835232865; Tue, 23 May 2023 02:47:12 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id 22-20020aa79116000000b0063b898b3502sm5457216pfh.153.2023.05.23.02.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 02:47:12 -0700 (PDT) From: Abel Wu To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Abel Wu Subject: [PATCH v3 1/5] net-memcg: Fold dependency into memcg pressure cond Date: Tue, 23 May 2023 17:46:48 +0800 Message-Id: <20230523094652.49411-2-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230523094652.49411-1-wuyun.abel@bytedance.com> References: <20230523094652.49411-1-wuyun.abel@bytedance.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org The callers of mem_cgroup_under_socket_pressure() should always make sure that (mem_cgroup_sockets_enabled && sk->sk_memcg) is true. So instead of coding around all the callsites, put the dependencies into mem_cgroup_under_socket_pressure() to avoid redundancy and possibly bugs. This change might also introduce slight function call overhead *iff* the function gets expanded in the future. But for now this change doesn't make binaries different (checked by vimdiff) except the one net/ipv4/tcp_input.o (by scripts/bloat-o-meter), which is probably negligible to performance: add/remove: 0/0 grow/shrink: 1/2 up/down: 5/-5 (0) Function old new delta tcp_grow_window 573 578 +5 tcp_try_rmem_schedule 1083 1081 -2 tcp_check_space 324 321 -3 Total: Before=44647, After=44647, chg +0.00% So folding the dependencies into mem_cgroup_under_socket_pressure() is generally a good thing and provides better readablility. Signed-off-by: Abel Wu --- include/linux/memcontrol.h | 2 ++ include/net/sock.h | 3 +-- include/net/tcp.h | 3 +-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 222d7370134c..a1aead140ff8 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1743,6 +1743,8 @@ void mem_cgroup_sk_alloc(struct sock *sk); void mem_cgroup_sk_free(struct sock *sk); static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg) { + if (!mem_cgroup_sockets_enabled || !memcg) + return false; if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && memcg->tcpmem_pressure) return true; do { diff --git a/include/net/sock.h b/include/net/sock.h index 8b7ed7167243..641c9373b44b 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1414,8 +1414,7 @@ static inline bool sk_under_memory_pressure(const struct sock *sk) if (!sk->sk_prot->memory_pressure) return false; - if (mem_cgroup_sockets_enabled && sk->sk_memcg && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) + if (mem_cgroup_under_socket_pressure(sk->sk_memcg)) return true; return !!*sk->sk_prot->memory_pressure; diff --git a/include/net/tcp.h b/include/net/tcp.h index 04a31643cda3..3c5e3718b454 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -261,8 +261,7 @@ extern unsigned long tcp_memory_pressure; /* optimized version of sk_under_memory_pressure() for TCP sockets */ static inline bool tcp_under_memory_pressure(const struct sock *sk) { - if (mem_cgroup_sockets_enabled && sk->sk_memcg && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) + if (mem_cgroup_under_socket_pressure(sk->sk_memcg)) return true; return READ_ONCE(tcp_memory_pressure); From patchwork Tue May 23 09:46:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13251997 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B0FF1DDE5 for ; Tue, 23 May 2023 09:47:19 +0000 (UTC) Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DA57109 for ; Tue, 23 May 2023 02:47:17 -0700 (PDT) Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-64d2b42a8f9so4052114b3a.3 for ; Tue, 23 May 2023 02:47:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1684835237; x=1687427237; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7JbrrHsu8KzXBp8abtNpZsLO9DbdsQWsXzCPkkoYMdc=; b=LeHmRtCQCbob1Ad8YEm2YLv5mAvn+aMuWIv78A1fohJzHjOiXc2rnwxby8WYnL/ewR MM/7pB74gbS7eWJdjpORMMbyBiKMsXNEt5il7o9ta5Xb8AQNhVbeCV53MtlCKD+g508Q blH+tGTg7Ad8TJmgxxgRqvyA+gGdI+HlgAxQPcczFNCWeQ9K2YrePJc4cC9xs85ovsqf KpGnxmHPLuTkrbE7lo+EzKQQENuGbRlZiS2MY4/iSmoOG0eV0ubhmDgjaQQ/+E313Fq0 xqIhDGAC3+YdMS3ra3frtU5ICniR63D2pB6KBB73hMsEgnpaEENCB6/OsfpVmRS6qRJc NL8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684835237; x=1687427237; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7JbrrHsu8KzXBp8abtNpZsLO9DbdsQWsXzCPkkoYMdc=; b=YOhIb9svO/WRuRutIMu1/nuIy6H/wz2Un6QoUc8bHXwqPYqhcHmjWPMmA9xYVo/56T nOBquISl5Y6ZPO+ligsdN/aS5bCBgcRoAEHXovMRxKyLKEvdFoATBszNavvU/50yksIm lLd+BjM6CXfA2/rN5YGQJoh2UhqbG6b0gziIqxNs1Lje9faMZ6lv/Bl86Iz90PDtTZJk Or3Y563sS/fNXxfBCzbmlaLlF0epYMfrfis4xxKZ9CUR/khAifQbYeBkYEwxasnDqymK DvhqQ/YQUs3WVIWbD+F+zBtmvR3+RSQOcmYTiHi1jbNJOi5A0mMB/h48NAfZg6+6dh6u Nofg== X-Gm-Message-State: AC+VfDzr0fbaCFTztFecb2IfP1ZvRDp97A0xwUOHvloJg7VC/rl1KPTz OWzPMPLMS8z7UpLVDUphjyMzE9e179tpIBv4Z0w= X-Google-Smtp-Source: ACHHUZ4S1nlt676DSbN0WKhez6GOfAS9tr9gLhf6vmcHbhMyBBQJuTFycQSDrPFN6/5nIhSjkHTd8w== X-Received: by 2002:a05:6a20:3d0c:b0:106:8b:99d2 with SMTP id y12-20020a056a203d0c00b00106008b99d2mr16375595pzi.51.1684835236832; Tue, 23 May 2023 02:47:16 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id 22-20020aa79116000000b0063b898b3502sm5457216pfh.153.2023.05.23.02.47.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 02:47:16 -0700 (PDT) From: Abel Wu To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Abel Wu Subject: [PATCH v3 2/5] sock: Always take memcg pressure into consideration Date: Tue, 23 May 2023 17:46:49 +0800 Message-Id: <20230523094652.49411-3-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230523094652.49411-1-wuyun.abel@bytedance.com> References: <20230523094652.49411-1-wuyun.abel@bytedance.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org The sk_under_memory_pressure() is called to check whether there is memory pressure related to this socket. But now it ignores the net- memcg's pressure if the proto of the socket doesn't care about the global pressure, which may put burden on its memcg compaction or reclaim path (also remember that socket memory is un-reclaimable). So always check the memcg's vm status to alleviate memstalls when it's in pressure. Signed-off-by: Abel Wu --- include/net/sock.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index 641c9373b44b..b0e5533e5909 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1411,13 +1411,11 @@ static inline bool sk_has_memory_pressure(const struct sock *sk) static inline bool sk_under_memory_pressure(const struct sock *sk) { - if (!sk->sk_prot->memory_pressure) - return false; - if (mem_cgroup_under_socket_pressure(sk->sk_memcg)) return true; - return !!*sk->sk_prot->memory_pressure; + return sk->sk_prot->memory_pressure && + *sk->sk_prot->memory_pressure; } static inline long From patchwork Tue May 23 09:46:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13251998 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 480871DDE5 for ; Tue, 23 May 2023 09:47:23 +0000 (UTC) Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DEE52132 for ; Tue, 23 May 2023 02:47:20 -0700 (PDT) Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-64d604cc0aaso2028896b3a.2 for ; Tue, 23 May 2023 02:47:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1684835240; x=1687427240; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RMeJmdZ3LE5akFulN2YKZF3yDR93Nfz5GrpK/FAs6Mo=; b=B880Cxou7jMNKnnSKqDvzpC383QnKFgaQTrZAh4pbCWGQDOYZUEc8zYZUAxp7NPvWc CsjI7yZYanPUalpRPeD2nCqoLZMUtdNB+Ms+r7OTLt11AnB6gE9PzZX2y4l/uEgB82su bk4IUOW9jAt+YQbn+pEhnisTxz4JT9+fK1UPMkDDqmvFehCp110UUTwT82dkhOCRSV2B OusFDNXP5SXR3tQoyhiFkMm9c3F5Ystg4nQLs88KZkmM4R+US3twYrGo4ZoU+pTU+56O Al76H1vOTCw/bvnS3oRpbJAO76zM5SiM+8ysyWNCRK02fSpe4Lv3Usptqw1AiFqFxgJ3 gufg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684835240; x=1687427240; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RMeJmdZ3LE5akFulN2YKZF3yDR93Nfz5GrpK/FAs6Mo=; b=I+NbI8RXpxoxsBEsOoRy9ZRG4aqvHY96+Mr1L6SdR6WNYSRRp1TtlXR9CAi7Dr14Di zwKXpYEHvN9AJQ8ITgq/UgUjlm/MdNHwRE0AtdRULWjMezWxlJMtxS0BHDFnl+uKIyTP UvCicSdji/cqPWKVRKQF2dr15RSxMMIx6dBR0ZrPRgnASN4CJTc8FI7ehAXYfHUUSVMI SuUYFQ7c1Xi4tI5LALciH5AprSzzfE7MNTTM8LtUTK97OSYVV/jEn4RBEYQbjQjCpSkl R7uC5MTVS60Vy93xC02kF890hWLAFgdVUjR2pLMqgHolZh2ZHhFvKjpw8Cv3//4aP8Wl gYHw== X-Gm-Message-State: AC+VfDw2Leo92AzjESjH7MuS+RaxN8GiJwwwBv0ks9ptPY9PrnLxAV53 1s9MMqFrcwIBWCQmfbJKZZ3MVbFEPlZUlx2B9q8= X-Google-Smtp-Source: ACHHUZ60ph85ln3tks03J8F3mcWJkoGwj8Q3B23yzE+pbnOSHqFgxY7Jd2TalWB1xt2fu/FZ4z/eVg== X-Received: by 2002:a05:6a00:2e0e:b0:646:3c2:4d30 with SMTP id fc14-20020a056a002e0e00b0064603c24d30mr16266161pfb.30.1684835240594; Tue, 23 May 2023 02:47:20 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id 22-20020aa79116000000b0063b898b3502sm5457216pfh.153.2023.05.23.02.47.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 02:47:20 -0700 (PDT) From: Abel Wu To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Abel Wu Subject: [PATCH v3 3/5] sock: Fix misuse of sk_under_memory_pressure() Date: Tue, 23 May 2023 17:46:50 +0800 Message-Id: <20230523094652.49411-4-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230523094652.49411-1-wuyun.abel@bytedance.com> References: <20230523094652.49411-1-wuyun.abel@bytedance.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org The status of global socket memory pressure is updated when: a) __sk_mem_raise_allocated(): enter: sk_memory_allocated(sk) > sysctl_mem[1] leave: sk_memory_allocated(sk) <= sysctl_mem[0] b) __sk_mem_reduce_allocated(): leave: sk_under_memory_pressure(sk) && sk_memory_allocated(sk) < sysctl_mem[0] So the conditions of leaving global pressure are inconstant, which may lead to the situation that one pressured net-memcg prevents the global pressure from being cleared when there is indeed no global pressure, thus the global constrains are still in effect unexpectedly on the other sockets. This patch fixes this by ignoring the net-memcg's pressure when deciding whether should leave global memory pressure. Fixes: e1aab161e013 ("socket: initial cgroup code.") Signed-off-by: Abel Wu --- include/net/sock.h | 9 +++++++-- net/core/sock.c | 2 +- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/include/net/sock.h b/include/net/sock.h index b0e5533e5909..257706710be5 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1409,13 +1409,18 @@ static inline bool sk_has_memory_pressure(const struct sock *sk) return sk->sk_prot->memory_pressure != NULL; } +static inline bool sk_under_global_memory_pressure(const struct sock *sk) +{ + return sk->sk_prot->memory_pressure && + *sk->sk_prot->memory_pressure; +} + static inline bool sk_under_memory_pressure(const struct sock *sk) { if (mem_cgroup_under_socket_pressure(sk->sk_memcg)) return true; - return sk->sk_prot->memory_pressure && - *sk->sk_prot->memory_pressure; + return sk_under_global_memory_pressure(sk); } static inline long diff --git a/net/core/sock.c b/net/core/sock.c index 5440e67bcfe3..801df091e37a 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3095,7 +3095,7 @@ void __sk_mem_reduce_allocated(struct sock *sk, int amount) if (mem_cgroup_sockets_enabled && sk->sk_memcg) mem_cgroup_uncharge_skmem(sk->sk_memcg, amount); - if (sk_under_memory_pressure(sk) && + if (sk_under_global_memory_pressure(sk) && (sk_memory_allocated(sk) < sk_prot_mem_limits(sk, 0))) sk_leave_memory_pressure(sk); } From patchwork Tue May 23 09:46:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13251999 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D71BC1DDE5 for ; Tue, 23 May 2023 09:47:28 +0000 (UTC) Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0061F139 for ; Tue, 23 May 2023 02:47:24 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-64d30ab1ef2so3422212b3a.2 for ; Tue, 23 May 2023 02:47:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1684835244; x=1687427244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wljfsHXHo/5dnZRd+Tp9MykDSGj/AnuCuvCHil+r0Q0=; b=Wk4AsMej2kFn2vk6ySqFSw0wYlfCctDm1y9dO/NXtgPyjoCFO7fD+mNjbXE5ii9egR fAzkSX11GOa1h9RD0fMJ/iJALgwbS7eEPS2c5FhLye8DHrs0smXM9Bm5IDvCFx7HlS1X gXaDjlgcvO1qiCK8OLndIWQUrFhEv7BLVe7ORpTI1Hzo4qdtIEjNodfDbYJMfQkuY3dL 5R2b/Jrsb3YHOZsEY8BhCksLDo4X6RM8z44pPg2ymI0PMn1CR6jWh2Ox321uXC/Tk7Tu 0HOJyipQKU7yVeh5qZ++ubCoWE2vN2VwaBSEG7tQMchuc1xwsd2NyO1uzg3Zj8tCqh3h qdZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684835244; x=1687427244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wljfsHXHo/5dnZRd+Tp9MykDSGj/AnuCuvCHil+r0Q0=; b=jBDo2zAtqjZcOp5CXfB2Scq5OJfp8fWgUpNYyb0R1LIC+ziTpOo14DcRgcnBLuTP0w 3GcISandPoqzNRkFcUQrda0RWS9/Dg+crd5L8Ng2OLnRXem+1NnMvdzB6SfqbcwHUTMb kNvTMg5ZfPq2tz29vO6Hxr/pI/fRhhyZo/UUjtnRq86QOacLLG9a8HSvKbceTHD0JzFv U7cyKMULt9tAMM4dNG8JAPqXZPTFYDsaU356YJn57jtq/v+3R5qZg30/Dkh30lemCsgb skmecXpFRUeIbxPyWhnAYJ1cvrO3JwG3kLfo2kw5Q/Rdb9V3L04eIoOTOIMWr5rdhXI1 P1ig== X-Gm-Message-State: AC+VfDw1j8IlensNy6vIVJwuVHoWyJ4E9lYXYIlTpJCH86p7epnnx9YM 4YqNZykL/05He4xhwghuzYnQBw== X-Google-Smtp-Source: ACHHUZ70YJnyM6ttX9OZc9Uu5PHLrLyAMVMkoYPM9DzbVinKXHtZToqQGMxLnElUinxlIrQUmcMvOA== X-Received: by 2002:a05:6a00:23c7:b0:63d:2f13:200 with SMTP id g7-20020a056a0023c700b0063d2f130200mr18310565pfc.16.1684835244458; Tue, 23 May 2023 02:47:24 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id 22-20020aa79116000000b0063b898b3502sm5457216pfh.153.2023.05.23.02.47.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 02:47:23 -0700 (PDT) From: Abel Wu To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Abel Wu Subject: [PATCH v3 4/5] sock: Consider memcg pressure when raising sockmem Date: Tue, 23 May 2023 17:46:51 +0800 Message-Id: <20230523094652.49411-5-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230523094652.49411-1-wuyun.abel@bytedance.com> References: <20230523094652.49411-1-wuyun.abel@bytedance.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org For now __sk_mem_raise_allocated() mainly considers global socket memory pressure and allows to raise if no global pressure observed, including the sockets whose memcgs are in pressure, which might result in longer memcg memstall. So take net-memcg's pressure into consideration when allocating socket memory to alleviate long tail latencies. Signed-off-by: Abel Wu --- net/core/sock.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/net/core/sock.c b/net/core/sock.c index 801df091e37a..b899e0b9feda 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2976,22 +2976,31 @@ EXPORT_SYMBOL(sk_wait_data); int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) { bool memcg_charge = mem_cgroup_sockets_enabled && sk->sk_memcg; + bool charged = true, pressured = false; struct proto *prot = sk->sk_prot; - bool charged = true; long allocated; sk_memory_allocated_add(sk, amt); allocated = sk_memory_allocated(sk); - if (memcg_charge && - !(charged = mem_cgroup_charge_skmem(sk->sk_memcg, amt, - gfp_memcg_charge()))) - goto suppress_allocation; + + if (memcg_charge) { + charged = mem_cgroup_charge_skmem(sk->sk_memcg, amt, + gfp_memcg_charge()); + if (!charged) + goto suppress_allocation; + if (mem_cgroup_under_socket_pressure(sk->sk_memcg)) + pressured = true; + } /* Under limit. */ - if (allocated <= sk_prot_mem_limits(sk, 0)) { + if (allocated <= sk_prot_mem_limits(sk, 0)) sk_leave_memory_pressure(sk); + else + pressured = true; + + /* No pressure observed in global/memcg. */ + if (!pressured) return 1; - } /* Under pressure. */ if (allocated > sk_prot_mem_limits(sk, 1)) From patchwork Tue May 23 09:46:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13252000 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B86241DDE5 for ; Tue, 23 May 2023 09:47:30 +0000 (UTC) Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16C50132 for ; Tue, 23 May 2023 02:47:29 -0700 (PDT) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-64d2c865e4eso3772266b3a.0 for ; Tue, 23 May 2023 02:47:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1684835248; x=1687427248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bJNRFp1URkkAv14Fvh/BUBB9PNFK79930mw7UGuUDQY=; b=U4Qt34Z6pN6W4AL25USGqBadP06Vqhmo9JSJfVQCZc70CiJ7M0OKga9craqFE0GnI8 NLMJIANE63+DHnEOMIxX7DO2qCdEbewRVqGS/O5pO5br4bfTrc3uDxLszVxc0OgolxnV aTa3JwnkdYOcbk3XEh+RIbsrLWwrH+lxONrUG5XwpcLq7ppOwFzilXmbro7u+yhFVsMP EN7fQjzet+Sv/Ubmgtxp0XjM+pbLUo9GrvYaHYWEq6DDhhD4nRGmFih1fAPFp9SgP5jG tRmJMdYXzqa4ezGKi45YbtXY94ukxpsPSM9JKp5hSByxj0OnJRYdhlAJFFui19XBhUR6 1qkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684835248; x=1687427248; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bJNRFp1URkkAv14Fvh/BUBB9PNFK79930mw7UGuUDQY=; b=aErr3G9hmRIVHvACaUgid7uxh8d0YtLHNXFfo+2ZN9AeW0a0iH+m+cMgv30Vou0OJ5 dA6GaonitSi5edcptnpOYOU5dySdQgEha6KeZmotecOevYwNZYhGufHoO6Z5idI5BlN7 5ihrisEYkwmvBUWXHHjuOsjhGzKCg6XGeKg1PFWJNCT+wy8F4tDVZIFXwaWMNs6nDpQ4 56PPbdRI75DPJASxWIs0ntiym6uT26GzmBOuLOvdkhB7UqU2eAHgXi5bcoVWkgQirouj 4XgwU5JQvTZ59E9Iy+Js0DUURl4KdPeHGj28cSGhM7K98UTcLDm89GTvms1fQBiAXiw4 L0Qg== X-Gm-Message-State: AC+VfDyBczHCo0QH4Fy7AJhjjBctzy6MqCfcOhJFRZrsHTqW9kNV+gwU pMmf3pKUG6/oZzdgCzYzayHUjw== X-Google-Smtp-Source: ACHHUZ5B4JuSyG1yjWcuVZ88SwoM2AMIukUX5pAMUXBebemZ2uyfglRo+Cw4QPlOctY7/9Er7B2+Rw== X-Received: by 2002:a05:6a20:3d09:b0:10a:cbe6:69f0 with SMTP id y9-20020a056a203d0900b0010acbe669f0mr11156449pzi.10.1684835248601; Tue, 23 May 2023 02:47:28 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id 22-20020aa79116000000b0063b898b3502sm5457216pfh.153.2023.05.23.02.47.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 02:47:28 -0700 (PDT) From: Abel Wu To: "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Simon Horman , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Abel Wu Subject: [PATCH v3 5/5] sock: Remove redundant cond of memcg pressure Date: Tue, 23 May 2023 17:46:52 +0800 Message-Id: <20230523094652.49411-6-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230523094652.49411-1-wuyun.abel@bytedance.com> References: <20230523094652.49411-1-wuyun.abel@bytedance.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org Now with the previous patch, __sk_mem_raise_allocated() considers the memory pressure of both global and the socket's memcg on a func- wide level, making the condition of memcg's pressure in question redundant. Signed-off-by: Abel Wu --- net/core/sock.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/net/core/sock.c b/net/core/sock.c index b899e0b9feda..b2deffb81c86 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3029,9 +3029,15 @@ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) if (sk_has_memory_pressure(sk)) { u64 alloc; - if (!sk_under_memory_pressure(sk)) + if (!sk_under_global_memory_pressure(sk)) return 1; + alloc = sk_sockets_allocated_read_positive(sk); + + /* If under global pressure, allow the sockets that are below + * average memory usage to raise, trying to be fair among all + * the sockets under global constrains. + */ if (sk_prot_mem_limits(sk, 2) > alloc * sk_mem_pages(sk->sk_wmem_queued + atomic_read(&sk->sk_rmem_alloc) +