From patchwork Sun Jan 23 01:35:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Yang X-Patchwork-Id: 12720873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75486C433FE for ; Sun, 23 Jan 2022 01:36:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ADCDA6B0083; Sat, 22 Jan 2022 20:36:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A8B7F6B0085; Sat, 22 Jan 2022 20:36:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92C226B0087; Sat, 22 Jan 2022 20:36:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 83A0F6B0083 for ; Sat, 22 Jan 2022 20:36:26 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3C8218248D52 for ; Sun, 23 Jan 2022 01:36:26 +0000 (UTC) X-FDA: 79059836772.14.55CD9E3 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf05.hostedemail.com (Postfix) with ESMTP id E643E10001F for ; Sun, 23 Jan 2022 01:36:25 +0000 (UTC) Received: by mail-ej1-f50.google.com with SMTP id m4so11147898ejb.9 for ; Sat, 22 Jan 2022 17:36:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VhpcLgiTZRWgcJmPmiVIMS50KRCO7XHLKSYxNUWC3Pk=; b=BwP/7FGLsImZA/ogpCkSY5lZvCJJ8j+FLvn7VvSSKMs/sl3M38tLK6tIh92n7VJgjT FlHwj2kbF2ACOp5FBBg3owK9k3MLpnK7zu6rKQTA8mv3817ztvjVpYLF1V8B4TV057F2 YMdLe0NucazEB/8533um7kVLQvixWyNEyYLM3fX7C3xo1k2xBaNeGQwRtGHkGHUSxPqJ gFBs+s9BTBHqsI3dm+/AEsBoEs0mLUzP0+8Su72My3EdBaJz4TDmWeG/AeaRvHtuqkmj qWvOIByDQFg7bhzawpNeN/ogTQkgmdm0wUlqs3V3jd5AIwq6yf52hU9xqM2yCdji2Bu6 xgpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VhpcLgiTZRWgcJmPmiVIMS50KRCO7XHLKSYxNUWC3Pk=; b=tkjaW58UGIzfV8DUB7u/N5JV7FItKhiRjvbOhStVrk9AtKgvfieNFtdmjzMCjTo4sr emr4vBISuC3xr17bMXcHHzDp1TdJYDEiv6XF6gzdooQCmWJoLKWIbGHgNsiyM4/DuxmZ EWzstsj0fQTaODH6fjqfZNxz7zwJqqOTbvC6F221e66k32+vAI+XPnqlexTNkWcWch3s JgAlbB8/ytEsBfegXyK8Pnu5TQ7ZyaPWb9jpZe0fO+C6o3+V74xtY6Yi0K6qVUgtEI0I uTShYbZimzDm1k8wdFQlTf19NthdNT/bq7fiihOszisGXDhzXSJcbNgPYBdAtsJi0Uw7 BKjw== X-Gm-Message-State: AOAM532BvWXzgDZZaKGMOlVZkArAyFaO2jOVp0THkn8bkQ0f2xthU9Ky EYTX6zRNA+gIaWy4Migt/CE= X-Google-Smtp-Source: ABdhPJzxPNAk450iZZImi9mVv6fGFRHEmD4pFY/kdULLN8Xhv+qFtBziYcfyBJq7RbkriFDGqStJdA== X-Received: by 2002:a17:907:d11:: with SMTP id gn17mr1496723ejc.559.1642901784601; Sat, 22 Jan 2022 17:36:24 -0800 (PST) Received: from localhost ([185.92.221.13]) by smtp.gmail.com with ESMTPSA id y23sm4251117edo.31.2022.01.22.17.36.24 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 22 Jan 2022 17:36:24 -0800 (PST) From: Wei Yang To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Wei Yang , Krupa Ramakrishnan , KAMEZAWA Hiroyuki , Michal Hocko Subject: [PATCH 2/2] mm/page_alloc: add penalty to local_node Date: Sun, 23 Jan 2022 01:35:37 +0000 Message-Id: <20220123013537.20491-2-richard.weiyang@gmail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20220123013537.20491-1-richard.weiyang@gmail.com> References: <20220123013537.20491-1-richard.weiyang@gmail.com> Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="BwP/7FGL"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of richard.weiyang@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=richard.weiyang@gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: E643E10001F X-Stat-Signature: a71uuqtqdzbdd9ou86m6bgoytmzfptt7 X-HE-Tag: 1642901785-903919 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 'Commit 54d032ced983 ("mm/page_alloc: use accumulated load when building node fallback list")' fix a bug on zonelist order. This let me think about what would happen if we have a node system with the following distance matrix. Node 0 1 2 3 4 5 6 7 ---------------------------- 0 10 12 12 12 32 32 32 32 1 12 10 12 12 32 32 32 32 2 12 12 10 12 32 32 32 32 3 12 12 12 10 32 32 32 32 4 32 32 32 32 10 12 12 12 5 32 32 32 32 12 10 12 12 6 32 32 32 32 12 12 10 12 7 32 32 32 32 12 12 12 10 Unfortunately for this case, the node fallback list gets built like this: Node Fallback list --------------------- 0: 0 1 2 3 4 5 6 7 1: 1 0 2 3 5 6 7 4 2: 2 3 0 1 6 7 4 5 3: 3 2 0 1 7 4 5 6 4: 4 5 6 7 0 1 2 3 5: 5 4 6 7 1 2 3 0 6: 6 7 4 5 2 3 0 1 7: 7 6 4 5 3 0 1 2 We found the order in diagonal block is not expected. The reason is we don't penalty local node. After penalty local node, the node fallback list gets built like this: Node Fallback list --------------------- 0: 0 1 2 3 4 5 6 7 1: 1 2 3 0 5 6 7 4 2: 2 3 0 1 6 7 4 5 3: 3 0 1 2 7 4 5 6 4: 4 5 6 7 0 1 2 3 5: 5 6 7 4 1 2 3 0 6: 6 7 4 5 2 3 0 1 7: 7 4 5 6 3 0 1 2 Now the fallback list is in round-robin order. I am not very familiar with the node distance pattern, while I tried the following distance matrix. Both of them works with this change. Node 0 1 2 3 ---------------- 0 10 10 10 10 1 10 10 10 10 2 10 10 10 10 3 10 10 10 10 Node 0 1 2 3 4 5 6 7 ---------------------------- 0 10 10 10 10 32 32 32 32 1 10 10 10 10 32 32 32 32 2 10 10 10 10 32 32 32 32 3 10 10 10 10 32 32 32 32 4 32 32 32 32 10 10 10 10 5 32 32 32 32 10 10 10 10 6 32 32 32 32 10 10 10 10 7 32 32 32 32 10 10 10 10 Signed-off-by: Wei Yang CC: Krupa Ramakrishnan CC: KAMEZAWA Hiroyuki CC: Michal Hocko --- mm/page_alloc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f27afd517652..0cc25429a17e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6260,8 +6260,9 @@ static void build_zonelists(pg_data_t *pgdat) * So adding penalty to the first node in same * distance group to make it round-robin. */ - if (node_distance(local_node, node) != - node_distance(local_node, prev_node)) + if ((node_distance(local_node, node) != + node_distance(local_node, prev_node)) || + node == local_node) node_load[node] += nr_online_nodes; node_order[nr_nodes++] = node;