From patchwork Sun Jul 23 19:09:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hyeonggon Yoo <42.hyeyoo@gmail.com> X-Patchwork-Id: 13323318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FC3DC001DC for ; Sun, 23 Jul 2023 19:09:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 592136B0071; Sun, 23 Jul 2023 15:09:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 51A486B0074; Sun, 23 Jul 2023 15:09:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 394D56B0075; Sun, 23 Jul 2023 15:09:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 20E686B0071 for ; Sun, 23 Jul 2023 15:09:31 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C8F6F8076D for ; Sun, 23 Jul 2023 19:09:30 +0000 (UTC) X-FDA: 81043815300.16.EB2B4BE Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf26.hostedemail.com (Postfix) with ESMTP id 76BA014000D for ; Sun, 23 Jul 2023 19:09:27 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=bMpFAC7E; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690139367; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=7v07lcKVDFmLScNKcZ0mUkKiHYa5lP2jZjbn8NMGSaA=; b=OupkGd2W46AiyvFmxaew5MiZqhgyeZ/zWZ7ycDZvXBNx2wC5oI1g3P9SjTYpEHGxM91WOa XGGOrHpyctbjHGg3mBKffDUcZE0QQRuQp75uOrXVxHyNfHWopeao6ziloh8U2XI9E8D1fB 5fdmJluDH3v2GV9d2NVLL5sRbb95ULY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=bMpFAC7E; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690139367; a=rsa-sha256; cv=none; b=a1QyUr63E0QT3j3H5Y2jdhEkESEh3Pg88T0HCjug21KME6ubRZ0nVbV+PWsv7qfSbMOtPd kb53zfd4woR0wCLZRvE6nKAj6NzdJUN79ysc17YrbhJyQu+gFntHJOtkeNZ+AwINTmNzVL dtmXjS3T22LbDeIYn0Xnis84M/l+4VM= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1bb8e45185bso2991485ad.1 for ; Sun, 23 Jul 2023 12:09:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690139366; x=1690744166; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=7v07lcKVDFmLScNKcZ0mUkKiHYa5lP2jZjbn8NMGSaA=; b=bMpFAC7EebSHAb+kUtyidIURprmXbm80OO72McgKUaOSlQZNSAAEIvY+ACn5NDZcqa 6NJw3+9TmE1BNTQ5l2qxknFwzQJKW03gHbtIDAS1GbUSxpLC3sgstiL2EuOmSp2YNKLu tI/hBQfwQw+DOA7xdYK8fAfuKws1Uy/uPLp1Z7kLlSiDeDK4/dv4wBhuiPz5pH+ei564 AkTxjjlgtMt/w0ZSLfhyQNGhsQZe4laf2wCZIee+hvcDYVONGhZDDAz1Cwo1eKD+n9SH K/x7pjUcYOl/3WDkwCY/nmEAKyBjW9RVsJOvZ5bmd1uzjubJQsWdP/QhZYC1ExMsPn2l LtLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690139366; x=1690744166; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7v07lcKVDFmLScNKcZ0mUkKiHYa5lP2jZjbn8NMGSaA=; b=YNG2NkheyTvjPI4estGIiP63RyCP9WH9l8L9vV9jUvrTwYeFyGTpUdnNhNak4BT23+ 69j/M0MdGge2j73yvR+o4VncZ7bQ/JdlEFBrkBn5I21T7gqQWzQ5JFL0rJ7hkzPcx3M9 P1XAuXVkBvgdb4vyrLs9Mg5izXH6ou9ZHOz5eoTL9gyIn3p7Q9XbycROVFoUpzyZOXz1 QBFowAVccXjwU/k4kTzhBqaLymfEJ1cHJxsqinN6IFny+Xg2/fZ1tkMM8xo0XDFQAorS PsVgbXB4rWyyBY3+qxhnqJncoIct+9N7qS9U9NmPIJxSZ4gnJiNLAYFq241Y6f1AhxnM 5OnQ== X-Gm-Message-State: ABy/qLYbtUr2FFlZYKRni9fOEkwYxmxDy5A9xjLP3ta9KeCBA/wYusHo 0QFH6ejmEK/dZkNb97ZEtRE= X-Google-Smtp-Source: APBJJlE+z1ZxrexXyi9TWVUOJNx3yFc0kYxggupimycaR2uzOUodTj/6xjrQkzfnpB/iSrPQasSGkA== X-Received: by 2002:a17:902:db06:b0:1b8:a2af:fe23 with SMTP id m6-20020a170902db0600b001b8a2affe23mr6244094plx.2.1690139365916; Sun, 23 Jul 2023 12:09:25 -0700 (PDT) Received: from fedora.. ([1.245.179.104]) by smtp.gmail.com with ESMTPSA id s10-20020a170902ea0a00b001b53d3d8f3dsm7168625plg.299.2023.07.23.12.09.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jul 2023 12:09:25 -0700 (PDT) From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Vlastimil Babka , Christoph Lameter , Pekka Enberg , Joonsoo Kim , David Rientjes , Andrew Morton Cc: Roman Gushchin , Feng Tang , "Sang, Oliver" , Jay Patel , Binder Makin , aneesh.kumar@linux.ibm.com, tsahu@linux.ibm.com, piyushs@linux.ibm.com, fengwei.yin@intel.com, ying.huang@intel.com, lkp , "oe-lkp@lists.linux.dev" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: [RFC 0/2] An attempt to improve SLUB on NUMA / under memory pressure Date: Mon, 24 Jul 2023 04:09:04 +0900 Message-ID: <20230723190906.4082646-1-42.hyeyoo@gmail.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: 76BA014000D X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ur5ytxeiptrgyfyqfy1aa1w1i7unst8k X-HE-Tag: 1690139367-78658 X-HE-Meta: U2FsdGVkX1/tyKt44KC7MbFV6d44b5MdWHNMdGWAjQMRbtFrSVQqOCRbK0UBAUtKOY3orqsuENwq/dkT7hWc0KOQCeVUH/hvuc/WNpKIyxLpKswenC3kSneil9I2XC2laRLUhg3J3EphTnfDiP4KJSXS84IwAlw/jn6B7fx3sdZbA/m5SY5AP25NrO0Xvdh95IUCH9KpFmoeOefwDfiK2Gta//i1qjYml94R0nJvks6WgGTBHKQNEgDj/IiAAyutlkOqmmwZy37HdhUE4jra3eYSDEPiN5SCqBHJvp1Z2hs22rWdZysfPHa1oHKPCA9Y1BSU+MTQbpg9KV7bYIu+vXEnTVZOTpfZU0UrMi2WiwY0N0op11ljp4q0umf20KNqvSgj+wCu3MDQMlJARR9TFRpGBZhBr1LcgV8w3md7Vh/spHiBIwqQI/yeBShwB22RqTk7sKWt6rJkOKqFJ3iCIuLuL2if09nGGr1fhf3KJeJqCyAOgh3Q2Wvfjhh6hHOBO0HAD4VODyVO1OHp7eHlNVVp/JyHnJ4bxHb7sczzqHCAXnafPqMgu2AldRUr34R36F63blxrOBzQnRswqv4vRd/NVZHpHppYF2Snqb1kOUuXFKnUJc2DidkqplFxgtRVbbvKiCq3rHPudK91O6WbWyzyHgp4l6vl4kMQsN74+UywUTMhHJ48nG+/y6u/0pSwpfQcgZ7OlJW4KYwX7TvkPAxxMVd4DOkIDS2PFvumsqQpn7hj+ft2+XRbNocV4ddJZ9nYVQf6VeGNm7clV1AxsolaUdC3eDahu7IVnAilPvH++XR3EdiR3SNiBilCtLIspD76zd9lZwxfl7U2dhUMHCMzBN6xIP8uwZF32oUV/VFxUnugwz34vym7YnFdU60JRhO9UccMnQ8vrxeY3jWyRxMwtTiC1/vGxGgE4aE5JDQM5Ss6XvGiOF5Ww/qsPrzd1YFxZ6MZRpfylN4mIFb dMh3RYIf bARx94p7UEILRbX9xQEUlaW99dtcxi1ZHwo9zMU+kLF1x1eZ/Y1JStC6S8BmagPvpR9Hv28yEb9Ux+faMPeO++RUTp2GvK7F/YtYIVwCFfsVf3RmRWTjW6l7AYNI/A3hUYUUnb2DIN043+6fA8kAkd4FeImkIm1PezVdbSNu+ftETKAQHgd2Jp7AumfRm6f9nmmB51i6pu2klKPJ3BA/nHaI9GMfTwT+7xECyBbnwlxaMjk2H2PfXn1KXvEzoygw4Nlo+zJCfypy7JOJ940e4mQlsgxmq5bVEFensf5Hj/B3WVHh6GBbAsjTLoz6RAEafvq/YfhpJHAje9eQ1QGzhtltvyl3R2Hpr3vNFhMXzbf+PMkApRFKihuVqAKX3U7KvHtVgtWg5gXBgr7CTZ6tAVxPzZgZbf5aH9sBgrfUr5PAD1yepQvbqvSPctqcdzW5ZXlhMFZP2QpFWRM2oUWN2T7Vqz/Y0xk0Njm/q4FNIsdEM1Oos9Li6l+k9yLNJrMpMztNukNN7sWTuben8btNgQ0VLFHG615/CTsQUw6AMAXICDwXS45zcK0mi/V5tNd4owMOyG1R4+qKKToHzCv03D9bglFzNIwBw+yHKdq7Yer2Yi4L5QGJys3idJmB2djeZEwCsSQgd1wGSU6+kRtuK7QoNhboUmm+Ow2pARE4r/Ii0YHU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hello folks, This series is motivated by kernel test bot report [1] on Jay's patch that modifies slab order. While the patch was not merged and not in the final form, I think it was a good lesson that changing slab order has more impacts on performance than we expected. While inspecting the report, I found some potential points to improve SLUB. [2] It's _potential_ because it shows no improvements on hackbench. but I believe more realistic workloads would benefit from this. Due to lack of resources and lack of my understanding of *realistic* workloads, I am asking you to help evaluating this together. It only consists of two patches. Patch #1 addresses inaccuracy in SLUB's heuristic, which can negatively affect workloads' performance when large folios are not available from buddy. Patch #2 changes SLUB's behavior when there are no slabs available on the local node's partial slab list, increasing NUMA locality when there are available memory (without reclamation) on the local node from buddy. This is early state, but I think it's a good enough to start discussion. Any feedbacks and ideas are welcome. Thank you in advance! Hyeonggon https://lore.kernel.org/linux-mm/202307172140.3b34825a-oliver.sang@intel.com [1] https://lore.kernel.org/linux-mm/CAB=+i9S6Ykp90+4N1kCE=hiTJTE4wzJDi8k5pBjjO_3sf0aeqg@mail.gmail.com [2] Hyeonggon Yoo (2): Revert "mm, slub: change percpu partial accounting from objects to pages" mm/slub: prefer NUMA locality over slight memory saving on NUMA machines include/linux/slub_def.h | 2 -- mm/slab.h | 6 ++++ mm/slub.c | 76 ++++++++++++++++++++++++++-------------- 3 files changed, 55 insertions(+), 29 deletions(-)