From patchwork Wed Mar 20 02:42:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kaiyang Zhao X-Patchwork-Id: 13597218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB5ACC54E68 for ; Wed, 20 Mar 2024 02:42:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 553BD6B0083; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4DC0E6B0085; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A40C6B0088; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2A2866B0083 for ; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E880BC03F0 for ; Wed, 20 Mar 2024 02:42:23 +0000 (UTC) X-FDA: 81915868566.17.183CE3E Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf11.hostedemail.com (Postfix) with ESMTP id 1D18340006 for ; Wed, 20 Mar 2024 02:42:21 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=CP1JX2zA; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf11.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.160.176 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710902542; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iqOS+o0oT1l/HNw34tsU/JsvphKmYASr0yvHH9hC1Ss=; b=lGQ0s2wA8b9tzRSMFnweaBtEI+t7NOBRIiJ7XzYS+hva2QLHZQpSbNb/txBUObLZ0bF6Rw Eit0VOr4l+8VMwF7jmWdgkQaLuK3NlWnZNRLg1ykcm+TIdBQM724BdeMvHDaT2iaAWmDXf u95EsLsrI5ccYtpW7mOOkwh7t7qGpYM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=CP1JX2zA; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf11.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.160.176 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710902542; a=rsa-sha256; cv=none; b=D4aIUkJznTxdcDXB2dsYseyFr5+Km3ORMrxEjVmSWLKTZ4efdb2w/XU1zLjcYqKX6dF5sF 7WtgPk3qxuFIVuSscA4lyhdAz8yLSBmapzUUoa8Onub6h9mURouLP2J4qy0BmlGQBKkqzb KgV3Ax+x9ibQ0/VSPzTs/GtTb7/mi8w= Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-430c4d0408eso19373191cf.3 for ; Tue, 19 Mar 2024 19:42:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902541; x=1711507341; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=iqOS+o0oT1l/HNw34tsU/JsvphKmYASr0yvHH9hC1Ss=; b=CP1JX2zAPknsAV4pr5aNxni5I0WRVzlyw/BGixtcLU0SX1CUk9XAdt3h4nNsuzbSq6 DaM5gWhshW+OBEFzSxAOM7B5Wn5EvfnFCxWpGrfAfQq1aV8ctuXI5wSB3RbmmjxAcjy0 fc6wax3YvgKqPCkM0xZYPPV6aMku3jd+oTCYYwlBc0k1yT+9gK4KQtNRrIFbz+nSM8QC W4rj7a+wrqU1CFNLoqxpFJC/GhpWmqg2+4SxetWfb62WjsSGBbNHFiu770tzHK3A3vRY 4u8XxeoMVTVeVJKBPQmutwFryZb/2DgyX4sUPII/68zNeB6Kk1tRWKyY91JEOBbbdNNi 7/KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902541; x=1711507341; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=iqOS+o0oT1l/HNw34tsU/JsvphKmYASr0yvHH9hC1Ss=; b=Beh0XRbTGaiCLzrlIT/33uDI0R99bwrOaNr2nx4QVAFIAqBiuHIYkbsHD+kx8UJfh/ C56vOBAo7MplCyApTBAL0VzNERgtuoDVBL7D054PvoPtANjdo5ZqELUG0cBDR1ARfqs2 S949rO8t1hWRyttHoLl02HOD07yuLCLHKGgLuwXYMp9BNIf+TrxE00nttRXZsfk4b0gU 7JnCa/9UJwx+LSJmrXKf2Te7mrRjYhGgBbJix9dG8prbdBYXqqdPYisK8yF0dWSops47 u+jUPjCVDm1JBz14B0fmkid1qe37Ggu03mCe5b4N4tquPZO1Yxt/vKkZ58smfo7Y+AQa nhtA== X-Gm-Message-State: AOJu0YwvjwB20dBucNUc32jdLNLOSTOorfInfbMAOg4mSP4ZdxgkzNdv 6BPvfCp565dX3gJf/SRkNDHrjGxHjiNHT4V8DGTWD6XOr2a4U1b8l2yowDw4C5SkyAITRBKqAfC 36+UBucu7D0ayE/OzR7J3yje/nwTIelnIJ6HyA2vM/+zDCc7qD+7z9JChOfMGVCmazi5WH5xqQs e49WtoGoG/2vEOtmXthuCtmia/oDrTgIcqOQQ= X-Google-Smtp-Source: AGHT+IFk+lAGf3V5CLYbVG5wQXqw/ssb09CFDjYho2uIfDPdBFEW3or4buauRl9+pxp4i58SExaXDQ== X-Received: by 2002:a05:622a:1896:b0:430:d8c9:b523 with SMTP id v22-20020a05622a189600b00430d8c9b523mr5159062qtc.8.1710902540913; Tue, 19 Mar 2024 19:42:20 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id hb4-20020a05622a2b4400b0042f0008b8b7sm4428856qtb.35.2024.03.19.19.42.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:20 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 1/7] sysfs interface for the boundary of movable zone Date: Wed, 20 Mar 2024 02:42:12 +0000 Message-Id: <20240320024218.203491-2-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: a89bor73ym3s49p1mt6bjzbmj5rhtbw6 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1D18340006 X-HE-Tag: 1710902541-569096 X-HE-Meta: U2FsdGVkX19xavU5XtCeW6mb5jFn2tSNk4CHHNHY8NlXEa50XmxVi+bPGQr4QaJS9aQRPtLOnuk0Z6JckReDD+m37ObC2IABORcANGcqs7Oz/gtZcBhIH9DyCOd/PU4lCyaLbX7c1i+xDxNlubFAwKS48e25MKpAKI8krLCj7UVIT+XtxGMQCu4nY5zdjXOZd+I5PjW3hoX7+g2ALPJoOv40bO8W9edN2sRLfv/bEF4FvnNJt5ddJtgm+tddFZjFxpuZoWQWL0M97e+Eta/9oTStiFmlV/ZXaBgzs1+YAh9SOZKIT9fOkHjfUEfn5ZIfGhyGsQDLZg0wMU6AT9focDKkWAgvhm3tYe7Ng/kKCjsU0j5Xsn/mBDExH2YSqPuPBfoGChkiMz5ic/FKVr6OJmCWLNy03Azd1tE4Ds2HUmIQO5TEYpkTGamxMKw2OqW6Xh5RnUrPylHO+qqcjtZmJtmHkjfTJ4kkM64+jnLmYvisodOGrDI9EeRV/239REgXomHEXS/UAct2kXjTXTTyUPxX5JVUUIDvRfzshu3z6wH4l8W4wiUGL/hP8cciExpJxOwoNx3Wk1LZrtgimMoVxlmnlVH2BMeUMC2X/T+JifZTo/nlSYPJ1IquYaIhpmST4XWei66ucKNhGqM55eUcXi5AChTKwaFqG/7EkphFRjEcQoKOPAakNH2TFjl1DlZRFJoqV22ekLi6/6bu3fNuRP7F9yAkoD2LhtQBZDth7xIb994BwLizqH/24FlKAuVZ0p+NunMSgEqYFZF0GF5EVooTw8AAXEK6Hkbo8pZg5jhj5FCrA8ZRHzxyOrvfGy4gs4YWb/cybrELcdLZOoR2w3wbQEv64DFVU5AUhHzwGO9g25MYnY2XQOR3BeHkXA0wFnbTummMzaMrLEJ1Aw3ItDEBkyOpEgMA/bnB6x15KcAmhloglgAxbHwoYzX2ivdgcMJScImGYi2u45gOpta 9pKXkt/X vaRUp9aOg9sjK4Ndid8sDvyaXz7gHvOAEMXLh1XBPV4znUV1ycPbARtLCL2J3GCZH8FUkDbY2ImTCoBH+eaH+OiLttFW5/3lvtzqEGyDAOauWJIXkMwuC3n8WT8DQ2K4094+pxskZB0la3pTbj93plJYFirCC/SDwriXh78XdpFcd/hQLiL8Xa0hu9mZIm+xjeEyS3iCrJjMrYBaQtYqYuWTZeV7HM9ZhJosQ1Pby2L/8GbGMmy1AVvvH7FSC3MyZM/wrPMhE4AxxMEgfFgAdP/mXcDL/fz1yDKIui+S4CntsSEKYMOXuprTlEGXMnN/0YOdn+bjqkveiL8RMGLzDlK8WFk43+v+Dqlau6ZNxihU67O9eR+tHeO7iUGTWv7HB/eruQ4hzI3jg5ftlnnE8EbGqa7efUTBJaAvA0v3S0fuyjZHZwaGVl2D0CnguCkvoTf1CbcU19RsGNxxdNTAiAQX+Sg10byWEowFjBvpOlc6Md05sxLNey9Ux/yHqTgdeVQP7DCzZvvxTj6exRrAW3uwrO50KTmDfPscObV39Wx3i8l/iRCX6cWDlDwAwzpDYNa3C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kaiyang Zhao Exports the pfn and memory block id for boundary Signed-off-by: Kaiyang Zhao --- drivers/base/memory.c | 2 +- drivers/base/node.c | 32 ++++++++++++++++++++++++++++++++ include/linux/memory.h | 1 + 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/base/memory.c b/drivers/base/memory.c index b456ac213610..281b229d7019 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -55,7 +55,7 @@ static inline unsigned long memory_block_id(unsigned long section_nr) return section_nr / sections_per_block; } -static inline unsigned long pfn_to_block_id(unsigned long pfn) +unsigned long pfn_to_block_id(unsigned long pfn) { return memory_block_id(pfn_to_section_nr(pfn)); } diff --git a/drivers/base/node.c b/drivers/base/node.c index b46db17124f3..f29ce07565ba 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -486,6 +486,37 @@ static ssize_t node_read_meminfo(struct device *dev, #undef K static DEVICE_ATTR(meminfo, 0444, node_read_meminfo, NULL); +static ssize_t node_read_movable_zone(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int len = 0; + struct zone *unmovable_zone; + unsigned long movable_start_pfn, unmovable_end_pfn; + unsigned long movable_start_block_id, unmovable_end_block_id; + + movable_start_pfn = NODE_DATA(dev->id)->node_zones[ZONE_MOVABLE].zone_start_pfn; + movable_start_block_id = pfn_to_block_id(movable_start_pfn); + + if (populated_zone(&(NODE_DATA(dev->id)->node_zones[ZONE_NORMAL]))) + unmovable_zone = &(NODE_DATA(dev->id)->node_zones[ZONE_NORMAL]); + else + unmovable_zone = &(NODE_DATA(dev->id)->node_zones[ZONE_DMA32]); + + unmovable_end_pfn = zone_end_pfn(unmovable_zone); + unmovable_end_block_id = pfn_to_block_id(unmovable_end_pfn); + + len = sysfs_emit_at(buf, len, + "movable_zone_start_pfn %lu\n" + "movable_zone_start_block_id %lu\n" + "unmovable_zone_end_pfn %lu\n" + "unmovable_zone_end_block_id %lu\n", + movable_start_pfn, movable_start_block_id, + unmovable_end_pfn, unmovable_end_block_id); + + return len; +} +static DEVICE_ATTR(movable_zone, 0444, node_read_movable_zone, NULL); + static ssize_t node_read_numastat(struct device *dev, struct device_attribute *attr, char *buf) { @@ -565,6 +596,7 @@ static DEVICE_ATTR(distance, 0444, node_read_distance, NULL); static struct attribute *node_dev_attrs[] = { &dev_attr_meminfo.attr, + &dev_attr_movable_zone.attr, &dev_attr_numastat.attr, &dev_attr_distance.attr, &dev_attr_vmstat.attr, diff --git a/include/linux/memory.h b/include/linux/memory.h index 31343566c221..17a92a5c1ae5 100644 --- a/include/linux/memory.h +++ b/include/linux/memory.h @@ -92,6 +92,7 @@ struct memory_block { int arch_get_memory_phys_device(unsigned long start_pfn); unsigned long memory_block_size_bytes(void); int set_memory_block_size_order(unsigned int order); +unsigned long pfn_to_block_id(unsigned long pfn); /* These states are exposed to userspace as text strings in sysfs */ #define MEM_ONLINE (1<<0) /* exposed to userspace */ From patchwork Wed Mar 20 02:42:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kaiyang Zhao X-Patchwork-Id: 13597220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14A99CD11BF for ; Wed, 20 Mar 2024 02:42:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D697D6B0088; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D19986B0089; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C07D96B008A; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AED336B0089 for ; Tue, 19 Mar 2024 22:42:24 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8C96514037B for ; Wed, 20 Mar 2024 02:42:24 +0000 (UTC) X-FDA: 81915868608.14.3F62351 Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) by imf15.hostedemail.com (Postfix) with ESMTP id B7020A0005 for ; Wed, 20 Mar 2024 02:42:22 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=O0HM53fb; spf=pass (imf15.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.222.174 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu; dmarc=pass (policy=none) header.from=cs.cmu.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710902542; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3nx6/aDk3VJUuN8aHF02HR1BsWsxIjtwUpot61xzcjI=; b=J7pkd4PI2IBAhwGxvBHSHmbQTAWMBOE855nqWCrx6B3x6QcDCDKKKxgG1OLZB9AyUZdFwm A9SLesR0G4cf79rv2hK3S1oHQXufWgDdkoJt89rbpT9t04tBwYG9TzUUKtEuno87Zq0AXG JyZl2wcfWzFfn6oyC5o+9PErCotWwfc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710902542; a=rsa-sha256; cv=none; b=Pl1qIWIwlykIdS0K0UccvacbfbGFRnCMn9TMXRS8D6w1+YV9QjAV645fc22xkawnTxUrZI 4+uQ9q+MRsHDcKdKPGZrW1WXzRmOxlVsGdKcHE403p04G77Awn9VZlrrM243yTAGZapO6p qaguZvhcc5itQrpe2YUXHEANRQRbjL0= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=O0HM53fb; spf=pass (imf15.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.222.174 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu; dmarc=pass (policy=none) header.from=cs.cmu.edu Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-789e6f7f748so271739585a.3 for ; Tue, 19 Mar 2024 19:42:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902542; x=1711507342; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=3nx6/aDk3VJUuN8aHF02HR1BsWsxIjtwUpot61xzcjI=; b=O0HM53fbYudJxtyv5dDsOzG7I7aECXRFfWFFvys4dZf2F67oPsaxhTuxR5j0aAZ1gZ qq3rrKXKZDLssodzOwsSSh73H5jgYrbyE4BNQdMazoEjvTXj2qG6E6/Ze28Qf1nIw1ho r4eEIDEHmIWyOHESV/JKBn39E7pZ++BTNhYE/BQLzCnERn5f3OxUJEr5l0sBmskfxBp8 8coDtIi3QTkMTQW/apSSnGLDB2GAblo05QOP3krtQMAa65dNN5VnSfZz7HhpfK5GLWP+ Sj96k4mR+e9ZMZ5MzpxWh8DbjDGkhB7cyHmvjsvjuFy7b4zICWbOp6W2VfsHH4z/ptYp r6Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902542; x=1711507342; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=3nx6/aDk3VJUuN8aHF02HR1BsWsxIjtwUpot61xzcjI=; b=Z/LoxGUjIau3LUQs5nyyBnzcQSmI09ol9fdfT2jcfk1obTVfdM3oUXaWm5hlYS6UWK CbmOh6X03CtfXMYOCRsIoxRkQ5G43YWrkv1+qpsw8vA1t4tTrSqvAWnAQct/dE6NkXjO 7PJCXjmqYGm0jdK8gszcsnKoPxVBNJwbOslzkDJ1/wEHKkn24zg25VTqgcU0m6k0bVVm uaDTpt4TrObTIHQ0JvPGDgvv7rPpRVdx7pfGjNncGlNMld9hzrtP1vbsVMIGnaGknoef p3TV3FbfY1KGj15gM9+JxX7gnpZvIljFfYdQloBS0Jvgn2Zy3FUiZcIAuSlxealJY9bn hIPQ== X-Gm-Message-State: AOJu0YwUE3dxCwRueNE0DfZRu01MZt3R8teQK2t5QF3JWGAw2QjeNCV4 HRp5B/xowyqkboj9o8hCRl9k/oOqt5JH6j8SCBSTamPg1CAT/w6cvzTLWbpRGlYxq1UOfm00A1N GrF8H0ABuoOQEkGji6t5eLq9UiGt5OluWwlvET6V+EfOrJ5fWOi0YCxwbckI4FrubOsRPtQtBvB u82p0fMV7NBr2Pb4ga+dFgHSUEHBivZuPuDbU= X-Google-Smtp-Source: AGHT+IE8ey/Y7RHs4C66DCwZWoviXuK1UpbUhJUPjIEBgTvetqJ0Lw+B0vwcdezjuc587wrWuN/E9w== X-Received: by 2002:a05:620a:4484:b0:788:30a5:6aaf with SMTP id x4-20020a05620a448400b0078830a56aafmr22805144qkp.52.1710902541638; Tue, 19 Mar 2024 19:42:21 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id vr17-20020a05620a55b100b00789ea5b08bcsm4074525qkn.23.2024.03.19.19.42.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:21 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 2/7] Disallows high-order movable allocations in other zones if ZONE_MOVABLE is populated Date: Wed, 20 Mar 2024 02:42:13 +0000 Message-Id: <20240320024218.203491-3-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao MIME-Version: 1.0 X-Rspamd-Queue-Id: B7020A0005 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: gjt31fsr1mt7k5xrtswob3ozdqyianp1 X-HE-Tag: 1710902542-122328 X-HE-Meta: U2FsdGVkX1+kvhHHEYh5wrzhOc1YRnuvr2YyHkYj3ntwXW5Ni/kDXoSMTSSqHE/wrFQQKF7eKREpjmCUWrHnW2PTWSPfjwdu3OOlZLl/YkhKSmwqalJJprSS3qWLKbwL6ZFlaOgMUh14jILqB/7X3L7eBvcTtJSean4Obm1NkA2spdN6khgKgq61sNPSeIuZZyB1DNzTO8OldawJL4YaKBOqug2n8/d2vWFL5DuUyugwOLN/9rcuKFXuFL9wmLP9ihBNVEYjD+XX/q7J+654oWtzyJArZk7v4ngZWJbtJP214CGObStXDHZwWclsB9axbxFYqdnzSFs6+kzzGyhnNxP2+FMMpZIHu4tmcRjswzQL4fDVWWL/bMoQ2wb3pJIlMSnDNHDxR5liFsIBN8KHuyYLwjUDPGLX0OIj5Kl1d1RU4EEUk6ew0MDFV24mypJz3zHq+U+AXXtATDzt0OgZlb5fiIUDcBBO+h3DdmCPvZQ1KBymHM97UE8aN7mdeJQBIPz+mRo/xI/A2O6zzFBP8nx2EU6Cok9+sT3Pk8bynA/HAzgSCQNcHZDDhrbdbP9kWffwdDOh8c3txwgtT/0a/A6Ut2HKYDtvHAQNSNvPF7U6Bec6RVYP2hAgJL3gRfxfP1XK0WDzUv5KqRBJ4Eby3NRdCZI/sx+5XNYH4t1He2alqY2GinIntD5er8dlAvu0M1F9Zv8/8TqIY3XTsVNp8RljRh2g2KL6eGfd9Dhp647FvW+aohIx1benyeiWwb+5FnII6TV1GMfTZ3vZC9m7YtxuovtujJFfnk+5wp+7hds56t6r5UrZCuKq1u+Ofa/eJcKpOhYklhhFTgZUyJOQiDYU5GKirN8yyxWfY75tWoVOJRWRkPtRzxVKmSPbBTzgsVeQUeXDe4RB33waQ2LCDxF+dc3tCS4+Rzj2VaSSLN+j+Yb4jn9zqyuK4KzgyPdhuGG9Td7SntX9cM3ueYm B1IJDQfP LGNLcA5CvzTafbkMlgmrTNMWFIsfViFMtTUN2oL6PI/ktTH/Bsv/C0J/aNFj8n2o3+IQdOCuhUFZ6iPVrFmaOIkLz1CEPER5ynAT0FbL+jqLozuQmQ2fX+bTw624NTSW/U8EN3PUcOmj73fdXB0Uq1Hg5WbGMgfF+udjXPZ84wYxIevFYgNc3whAfaCMkZ3W40pYBEgChlWZ75JrVkCtw7zwaaCjEdrdVIUQoHLNyPQkfM3G7ybg1JMgf9WtLKNhs9ezohtYg1nGno958akwAHbibAxgAjlwmnChqwmGribztSUfduOhttifsUNkDnuyGbIlkYxaawmS8m6c0+stkGX9qIX+lS1Lla8eva9Nvs4MOPSxJzrDIJojuonYY2OVxx6LbdzJviJqoRKdv02qoSsKwIpZohe4Bs8FKdAJ7p2k3QSncGZ3+4MQAOMGGHhKpcmE4aKnRHVjGl1XqJxRdjc6/zhRDoioOP0JtV5GmEH8iJZZV0r9b2jovVst/bDt26hbuy3fztB0CIVu8zcNJD7GnqwSBsqyxS/4sl/EVDPWOiiWGQsPbk+gCl2Q8gzfLdZ9GbHi2j8p6ZOkK4d941ZfvIsc+7tJwEwDOSDPrpcSKB2U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.383383, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kaiyang Zhao Use ZONE_MOVABLE exclusively for non-0 order allocations Signed-off-by: Kaiyang Zhao --- mm/page_alloc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 47421bedc12b..9ad9357e340a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3403,6 +3403,16 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags, struct page *page; unsigned long mark; + /* + * Disallows high-order movable allocations in other zones if + * ZONE_MOVABLE is populated on this node. + */ + if (ac->highest_zoneidx >= ZONE_MOVABLE && + order > 0 && + zone_idx(zone) != ZONE_MOVABLE && + populated_zone(&(zone->zone_pgdat->node_zones[ZONE_MOVABLE]))) + continue; + if (cpusets_enabled() && (alloc_flags & ALLOC_CPUSET) && !__cpuset_zone_allowed(zone, gfp_mask)) From patchwork Wed Mar 20 02:42:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kaiyang Zhao X-Patchwork-Id: 13597221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 685FAC54E71 for ; Wed, 20 Mar 2024 02:42:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B7B96B0089; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4690D6B008A; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 32EE16B008C; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 22F966B0089 for ; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A2BCA1A087E for ; Wed, 20 Mar 2024 02:42:25 +0000 (UTC) X-FDA: 81915868650.12.F7B2958 Received: from mail-ot1-f53.google.com (mail-ot1-f53.google.com [209.85.210.53]) by imf24.hostedemail.com (Postfix) with ESMTP id CDECF18000C for ; Wed, 20 Mar 2024 02:42:23 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=hLelrzc+; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf24.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.210.53 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710902543; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UljsMw4YXxvG2im1vY4a6XAU2ZaTJQdUEua3lUBpIog=; b=IL45OSXHI1F6u+iwRRMn8p67+Dg0K5/+SMTdLsJpAcj6c4lRyKDCJoyq/r4a0/Z3D+uFZB MAQvvIF+KGdgOgcTB7WfetkGdmjvIxepouYcWuXRXjP9GqdhtZMvYKtAahdVUs+pgwJNTT tsgMz/tZ4bdIO0bo9OMVXhg+HCrOVhU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=hLelrzc+; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf24.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.210.53 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710902543; a=rsa-sha256; cv=none; b=k+NUnBFdCWOKPofEbIqHFhyiEiWW3/+7z5JtRhzSlB3P50/67HUzg9JRBJ8vWL98wjlwAH +i01y+OAw3niT/rB6zO7cUeUG6akYZPHtKzD+qkUbv0aq9PoAIv3FR8P61JHeuC3tPJfBC 1SKJ9d3FsBYsmwIiBUiARNw09la0CGU= Received: by mail-ot1-f53.google.com with SMTP id 46e09a7af769-6e674136ba4so3980893a34.2 for ; Tue, 19 Mar 2024 19:42:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902542; x=1711507342; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=UljsMw4YXxvG2im1vY4a6XAU2ZaTJQdUEua3lUBpIog=; b=hLelrzc+vRZzC6564Kt/GIDw9tX8SKOAlvo7HWM8vFHgIlX41Q5901vuo7MKpku+2r vKJTESaRXrWYFcDFZsUqPLX118iMgqBGMkAzN3gIDvPlUJAwJXJ3rRDxs7TCUad7tkJz +Bb5n79MUbu9KNyaSfYpyxqadfzJdUuMb7sy/jVWKI7SffeNPM5CFUpovS28b600xUXc FsJ9rk1AbenvN4CSCw7SuFT1zNBpkC0gKPIeQnMBTkbgNs0Z71EEIZITx92u3oNXdcWP W9s9/pJ9Cd6BP/zp8mZAbDnoov9ThoBVqmQ8gJHPR5ZsgWKDnXgs+7JFav9tJ1kFVtNa LIUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902542; x=1711507342; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=UljsMw4YXxvG2im1vY4a6XAU2ZaTJQdUEua3lUBpIog=; b=XTsXyBUQ1xLGQueW33iHj85xOIXS7BmDv5B5p5cza04CLJpha5W8JxB9bzalHfPwxD hJMg8ZUrqcYCYarkLBhMRQGiCFKLuC+jWPjvw7nlqRVTiGHsz6uAKTO5zeIzBnSHoCde IznqcSE9NShsOAFg0zMT6shIJxhQof2WnLd5Js9X/dquTJzwr70kV+ETPnraam/dvbIJ v1rYlnyDomMGPppxpSOy55tEcwarKSxPywUPGzLdaJGZa+o+SXFhaozSW11EG+F3c6+4 k+EVRkgQDQoLwT8sfqjKK5iYrcEPL1nAPnl3cHQhpsbUm2IHwOVeURrV31UvVFGRsK3t Cytg== X-Gm-Message-State: AOJu0YxXr51IAiRp24qBsvdM35nmteDdXy4T66OjO9YkFN+5P47Ww7Vt 0/BF96+l5mZEVqbVDu14ybS2swaIQyc45iW/DMD90qwu8pR5XNx/YbNMU8EsuoHe9+UmHqaWPjb 05wMDZLF+ZvxzzzxLypNS8gRcECb2PCykYfuhH1atnQYf4y+jbUvxAgHZUJurh5F2tlg3SFwhbZ jxARWhjDTuXpm4EmSvVNXuz5wgLCUvd9n5m3Q= X-Google-Smtp-Source: AGHT+IESptNQKDLD8MOe2hQxpKnyZd4H1raJLsnnYtjWGvQGBbhnaHWmiZ6IFtzpyT4LI3UuB+xfsg== X-Received: by 2002:a9d:74c5:0:b0:6e4:f961:e444 with SMTP id a5-20020a9d74c5000000b006e4f961e444mr5504940otl.4.1710902542503; Tue, 19 Mar 2024 19:42:22 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id y27-20020a05620a09db00b00789f0d9e6dcsm3560797qky.93.2024.03.19.19.42.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:22 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 3/7] compaction accepts a destination zone Date: Wed, 20 Mar 2024 02:42:14 +0000 Message-Id: <20240320024218.203491-4-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao MIME-Version: 1.0 X-Rspamd-Queue-Id: CDECF18000C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: d89wcmfpedi7c3uof9sho6c9dmrnpg6e X-HE-Tag: 1710902543-463704 X-HE-Meta: U2FsdGVkX1//oyUnEVtOSCEHMdbuqNqW71+j9QMv77oU8frk2pAJ3yWKHbFiPpI/cq+BzfPgD9NF6zSRE0vinQIGqDDG94JKxfaGZGKjkfEFcllJSRXwdVBTorkZ0Jc5cSFWXp6sv9sCyq40nCm1K0Nl+p3G41QzG6mrqZBmEByN+2Bb41zS7EQhASo6iuN+fyJNNXe54Ny95FRbdvUDn85ZWV2W2WGkSSpbXcO+qLi4yoyax5tt62LLyfXohhX8Azz2ty2oTy3ACUwiLorpei1OzkcE8493tQoUPFcwfNNrfN6h2i2gR5ajSgRSZLM1P0lh8URcftfGnb3L/RTf4zCNYxmAMrLMIBTFTAKHOzv5yB5Oi4cCttXNk69jzIeE+d4RSXh2khO+yHoxgPHa50VpBD+fRIyG4mvz67htc9GMydflKU/nEo6E67Ry+/GBQ5HVfLozVKbMDWvMVQ1SlEYWcfHyQiSl+HEGUxOLeZlQbTMzFKG7+OoNpViB8TPG6Vsj6ffpkWUp63w9gDs1TOEZU1eodO9M4o/o+PRy3pdecAxzMS7OsEZlAypV5wDfRdU2J2Uv6tpLtGe0kZC2MFYJy/KYL3UjTfCY5dLVbop5aY4pne/awCzCN3ToQcBYFGMT3MXEp4QSPrl02i1z6oXGYoEyay6q266e50cKbolbUsLrkbaFaVdXeCJyzwdsihVyZOdmwfyaL01RWXvKbgZFkmYhp6vfg4+6aZ6jM55/y+mZJs2/cnpl29DajXnu0OyZbG13Xcwn8vhH1Z9tZ2W9EtnqDB/jZUWIgXWuIRatrhbCm96ZyZ8wkpjwecr8N8WkDHXNie9l/Pwn9e7hi4xXQrHIsRx1cHQmx4rsw2u1FArgMb8t8BQk7eIUwAxbmDfaJTwDMypFy3NQlhttXnoBFCPMCMKJ5QTbVC/znTV5vkzWEo580Clzlymf3usW5En/zU519GB2lhGffh1 RJmIQlo/ 3lDNCRVW7j3fXCMyZKM+27GQutIu0iWOEu5l2ZJSRVrMVJ61ASERKtIn0Xacktwl30gJDoZM4jLVxGyNA6D4GIYsI8McFe41Geonx64kuwPGTHIspmcvAXCaskjB5RDrZ2mPLbxBTDlReEyO7AFniIdqa0m3t6lAYqmXOjlxMlTj/pykob49d5fpiBprqi70oh9bbXUu2paTsg/TvmnVILJd4tuTZXjknj/dT6j68w77KxJELbclV0qZwruejgk2bvRJevLb9uPBcerps9/X10kxtDkgfWf2YmtL/ZfY98ShCRqSVq3K2MUTY9+LXbxbRm0xxbOvfctwut1glWHyEZg0YG8kBHUgESxI/dSvsVXva7KwkqFF5hFDxLf+ttzRd5EPpCK18AJyVMbQGAlAe39Imri8JN334Da8Jb/7InPWcNsTyMgO0xLI8cTdMNB1vv34R+wOl/LDUx+YRvipaEyxjtC3oJsecXz3afhcvR6IeFaJ+XqNKqu57QB9iC9vQN2hYKR/tzHetQ38XH1wuOi38YQnnHgNdJlqDO1NIjSrvmegdLY4Zk5WqFePM6tUgYVCG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kaiyang Zhao Distinguishes the source and destination zones in compaction Signed-off-by: Kaiyang Zhao --- include/linux/compaction.h | 4 +- mm/compaction.c | 106 +++++++++++++++++++++++-------------- mm/internal.h | 1 + mm/vmscan.c | 4 +- 4 files changed, 70 insertions(+), 45 deletions(-) diff --git a/include/linux/compaction.h b/include/linux/compaction.h index a6e512cfb670..11f5a1a83abb 100644 --- a/include/linux/compaction.h +++ b/include/linux/compaction.h @@ -90,7 +90,7 @@ extern enum compact_result try_to_compact_pages(gfp_t gfp_mask, struct page **page); extern void reset_isolation_suitable(pg_data_t *pgdat); extern enum compact_result compaction_suitable(struct zone *zone, int order, - unsigned int alloc_flags, int highest_zoneidx); + unsigned int alloc_flags, int highest_zoneidx, struct zone *dst_zone); extern void compaction_defer_reset(struct zone *zone, int order, bool alloc_success); @@ -180,7 +180,7 @@ static inline void reset_isolation_suitable(pg_data_t *pgdat) } static inline enum compact_result compaction_suitable(struct zone *zone, int order, - int alloc_flags, int highest_zoneidx) + int alloc_flags, int highest_zoneidx, struct zone *dst_zone) { return COMPACT_SKIPPED; } diff --git a/mm/compaction.c b/mm/compaction.c index c8bcdea15f5f..03b5c4debc17 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -435,7 +435,7 @@ static void update_cached_migrate(struct compact_control *cc, unsigned long pfn) static void update_pageblock_skip(struct compact_control *cc, struct page *page, unsigned long pfn) { - struct zone *zone = cc->zone; + struct zone *dst_zone = cc->dst_zone ? cc->dst_zone : cc->zone; if (cc->no_set_skip_hint) return; @@ -446,8 +446,8 @@ static void update_pageblock_skip(struct compact_control *cc, set_pageblock_skip(page); /* Update where async and sync compaction should restart */ - if (pfn < zone->compact_cached_free_pfn) - zone->compact_cached_free_pfn = pfn; + if (pfn < dst_zone->compact_cached_free_pfn) + dst_zone->compact_cached_free_pfn = pfn; } #else static inline bool isolation_suitable(struct compact_control *cc, @@ -550,6 +550,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, bool locked = false; unsigned long blockpfn = *start_pfn; unsigned int order; + struct zone *dst_zone = cc->dst_zone ? cc->dst_zone : cc->zone; /* Strict mode is for isolation, speed is secondary */ if (strict) @@ -568,7 +569,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, * pending. */ if (!(blockpfn % COMPACT_CLUSTER_MAX) - && compact_unlock_should_abort(&cc->zone->lock, flags, + && compact_unlock_should_abort(&dst_zone->lock, flags, &locked, cc)) break; @@ -596,7 +597,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, /* If we already hold the lock, we can skip some rechecking. */ if (!locked) { - locked = compact_lock_irqsave(&cc->zone->lock, + locked = compact_lock_irqsave(&dst_zone->lock, &flags, cc); /* Recheck this is a buddy page under lock */ @@ -634,7 +635,7 @@ static unsigned long isolate_freepages_block(struct compact_control *cc, } if (locked) - spin_unlock_irqrestore(&cc->zone->lock, flags); + spin_unlock_irqrestore(&dst_zone->lock, flags); /* * There is a tiny chance that we have read bogus compound_order(), @@ -683,11 +684,12 @@ isolate_freepages_range(struct compact_control *cc, { unsigned long isolated, pfn, block_start_pfn, block_end_pfn; LIST_HEAD(freelist); + struct zone *dst_zone = cc->dst_zone ? cc->dst_zone : cc->zone; pfn = start_pfn; block_start_pfn = pageblock_start_pfn(pfn); - if (block_start_pfn < cc->zone->zone_start_pfn) - block_start_pfn = cc->zone->zone_start_pfn; + if (block_start_pfn < dst_zone->zone_start_pfn) + block_start_pfn = dst_zone->zone_start_pfn; block_end_pfn = pageblock_end_pfn(pfn); for (; pfn < end_pfn; pfn += isolated, @@ -710,7 +712,7 @@ isolate_freepages_range(struct compact_control *cc, } if (!pageblock_pfn_to_page(block_start_pfn, - block_end_pfn, cc->zone)) + block_end_pfn, dst_zone)) break; isolated = isolate_freepages_block(cc, &isolate_start_pfn, @@ -1359,6 +1361,7 @@ fast_isolate_around(struct compact_control *cc, unsigned long pfn) { unsigned long start_pfn, end_pfn; struct page *page; + struct zone *dst_zone = cc->dst_zone ? cc->dst_zone : cc->zone; /* Do not search around if there are enough pages already */ if (cc->nr_freepages >= cc->nr_migratepages) @@ -1369,10 +1372,10 @@ fast_isolate_around(struct compact_control *cc, unsigned long pfn) return; /* Pageblock boundaries */ - start_pfn = max(pageblock_start_pfn(pfn), cc->zone->zone_start_pfn); - end_pfn = min(pageblock_end_pfn(pfn), zone_end_pfn(cc->zone)); + start_pfn = max(pageblock_start_pfn(pfn), dst_zone->zone_start_pfn); + end_pfn = min(pageblock_end_pfn(pfn), zone_end_pfn(dst_zone)); - page = pageblock_pfn_to_page(start_pfn, end_pfn, cc->zone); + page = pageblock_pfn_to_page(start_pfn, end_pfn, dst_zone); if (!page) return; @@ -1414,6 +1417,7 @@ fast_isolate_freepages(struct compact_control *cc) struct page *page = NULL; bool scan_start = false; int order; + struct zone *dst_zone = cc->dst_zone ? cc->dst_zone : cc->zone; /* Full compaction passes in a negative order */ if (cc->order <= 0) @@ -1423,7 +1427,7 @@ fast_isolate_freepages(struct compact_control *cc) * If starting the scan, use a deeper search and use the highest * PFN found if a suitable one is not found. */ - if (cc->free_pfn >= cc->zone->compact_init_free_pfn) { + if (cc->free_pfn >= dst_zone->compact_init_free_pfn) { limit = pageblock_nr_pages >> 1; scan_start = true; } @@ -1448,7 +1452,7 @@ fast_isolate_freepages(struct compact_control *cc) for (order = cc->search_order; !page && order >= 0; order = next_search_order(cc, order)) { - struct free_area *area = &cc->zone->free_area[order]; + struct free_area *area = &dst_zone->free_area[order]; struct list_head *freelist; struct page *freepage; unsigned long flags; @@ -1458,7 +1462,7 @@ fast_isolate_freepages(struct compact_control *cc) if (!area->nr_free) continue; - spin_lock_irqsave(&cc->zone->lock, flags); + spin_lock_irqsave(&dst_zone->lock, flags); freelist = &area->free_list[MIGRATE_MOVABLE]; list_for_each_entry_reverse(freepage, freelist, lru) { unsigned long pfn; @@ -1469,7 +1473,7 @@ fast_isolate_freepages(struct compact_control *cc) if (pfn >= highest) highest = max(pageblock_start_pfn(pfn), - cc->zone->zone_start_pfn); + dst_zone->zone_start_pfn); if (pfn >= low_pfn) { cc->fast_search_fail = 0; @@ -1516,7 +1520,7 @@ fast_isolate_freepages(struct compact_control *cc) } } - spin_unlock_irqrestore(&cc->zone->lock, flags); + spin_unlock_irqrestore(&dst_zone->lock, flags); /* * Smaller scan on next order so the total scan is related @@ -1541,17 +1545,17 @@ fast_isolate_freepages(struct compact_control *cc) if (cc->direct_compaction && pfn_valid(min_pfn)) { page = pageblock_pfn_to_page(min_pfn, min(pageblock_end_pfn(min_pfn), - zone_end_pfn(cc->zone)), - cc->zone); + zone_end_pfn(dst_zone)), + dst_zone); cc->free_pfn = min_pfn; } } } } - if (highest && highest >= cc->zone->compact_cached_free_pfn) { + if (highest && highest >= dst_zone->compact_cached_free_pfn) { highest -= pageblock_nr_pages; - cc->zone->compact_cached_free_pfn = highest; + dst_zone->compact_cached_free_pfn = highest; } cc->total_free_scanned += nr_scanned; @@ -1569,7 +1573,7 @@ fast_isolate_freepages(struct compact_control *cc) */ static void isolate_freepages(struct compact_control *cc) { - struct zone *zone = cc->zone; + struct zone *zone = cc->dst_zone ? cc->dst_zone : cc->zone; struct page *page; unsigned long block_start_pfn; /* start of current pageblock */ unsigned long isolate_start_pfn; /* exact pfn we start at */ @@ -2089,11 +2093,19 @@ static enum compact_result __compact_finished(struct compact_control *cc) unsigned int order; const int migratetype = cc->migratetype; int ret; + struct zone *dst_zone = cc->dst_zone ? cc->dst_zone : cc->zone; - /* Compaction run completes if the migrate and free scanner meet */ - if (compact_scanners_met(cc)) { + /* + * Compaction run completes if the migrate and free scanner meet + * or when either the src or dst zone has been completely scanned + */ + if (compact_scanners_met(cc) || + cc->migrate_pfn >= zone_end_pfn(cc->zone) || + cc->free_pfn < dst_zone->zone_start_pfn) { /* Let the next compaction start anew. */ reset_cached_positions(cc->zone); + if (cc->dst_zone) + reset_cached_positions(cc->dst_zone); /* * Mark that the PG_migrate_skip information should be cleared @@ -2196,10 +2208,13 @@ static enum compact_result compact_finished(struct compact_control *cc) static enum compact_result __compaction_suitable(struct zone *zone, int order, unsigned int alloc_flags, int highest_zoneidx, - unsigned long wmark_target) + unsigned long wmark_target, struct zone *dst_zone) { unsigned long watermark; + if (!dst_zone) + dst_zone = zone; + if (is_via_compact_memory(order)) return COMPACT_CONTINUE; @@ -2227,9 +2242,9 @@ static enum compact_result __compaction_suitable(struct zone *zone, int order, * suitable migration targets */ watermark = (order > PAGE_ALLOC_COSTLY_ORDER) ? - low_wmark_pages(zone) : min_wmark_pages(zone); + low_wmark_pages(dst_zone) : min_wmark_pages(dst_zone); watermark += compact_gap(order); - if (!__zone_watermark_ok(zone, 0, watermark, highest_zoneidx, + if (!__zone_watermark_ok(dst_zone, 0, watermark, highest_zoneidx, ALLOC_CMA, wmark_target)) return COMPACT_SKIPPED; @@ -2245,13 +2260,16 @@ static enum compact_result __compaction_suitable(struct zone *zone, int order, */ enum compact_result compaction_suitable(struct zone *zone, int order, unsigned int alloc_flags, - int highest_zoneidx) + int highest_zoneidx, struct zone *dst_zone) { enum compact_result ret; int fragindex; + if (!dst_zone) + dst_zone = zone; + ret = __compaction_suitable(zone, order, alloc_flags, highest_zoneidx, - zone_page_state(zone, NR_FREE_PAGES)); + zone_page_state(dst_zone, NR_FREE_PAGES), dst_zone); /* * fragmentation index determines if allocation failures are due to * low memory or external fragmentation @@ -2305,7 +2323,7 @@ bool compaction_zonelist_suitable(struct alloc_context *ac, int order, available = zone_reclaimable_pages(zone) / order; available += zone_page_state_snapshot(zone, NR_FREE_PAGES); compact_result = __compaction_suitable(zone, order, alloc_flags, - ac->highest_zoneidx, available); + ac->highest_zoneidx, available, NULL); if (compact_result == COMPACT_CONTINUE) return true; } @@ -2317,8 +2335,9 @@ static enum compact_result compact_zone(struct compact_control *cc, struct capture_control *capc) { enum compact_result ret; + struct zone *dst_zone = cc->dst_zone ? cc->dst_zone : cc->zone; unsigned long start_pfn = cc->zone->zone_start_pfn; - unsigned long end_pfn = zone_end_pfn(cc->zone); + unsigned long end_pfn = zone_end_pfn(dst_zone); unsigned long last_migrated_pfn; const bool sync = cc->mode != MIGRATE_ASYNC; bool update_cached; @@ -2337,7 +2356,7 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) cc->migratetype = gfp_migratetype(cc->gfp_mask); ret = compaction_suitable(cc->zone, cc->order, cc->alloc_flags, - cc->highest_zoneidx); + cc->highest_zoneidx, dst_zone); /* Compaction is likely to fail */ if (ret == COMPACT_SUCCESS || ret == COMPACT_SKIPPED) return ret; @@ -2346,14 +2365,19 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) * Clear pageblock skip if there were failures recently and compaction * is about to be retried after being deferred. */ - if (compaction_restarting(cc->zone, cc->order)) + if (compaction_restarting(cc->zone, cc->order)) { __reset_isolation_suitable(cc->zone); + if (dst_zone != cc->zone) + __reset_isolation_suitable(dst_zone); + } /* * Setup to move all movable pages to the end of the zone. Used cached * information on where the scanners should start (unless we explicitly * want to compact the whole zone), but check that it is initialised * by ensuring the values are within zone boundaries. + * + * If a destination zone is provided, use it for free pages. */ cc->fast_start_pfn = 0; if (cc->whole_zone) { @@ -2361,12 +2385,12 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) cc->free_pfn = pageblock_start_pfn(end_pfn - 1); } else { cc->migrate_pfn = cc->zone->compact_cached_migrate_pfn[sync]; - cc->free_pfn = cc->zone->compact_cached_free_pfn; - if (cc->free_pfn < start_pfn || cc->free_pfn >= end_pfn) { + cc->free_pfn = dst_zone->compact_cached_free_pfn; + if (cc->free_pfn < dst_zone->zone_start_pfn || cc->free_pfn >= end_pfn) { cc->free_pfn = pageblock_start_pfn(end_pfn - 1); - cc->zone->compact_cached_free_pfn = cc->free_pfn; + dst_zone->compact_cached_free_pfn = cc->free_pfn; } - if (cc->migrate_pfn < start_pfn || cc->migrate_pfn >= end_pfn) { + if (cc->migrate_pfn < start_pfn || cc->migrate_pfn >= zone_end_pfn(cc->zone)) { cc->migrate_pfn = start_pfn; cc->zone->compact_cached_migrate_pfn[0] = cc->migrate_pfn; cc->zone->compact_cached_migrate_pfn[1] = cc->migrate_pfn; @@ -2522,8 +2546,8 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) * Only go back, not forward. The cached pfn might have been * already reset to zone end in compact_finished() */ - if (free_pfn > cc->zone->compact_cached_free_pfn) - cc->zone->compact_cached_free_pfn = free_pfn; + if (free_pfn > dst_zone->compact_cached_free_pfn) + dst_zone->compact_cached_free_pfn = free_pfn; } count_compact_events(COMPACTMIGRATE_SCANNED, cc->total_migrate_scanned); @@ -2834,7 +2858,7 @@ static bool kcompactd_node_suitable(pg_data_t *pgdat) continue; if (compaction_suitable(zone, pgdat->kcompactd_max_order, 0, - highest_zoneidx) == COMPACT_CONTINUE) + highest_zoneidx, NULL) == COMPACT_CONTINUE) return true; } @@ -2871,7 +2895,7 @@ static void kcompactd_do_work(pg_data_t *pgdat) if (compaction_deferred(zone, cc.order)) continue; - if (compaction_suitable(zone, cc.order, 0, zoneid) != + if (compaction_suitable(zone, cc.order, 0, zoneid, NULL) != COMPACT_CONTINUE) continue; diff --git a/mm/internal.h b/mm/internal.h index 68410c6d97ac..349223cc0359 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -465,6 +465,7 @@ struct compact_control { unsigned long migrate_pfn; unsigned long fast_start_pfn; /* a pfn to start linear scan from */ struct zone *zone; + struct zone *dst_zone; /* use another zone as the destination */ unsigned long total_migrate_scanned; unsigned long total_free_scanned; unsigned short fast_search_fail;/* failures to use free list searches */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 5bf98d0a22c9..aa21da983804 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -6383,7 +6383,7 @@ static inline bool should_continue_reclaim(struct pglist_data *pgdat, if (!managed_zone(zone)) continue; - switch (compaction_suitable(zone, sc->order, 0, sc->reclaim_idx)) { + switch (compaction_suitable(zone, sc->order, 0, sc->reclaim_idx, NULL)) { case COMPACT_SUCCESS: case COMPACT_CONTINUE: return false; @@ -6580,7 +6580,7 @@ static inline bool compaction_ready(struct zone *zone, struct scan_control *sc) unsigned long watermark; enum compact_result suitable; - suitable = compaction_suitable(zone, sc->order, 0, sc->reclaim_idx); + suitable = compaction_suitable(zone, sc->order, 0, sc->reclaim_idx, NULL); if (suitable == COMPACT_SUCCESS) /* Allocation should succeed already. Don't reclaim. */ return true; From patchwork Wed Mar 20 02:42:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kaiyang Zhao X-Patchwork-Id: 13597222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57915CD11DC for ; Wed, 20 Mar 2024 02:42:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B05A6B008A; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8613F6B008C; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6913B6B0093; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5A38B6B008C for ; Tue, 19 Mar 2024 22:42:26 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 32A3DA0B12 for ; Wed, 20 Mar 2024 02:42:26 +0000 (UTC) X-FDA: 81915868692.12.71CBBF3 Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) by imf11.hostedemail.com (Postfix) with ESMTP id 66AA740006 for ; Wed, 20 Mar 2024 02:42:24 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=Mp6MRCb6; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf11.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.222.174 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710902544; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Gwth4DHc3n7Jk8RbQant6q4qmoLsduwQhc9A8tF9LNs=; b=glTVh74ySlIkqkyGaDBpJIt39het6x614RVt2RMP1J6SY0icb8ppSZifn11c53p91d/Kw/ vzqIh+vhNacOwY7tTiMHk/VwrmjykCnaRkn3pymzSTrOa9QO+GX6un2lNaGICFOIg1y1Gq PE3zbkGHsTqv4sMQG0yzanAVy5Do+GQ= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=Mp6MRCb6; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf11.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.222.174 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710902544; a=rsa-sha256; cv=none; b=49YW6ySvV/fbwfxjgZ88hZ7F7eJX7b7Rq7Z64k7nm6NRDP5FiU7Puy0JOULQFNUXSEBSf6 AaGcqno3nftwksQmQui9V+N9as9HSC9XEJ+WZVudHePfdSIkV4idnlViuYFsLOpgFvW6NH 0vnpDa0hywhonBiwdWuleRqKN5NBriE= Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-78a13117a3dso41383685a.1 for ; Tue, 19 Mar 2024 19:42:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902543; x=1711507343; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Gwth4DHc3n7Jk8RbQant6q4qmoLsduwQhc9A8tF9LNs=; b=Mp6MRCb6kaZ51Z1RvIPJD6kHJ5A5OAmnUF49Dv+YDah73SXP0yYexccSFiIVz8ItoF w/eDtFeIrA+SgA7mT2chA1uj62a1MHPC0iK617jcjRQiZfKTeaILGO+6Lhog//ItnbQe TtBYkyMr+D88U44GPukKKi8VS9e2Xi9Eit0mb5FSE8jYyrve8+kdzFmKYZWbL3i71NEU hHuauChUst6OGLfcwky0ZhzzJaVBvTZk21o3E6vGx0ThSRFHfKWZjdwuZ5cT3JmdAhPS lgaVNbBpBLy3BUcZeZYX10RGSwqcQvamztEPnjL6fP3tOkXcczciGY6bxpxd/muKDak2 bL+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902543; x=1711507343; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Gwth4DHc3n7Jk8RbQant6q4qmoLsduwQhc9A8tF9LNs=; b=hJVeiOaXD1vcG4F6tgMOcXC9kEx93vhgK2jzHWHCoz4OwNsnWVZGz4CkK2wDEuJPKV 9+ts0vbdlCWg++WOSe7sPcLZOZcLQGsp2hcPvYQBrSdSdBQbp06XiFtQ+dlJ3Qk3/FPL zUNoOzgu+wbdb6f20xsl517K8y6SSu5ZgJE/PW7ZCCSq1Qebc/iDVPyGZQP8YqMCZt9V upHuHBkF6iPj8+4cA1Aqad3JE6phUr3TYQ5bLvgpkgBY3gMfUwGHRhvheI1PITcu8Nm4 e+XRktyOD17bHLvjUrZkS91XcZ1AI91BOhWBc/o/oAr9x6PLslBN0QiDetrWmAA9jTIa jKOQ== X-Gm-Message-State: AOJu0Yx5qsDkKMsgCX2M8W7B8+DCEmxS0dI4+fSjFy9jY946iSdITNaF WXjTxqsuQ5i+mfwiOSLMsX4dSJOCBbrrw/nUmiyfiJjCN2s6yA0uocbO/lKY1ORK/VA4xfxbqy2 Be9n8ErZInNRm+m0cy/CE1ro+Nj10raAZr7uDvY9GthYWArACsS4JwoQ9P/h8on2JbqB2bifNrK zaIu199kKzi2v7MNGdW5prMJiXKiKQn67R9iA= X-Google-Smtp-Source: AGHT+IGHswrTM4FTiBPD0YXAQUMLyxW4PRx4gQjvpjDC8ST6/12GayLSbc0ihrs1ql7V1uIy1W3KoA== X-Received: by 2002:a05:620a:4691:b0:78a:120f:ee44 with SMTP id bq17-20020a05620a469100b0078a120fee44mr2253073qkb.28.1710902543396; Tue, 19 Mar 2024 19:42:23 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id d7-20020a05620a136700b0078863e0f829sm5213075qkl.12.2024.03.19.19.42.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:23 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 4/7] vmstat counter for pages migrated across zones Date: Wed, 20 Mar 2024 02:42:15 +0000 Message-Id: <20240320024218.203491-5-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 66AA740006 X-Stat-Signature: giu9ci68u9phi7m9wysean1wego8n1j1 X-Rspam-User: X-HE-Tag: 1710902544-332077 X-HE-Meta: U2FsdGVkX18NYwDUethVwxjdsu/ZbXpmohKcOHHB16p9/rAv5mDTKu45MA8ct0gZ3hrbKMmHtvsNvs7R0pzCEVbKvgA/qE5gLj6k/U22JzzYNyqb6HaXi8R5CfWI0q+cpaodEQ6B6ujujVwU2S3VL49QQiqSzAAJyhbBbkx6/V2J6CIKNTPy881Eyn5glxz317SlUUnqtBKw0X0SoBWm4nFdxluasHGKJM2KosOf1MQgHkNo6nlsm+DJUx2SuplvFxRkil5bOApUZLPTwjvgBKg7DpIwh9OZAHpSOc/EC1uGwPAaxZATyOYgkK0OshEG1eXWdog2AYgq9NACLRowUobrZSmZe07IzJNs2KQF1zoVL4Ca+f04wWIp2MrK21Uy4L/QzUFsXJiPNrRLpf7C5uwTmgoJPNvv5RfRQrcc8bhEfJNCU99V84jZQcRTnRetV7d0dLvtcgYBAsuXRpnaSpkrEKHMmJtZW4R36vgNiC96vYE7vVnmYqpmCA2IMSqo98Gyqm037amqC9ZUoPA//8ia/ZGcegONeh2afOr9QxrwEmUHgGzMlhztSmGssVpvdcraiWvfT9i3mQgEZCFbo2TvjhizuXt0YbjEEKd/J0zvEe3eEwl6NgbXrjb7DDRrSisZBJ4u6gIhuMdz08FYxJRvsEPEllVgkH9SC6c0A8TtwzdgEyZHXtPGY7znm5++Dy5MFHSOu0HRxorCUO6uFneVnPVWVb3MxJDWDN/TvxJfDWpHuvhw0Jb0dD6AEsSc9au5RNjg/RI4dhHcjpNSbieCPIQY/Go5zK4rRhpQ2UJPpp66roWmlnQ2w90oZS5kPfmNiu4a30a1sXvMHsLRTvrNL/ilKHh8Lf/WvDD6fPmeYpS4NJKIQZkVpxeZ7I6PuupGL2+6HtTvauQrs+WBpsh5T9cFRMIE1rfG+pEE3GjgK4KshiUqNGZZqGhdcFM1cCMIWcrP+AvCFexlQBI O0Iu8WcO cCwMgyG4BNkUjfW0lbbHAWKW8037sNthIMSg0f0EG0OxJaHjOOQDRloXzHJUKBDbHa6HAePLZtK3+a4gDyQFbkbLJFNAMVdsxaSdcbSca4zt2ILKpfYM8LhhecaP4nKVLyQ1LWJrQOaIPDK5v0MEAFgWdkm4/osr+SqHgWZdZK2+VGwA11QbO5OW05U79m/q/YqEJxDizdNQPPXMVYHj/19U3Dvw2SwpTURDq9FUCRr2zbVXDFvAjy+gu9b+Rsrbco2kG5nxDVVPiQCYaebbyylL02rYnE2ZJgEnrjlbKM3uVZCembyfGqy7KrRH4uDF64Shf8AL0gcEhy6R21yr3b0Z8poqFeChNCDRCqytQwKrpnlX1Xx/AMQ/OWLhOObTFWSQLHMPIHsmLRxYoAAjbLr1K3QUVNWX0M3cekE878Dxrtg4/ww47D+PrBeGWrAMFhVrC0sG6TcXO5e8tQ62hjUyvw3t0KJc+KMyKT1Kir9AInNGlpq2QWd2pbnn9SvHhMPwZYuwjDL4b2ZFEPKd7MVGbsZrUA05ueTOOTIpUBKydL+o16PAhNMDP0G+w6FkmcT7D X-Bogosity: Ham, tests=bogofilter, spamicity=0.044797, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kaiyang Zhao Add a counter for the number of pages migrated across zones in vmstat Signed-off-by: Kaiyang Zhao --- include/linux/vm_event_item.h | 1 + mm/compaction.c | 2 ++ mm/vmstat.c | 1 + 3 files changed, 4 insertions(+) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index 8abfa1240040..be88819085b6 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -79,6 +79,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, COMPACTSTALL, COMPACTFAIL, COMPACTSUCCESS, KCOMPACTD_WAKE, KCOMPACTD_MIGRATE_SCANNED, KCOMPACTD_FREE_SCANNED, + COMPACT_CROSS_ZONE_MIGRATED, #endif #ifdef CONFIG_HUGETLB_PAGE HTLB_BUDDY_PGALLOC, HTLB_BUDDY_PGALLOC_FAIL, diff --git a/mm/compaction.c b/mm/compaction.c index 03b5c4debc17..dea10ad8ec64 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -2552,6 +2552,8 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) count_compact_events(COMPACTMIGRATE_SCANNED, cc->total_migrate_scanned); count_compact_events(COMPACTFREE_SCANNED, cc->total_free_scanned); + if (dst_zone != cc->zone) + count_compact_events(COMPACT_CROSS_ZONE_MIGRATED, nr_succeeded); trace_mm_compaction_end(cc, start_pfn, end_pfn, sync, ret); diff --git a/mm/vmstat.c b/mm/vmstat.c index c28046371b45..98af82e65ad9 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1324,6 +1324,7 @@ const char * const vmstat_text[] = { "compact_daemon_wake", "compact_daemon_migrate_scanned", "compact_daemon_free_scanned", + "compact_cross_zone_migrated", #endif #ifdef CONFIG_HUGETLB_PAGE From patchwork Wed Mar 20 02:42:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kaiyang Zhao X-Patchwork-Id: 13597223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30FAEC54E68 for ; Wed, 20 Mar 2024 02:42:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A86966B008C; Tue, 19 Mar 2024 22:42:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A33FB6B0093; Tue, 19 Mar 2024 22:42:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BD306B0095; Tue, 19 Mar 2024 22:42:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7A12B6B008C for ; Tue, 19 Mar 2024 22:42:27 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4B5371A103E for ; Wed, 20 Mar 2024 02:42:27 +0000 (UTC) X-FDA: 81915868734.03.8C67320 Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by imf14.hostedemail.com (Postfix) with ESMTP id 76E0D10000B for ; Wed, 20 Mar 2024 02:42:25 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=bmxKdpLW; spf=pass (imf14.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.219.45 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu; dmarc=pass (policy=none) header.from=cs.cmu.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710902545; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Yh8JatHXgZIgb1eyxT1DhEdr2aiRbmcRrkpZMUB4hHw=; b=epbBr9VJ4xDSBEayAc64+Ef5gfYYyeFuT1yRX3M4kamXAdePF5E0O74W9vSUY7dukHkmhj yFMfwqkjyCvjbe/kII6nZktIY3GuyTnT082Y87nlO98K2N7M9U/BPISWHW5fQy4FLmFLGr slraGqfqpAREo1TwrDnX9+TcPx9mQLU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=bmxKdpLW; spf=pass (imf14.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.219.45 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu; dmarc=pass (policy=none) header.from=cs.cmu.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710902545; a=rsa-sha256; cv=none; b=Q2Su1nuezzpCROpe7IJZW9bdq2/ppeEELRvJysWh/uvjZeg3C7F6mbRHsyIKrMYNs1/WPz 5vu4axPJgE5VbQ5GiJ+ZGT6r1oGgji+gGNtFmrd9n46YGwMQELhiPeUmRJzsMf106mlOBJ kT0gAohriU1MNAaAXBqnvxH182uzfmA= Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-6963c0c507eso5139086d6.1 for ; Tue, 19 Mar 2024 19:42:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902544; x=1711507344; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=Yh8JatHXgZIgb1eyxT1DhEdr2aiRbmcRrkpZMUB4hHw=; b=bmxKdpLWpsHtAhyqW7RYpRwqPFO6zU452FsDtQQ/+meP304u/kVPzXriaeUrGx51ub XOfVLtOazuXrTkEe5st0HGz828XBWET4F+2DrTjlc4v2jMzCyEYYu49b5wFG8SseOSBC MERWeZGS81dPWH3LGqsFDVf+f9VuR+z1SiTEQAWYOy+Wk294a6bpZTdm5vvyjIi+AHfk eobOyUVsJQxZ/lpvS/7dDHqx5Z/VJXJ/3OY05h2rMFN0/tqsP/SuoEtm3jzcdQg25uyb u+SpgQ2tyO9NHfG2BjJOrXKY+ZCPVeRTaBv/Xn9nQBbGvmpXOLesO+Al5kK2AodCtH94 nubg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902544; x=1711507344; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Yh8JatHXgZIgb1eyxT1DhEdr2aiRbmcRrkpZMUB4hHw=; b=EdMyj1lpGlGBQCo/Xjjz3/PrP3aAa2EydPPDOCKcI150LzLIdc9rQcQMDMhrz/ucM5 E3nANgo2x9ca5IOfFXdvStdjURully+jYo943G6LGD+zAiS+xVpsmeyCJXPPudv4/5J7 3m4eepVhCqzi7QNFt3wcSmNk9tdquawvOOteaTxLZx80ZVBxdotTJiWk1DSi3BEoRaZ9 o9ksUKXWMPnNkkk35w1DUpqdKiyXH2pQmC9UBijZKtKTCX+4ZoYKU3tOr++Z0laFoReu SIqHzlBP7xYyvrQJ1SNAeoJTt8lioiCANdtcpvP+Qu99PWR04hvvZfRCyjTvZvYEq/eS pWcg== X-Gm-Message-State: AOJu0YyquqnZ7xHbrEAdBCboLetu5sV+8USUQSesnpg/QYbvpRPzY1cK jKg1A8lCxUiAFcNEYMXrFGaHq2WE3to+gJGB+pRE156fcamNGdLYnnbSWzf9dSBht4r+k4YJcDv 5wDwoviYaWGpOJPZyY1SMLOMOsKGbnmiVCvAeUSyjhwKcJochsP2WttLxPBdi78ctKdKYTZoteT bNcJn5N2luviOpdo3owP+NUd0pgwoqqTy8b7U= X-Google-Smtp-Source: AGHT+IGR8vrTWY1WaI5oE4IRdj4GRFNX5p3VLQAJ1dsWkq6eebPZOQskjAK08wq1ThkjZxRjS5rZPw== X-Received: by 2002:ad4:430b:0:b0:691:4d1f:6a65 with SMTP id c11-20020ad4430b000000b006914d1f6a65mr2449565qvs.27.1710902544204; Tue, 19 Mar 2024 19:42:24 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id k12-20020ad45bec000000b006961199b96asm3707324qvc.13.2024.03.19.19.42.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:23 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 5/7] proactively move pages out of unmovable zones in kcompactd Date: Wed, 20 Mar 2024 02:42:16 +0000 Message-Id: <20240320024218.203491-6-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao MIME-Version: 1.0 X-Rspamd-Queue-Id: 76E0D10000B X-Rspam-User: X-Stat-Signature: c9o49syu5qeeot8s6sr8m5w18wga8o74 X-Rspamd-Server: rspam01 X-HE-Tag: 1710902545-554266 X-HE-Meta: U2FsdGVkX1/N58zxWFXReE6cJFayl1TArHx5twXQJenmcl5dKevBtmhJFgCypXpl1PKEnzLfagRgyW/inFg/kcDs6t1fENTA66rIcRPJiC/arFMExL74TYZgkOH8YAPFwJEXqgupbWxdgmLwDQ2OGCT7bHijYU3O110iHS7e+gCAeMe023kK+DC1t85S3+3yLq8S5G5sB4spc0Ij31MmRbDyBFFc6rLelqxgX0F7BHOGLJ9LWe8XyeuY6w60z5KWxECsS96exfNhzyn0xe9/C9BfsRIGqWWRYPWyXGZ8qtHVa9UbbIjxm+QCB95HOVOFcX1iMFTAREy9zUbQQeK9PFigvVreyLEWH7Gd4IgCNYlN5QC/sz5zMJ05K5fD1rLpNVlaluyBf9vIAq9pRYI/BMGxtdVevLRmyxbfJMshuIMc5E83TeVMpZxHOy/kOrRuNml4dezTVTlpPUP+67t8+gle0Nikfm+YCaKbzyvlmTSnL3XR5Uh0mE9g9m16PPbaJ0EPe9G67UPooSeAD2TeFYWUunuW8+K6c1H3w3VinDwbNMfE7/cyDjmrNQmJXUTME5HKQV2JgMa6eltwZlnKRdzgPgu8Lc5LgUo7gJeunF0PpL/PlAXoSH6qdT56Yt/zVASoHNyBIthg7lxJre/vOllg0bo7b06UGuT5HDBM+W8eQhZAfIfHu2E7F0bWjgr1mUCCw6w04G9dCcg6CGDWsJ8IbdhE6St0HorEV3O4sceIBdGPVmGaDQmFXAf4KxCHMTPsNs0fHolYavbCp+PQ4w+BKhmCPuVh3UqEyXpGq5LCtLafpyet5pcPpS2sUsDEy7NPFN/VeD1XS271J+qv3vxIaF5WqwFxtCGTjDH4h8nG19ezVJe7L65aU/EOCS4TfLoZd85meNlldKfkVtkxbElUVFa6gnY24F7ZV+hpEixe4IFrT94umqEHBgqVH882TL3z3w07HwkPezybpXX dObYn5se Ep5Zg4w/QUCCeYjSz/Cd8+Nx0AdJdcdeg8DOnzrGVzylNJYjeBowkEczh7K9cTXhlusvH92naoNoTkHeB+7vz0zRarDFcNF1g6dnexgCxAdXNXxWoQPU039E59ch9Mm/zNscoeLduQ/DtRMfnZekZ+LpZz7GfrxxgsnMbCFvR+dzkTX/ImetYnisFPtI5qXnArJ+MXwCgqMftxgs0ZrKxa8aFUwISyWyTaUdsxBt/AuXF94TiUTxKtluZulRdldP2ecdcA6hkQ5lPnBbUSHATFuT+bXFXzBV8RW3Db9F56gXHfreusz1kRHFh9UjCXh44OXJq4WFJX5/gddzphdX1E0jZwd0dFdQyGRoy3UuLIUtXP7F9oyLJoFSpfLrYcwfO5qqnGFaDBGjVVO3tw70dXvVhLf+mv68Rgjcl3Vfh8N1bBRqLzDnZZVMe5XXQqUT/IdMx2mv0+Zx7dbR3pb4bDNwA9O/Fwc5yA4YhKSM0ObOb8dpwdStGe0hSEJfKS/mprNYTeOVO+IuGWMkfXipwmUAoovuxq/n3HC03eDh3uI5Xmt2Uvl4J1jg/Nnj3zStK8KIN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kaiyang Zhao Proactively move pages out of unmovable zones in kcompactd Debug only: zone start and end pfn printed in vmstat Added counters for cross zone compaction start and scan Signed-off-by: Kaiyang Zhao --- include/linux/vm_event_item.h | 3 + mm/compaction.c | 101 +++++++++++++++++++++++++++++++--- mm/vmstat.c | 11 +++- 3 files changed, 104 insertions(+), 11 deletions(-) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index be88819085b6..c9183117c8f7 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -80,6 +80,9 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, KCOMPACTD_WAKE, KCOMPACTD_MIGRATE_SCANNED, KCOMPACTD_FREE_SCANNED, COMPACT_CROSS_ZONE_MIGRATED, + KCOMPACTD_CROSS_ZONE_START, + COMPACT_CROSS_ZONE_MIGRATE_SCANNED, + COMPACT_CROSS_ZONE_FREE_SCANNED, #endif #ifdef CONFIG_HUGETLB_PAGE HTLB_BUDDY_PGALLOC, HTLB_BUDDY_PGALLOC_FAIL, diff --git a/mm/compaction.c b/mm/compaction.c index dea10ad8ec64..94ce1282f17b 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1436,7 +1436,10 @@ fast_isolate_freepages(struct compact_control *cc) * Preferred point is in the top quarter of the scan space but take * a pfn from the top half if the search is problematic. */ - distance = (cc->free_pfn - cc->migrate_pfn); + if (cc->zone != dst_zone) + distance = (cc->free_pfn - dst_zone->zone_start_pfn) >> 1; + else + distance = (cc->free_pfn - cc->migrate_pfn); low_pfn = pageblock_start_pfn(cc->free_pfn - (distance >> 2)); min_pfn = pageblock_start_pfn(cc->free_pfn - (distance >> 1)); @@ -1602,7 +1605,10 @@ static void isolate_freepages(struct compact_control *cc) block_start_pfn = pageblock_start_pfn(isolate_start_pfn); block_end_pfn = min(block_start_pfn + pageblock_nr_pages, zone_end_pfn(zone)); - low_pfn = pageblock_end_pfn(cc->migrate_pfn); + if (cc->dst_zone && cc->zone != cc->dst_zone) + low_pfn = pageblock_end_pfn(cc->dst_zone->zone_start_pfn); + else + low_pfn = pageblock_end_pfn(cc->migrate_pfn); stride = cc->mode == MIGRATE_ASYNC ? COMPACT_CLUSTER_MAX : 1; /* @@ -1822,7 +1828,11 @@ static unsigned long fast_find_migrateblock(struct compact_control *cc) * within the first eighth to reduce the chances that a migration * target later becomes a source. */ - distance = (cc->free_pfn - cc->migrate_pfn) >> 1; + if (cc->dst_zone && cc->zone != cc->dst_zone) + distance = (zone_end_pfn(cc->zone) - cc->migrate_pfn) >> 1; + else + distance = (cc->free_pfn - cc->migrate_pfn) >> 1; + if (cc->migrate_pfn != cc->zone->zone_start_pfn) distance >>= 2; high_pfn = pageblock_start_pfn(cc->migrate_pfn + distance); @@ -1897,7 +1907,7 @@ static isolate_migrate_t isolate_migratepages(struct compact_control *cc) { unsigned long block_start_pfn; unsigned long block_end_pfn; - unsigned long low_pfn; + unsigned long low_pfn, high_pfn; struct page *page; const isolate_mode_t isolate_mode = (sysctl_compact_unevictable_allowed ? ISOLATE_UNEVICTABLE : 0) | @@ -1924,11 +1934,16 @@ static isolate_migrate_t isolate_migratepages(struct compact_control *cc) /* Only scan within a pageblock boundary */ block_end_pfn = pageblock_end_pfn(low_pfn); + if (cc->dst_zone && cc->zone != cc->dst_zone) + high_pfn = zone_end_pfn(cc->zone); + else + high_pfn = cc->free_pfn; + /* * Iterate over whole pageblocks until we find the first suitable. * Do not cross the free scanner. */ - for (; block_end_pfn <= cc->free_pfn; + for (; block_end_pfn <= high_pfn; fast_find_block = false, cc->migrate_pfn = low_pfn = block_end_pfn, block_start_pfn = block_end_pfn, @@ -1954,6 +1969,7 @@ static isolate_migrate_t isolate_migratepages(struct compact_control *cc) * before making it "skip" so other compaction instances do * not scan the same block. */ + if (pageblock_aligned(low_pfn) && !fast_find_block && !isolation_suitable(cc, page)) continue; @@ -1976,6 +1992,10 @@ static isolate_migrate_t isolate_migratepages(struct compact_control *cc) isolate_mode)) return ISOLATE_ABORT; + /* free_pfn may have changed. update high_pfn. */ + if (!cc->dst_zone || cc->zone == cc->dst_zone) + high_pfn = cc->free_pfn; + /* * Either we isolated something and proceed with migration. Or * we failed and compact_zone should decide if we should @@ -2141,7 +2161,9 @@ static enum compact_result __compact_finished(struct compact_control *cc) goto out; } - if (is_via_compact_memory(cc->order)) + /* Don't check if a suitable page is free if doing cross zone compaction. */ + if (is_via_compact_memory(cc->order) || + (cc->dst_zone && cc->dst_zone != cc->zone)) return COMPACT_CONTINUE; /* @@ -2224,7 +2246,8 @@ static enum compact_result __compaction_suitable(struct zone *zone, int order, * should be no need for compaction at all. */ if (zone_watermark_ok(zone, order, watermark, highest_zoneidx, - alloc_flags)) + alloc_flags) && + dst_zone == zone) return COMPACT_SUCCESS; /* @@ -2270,6 +2293,11 @@ enum compact_result compaction_suitable(struct zone *zone, int order, ret = __compaction_suitable(zone, order, alloc_flags, highest_zoneidx, zone_page_state(dst_zone, NR_FREE_PAGES), dst_zone); + + /* Allow migrating movable pages to ZONE_MOVABLE regardless of frag index */ + if (ret == COMPACT_CONTINUE && dst_zone != zone) + return ret; + /* * fragmentation index determines if allocation failures are due to * low memory or external fragmentation @@ -2841,6 +2869,14 @@ void compaction_unregister_node(struct node *node) } #endif /* CONFIG_SYSFS && CONFIG_NUMA */ +static inline bool should_compact_unmovable_zones(pg_data_t *pgdat) +{ + if (populated_zone(&pgdat->node_zones[ZONE_MOVABLE])) + return true; + else + return false; +} + static inline bool kcompactd_work_requested(pg_data_t *pgdat) { return pgdat->kcompactd_max_order > 0 || kthread_should_stop() || @@ -2942,6 +2978,48 @@ static void kcompactd_do_work(pg_data_t *pgdat) pgdat->kcompactd_highest_zoneidx = pgdat->nr_zones - 1; } +static void kcompactd_clean_unmovable_zones(pg_data_t *pgdat) +{ + int zoneid; + struct zone *zone; + struct compact_control cc = { + .order = 0, + .search_order = 0, + .highest_zoneidx = ZONE_MOVABLE, + .mode = MIGRATE_SYNC, + .ignore_skip_hint = true, + .gfp_mask = GFP_KERNEL, + .dst_zone = &pgdat->node_zones[ZONE_MOVABLE], + .whole_zone = true + }; + count_compact_event(KCOMPACTD_CROSS_ZONE_START); + + for (zoneid = 0; zoneid < ZONE_MOVABLE; zoneid++) { + int status; + + zone = &pgdat->node_zones[zoneid]; + if (!populated_zone(zone)) + continue; + + if (compaction_suitable(zone, cc.order, 0, zoneid, cc.dst_zone) != + COMPACT_CONTINUE) + continue; + + if (kthread_should_stop()) + return; + + /* Not participating in compaction defer. */ + + cc.zone = zone; + status = compact_zone(&cc, NULL); + + count_compact_events(COMPACT_CROSS_ZONE_MIGRATE_SCANNED, + cc.total_migrate_scanned); + count_compact_events(COMPACT_CROSS_ZONE_FREE_SCANNED, + cc.total_free_scanned); + } +} + void wakeup_kcompactd(pg_data_t *pgdat, int order, int highest_zoneidx) { if (!order) @@ -2994,9 +3072,10 @@ static int kcompactd(void *p) /* * Avoid the unnecessary wakeup for proactive compaction - * when it is disabled. + * and cleanup of unmovable zones + * when they are disabled. */ - if (!sysctl_compaction_proactiveness) + if (!sysctl_compaction_proactiveness && !should_compact_unmovable_zones(pgdat)) timeout = MAX_SCHEDULE_TIMEOUT; trace_mm_compaction_kcompactd_sleep(pgdat->node_id); if (wait_event_freezable_timeout(pgdat->kcompactd_wait, @@ -3017,6 +3096,10 @@ static int kcompactd(void *p) continue; } + /* Migrates movable pages out of unmovable zones if ZONE_MOVABLE exists */ + if (should_compact_unmovable_zones(pgdat)) + kcompactd_clean_unmovable_zones(pgdat); + /* * Start the proactive work with default timeout. Based * on the fragmentation score, this timeout is updated. diff --git a/mm/vmstat.c b/mm/vmstat.c index 98af82e65ad9..444740605f2f 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1325,6 +1325,9 @@ const char * const vmstat_text[] = { "compact_daemon_migrate_scanned", "compact_daemon_free_scanned", "compact_cross_zone_migrated", + "compact_cross_zone_start", + "compact_cross_zone_migrate_scanned", + "compact_cross_zone_free_scanned", #endif #ifdef CONFIG_HUGETLB_PAGE @@ -1692,7 +1695,9 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, "\n spanned %lu" "\n present %lu" "\n managed %lu" - "\n cma %lu", + "\n cma %lu" + "\n start %lu" + "\n end %lu", zone_page_state(zone, NR_FREE_PAGES), zone->watermark_boost, min_wmark_pages(zone), @@ -1701,7 +1706,9 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, zone->spanned_pages, zone->present_pages, zone_managed_pages(zone), - zone_cma_pages(zone)); + zone_cma_pages(zone), + zone->zone_start_pfn, + zone_end_pfn(zone)); seq_printf(m, "\n protection: (%ld", From patchwork Wed Mar 20 02:42:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kaiyang Zhao X-Patchwork-Id: 13597224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06A59C54E71 for ; Wed, 20 Mar 2024 02:42:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21F936B0093; Tue, 19 Mar 2024 22:42:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D0696B0095; Tue, 19 Mar 2024 22:42:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 049546B0096; Tue, 19 Mar 2024 22:42:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E41966B0093 for ; Tue, 19 Mar 2024 22:42:27 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B4B174030E for ; Wed, 20 Mar 2024 02:42:27 +0000 (UTC) X-FDA: 81915868734.01.9D546D9 Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf30.hostedemail.com (Postfix) with ESMTP id 1FBF58000C for ; Wed, 20 Mar 2024 02:42:25 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=BrTpeNGK; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf30.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.160.181 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710902546; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WzVosZB8NV09/ZiVxYfUQEEiRP/cgORaiI2cKH0KzxM=; b=1uZANdcjalblLlqZvxQVhjhEh3fQwcNfg/l2Rhx1jg5xfHf9s3g2jOVBKu7DH0EbnCUG4e 3MbBBHYUgWFrpHglwexqIFh9RA1D2m9TJ6+5kW1XLL+z4WOjHy/PBrOs01EmGTW/KgHwhX qJ/KV0iLfm+QZFnkiI4JnVTu2eBZPkY= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=BrTpeNGK; dmarc=pass (policy=none) header.from=cs.cmu.edu; spf=pass (imf30.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.160.181 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710902546; a=rsa-sha256; cv=none; b=4FB0AGHsWjYcItLb1lPejbA3evk0sYj+uez+8E15BUZAyj3P9gyApEv08tVwaCBXHf063Y sTaiq3Ds4ezKVitVm/iyZUkYA3Vl5oXnp4UGanQheHXoHodCuGwLAob+iydVkTpjs/tpwX i3EFM2LnGW3RwHxma/XUxYNbP1bwEbU= Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-430ccb3d65eso20002841cf.2 for ; Tue, 19 Mar 2024 19:42:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902545; x=1711507345; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=WzVosZB8NV09/ZiVxYfUQEEiRP/cgORaiI2cKH0KzxM=; b=BrTpeNGK4HpNiDUjkQCasZRRf0kObmlN4ZYBoZb2ogi3pkiLqjVAYEFGhJPtvr8EZp Zjgs1IKNEPHs6Oj83i0JiWgO7PZ0krG6PhKxxTmYse6FeoXDaeJYZe27LTDAhFjhe78W lZoIN1R03MoDQBJtMR7efFtdy8g4q4TlNO0woasU22HXM8ZKWcaPvDBgyIEhNm3P0+gv aFsiaBbwn8pfcYacXLdS9HGEZv7nRxczry3JnhAJVO3Gg9lW79T2aeYbFczL31spD/Fb FrAPe9Np7W5Szfz7um54XbLXx4zdWtSTPn74k1RfEuj8Jxxyv2V7EwFiZ7Qw3NWa9vsZ +nVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902545; x=1711507345; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=WzVosZB8NV09/ZiVxYfUQEEiRP/cgORaiI2cKH0KzxM=; b=wTKGgWgUso96VrSpPvJWEylQxgchJWhLuzwZKfD+ZMs+kjhd07Vtwv6kwvrKItyeVn AFKjTTL4tbx4eiWiAZ5Gw5s3IRV2KUGPFAbGobNJ4RsfMapBHJjNbWzGbw3DVc7h2xnZ tOu/xJw8PYw0o48lxYiU5qxXPbPEigfXQVmUoxTR9yYHk8nTTSaAzvsdWplrntmkrZCg MXnpHGkHh17Syvm6yFwyfB/mhj+KIZI4a+WM0qFG7Kb9RBjd2nBHKkyRN+hMHKsO6BQf eNe788cfTdfzLBDX87dAXVQJhwzpidovi3IJCn6eXAk+WG1Nmkpn9TD4JcLyrCjEgtD1 OaiA== X-Gm-Message-State: AOJu0YxSEj+uE6T2YrTvu+wav9xlexm9o+SPk9tvStOCrCAqPpWVFuVf bGIXlZ347JcV7pSwoW+6WVf0v9mtw14ZdhFD69CgWwrfn2o7cNR4LbWpEog39UMoZZzoFoMRdtr Qkay+l61ZZgAIHbKY+YqEEOJNBNJoyPg/OyxwAuw/dkTRSdxQ3zJiYjEE3ar0rq0iWOjbhcuft4 lmSc73yhc3iCwQMYmYpbQk1O0Hf5eMr6J6I1Q= X-Google-Smtp-Source: AGHT+IGEfbBr6cutAmN+mlO7n/aShmIQU3GTmhE5CM9Lh4mCEbZf0tbj0IjwPIaD6NVNuXa3bBmlRg== X-Received: by 2002:a05:622a:110a:b0:431:baa:ab0c with SMTP id e10-20020a05622a110a00b004310baaab0cmr135047qty.50.1710902544986; Tue, 19 Mar 2024 19:42:24 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id fg14-20020a05622a580e00b00430bddc75a5sm4582852qtb.23.2024.03.19.19.42.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:24 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 6/7] pass gfp mask of the allocation that waked kswapd to track number of pages scanned on behalf of each alloc type Date: Wed, 20 Mar 2024 02:42:17 +0000 Message-Id: <20240320024218.203491-7-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao MIME-Version: 1.0 X-Rspamd-Queue-Id: 1FBF58000C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: mi49jk5hh1ctngbfjfx9emokwojmgge5 X-HE-Tag: 1710902545-722042 X-HE-Meta: U2FsdGVkX18Qzq2ePisLP/+4dIAwkwRaVbd9iPGMvlxnpOn+WEwlvyWw7SDqCU3njcEkXszcCyyeCI2lTMKieym3lbhhvlikjUEh5OVC8xvs3BRRoft5K7q34f/rML1oRJzXQB+FORgg8SgrdilQOgTPd7BDYVC8faWFyPimaFUts7mQ9Va/fTktBjOdN7J596zqjoZbq0xVfbY70wA5x4Wu6Yb+lxC0wG57Hhg1ybjeSISZat9jB3yuPo/VYMOD6JO4gCQKsGrRGdfOlmx0Jd1BpRcm4Y3lJkFRhg0EzzyApXPdy//frKhCBvaWiOiDqgAKOC0zcMno052chFLa6QryDllkTI0fZ+g/pg2WZ04/7eSgYybrbUrtXW65K5Y+laUuA1PrkjAVDDLKWdifv+BDn0lptG4jVH+Imx+c7lN4ML26cI176O/coznnpmxdOfmRjgv9AQjIl1uLPSTmYY4FMSHF1WwzB1sCpmj70HsVTaYfNqqOpso1MY2l3uHb5UtEOr3bJGahAboJ97eliJT2N2eJ14cbIygJL4Ua77ZGdxQjIiy6zKUDRvV9rLcJn3silBR6GamyCyA19DhBcwgkRqaVqWmE5Xg+ZjFLfypta9Mbe4oMTedcTa82d8R19w2xF48Qpy3IJgYumBy10ig9cIWki2d31G4gdvOoip1rOPzXDPHjFjK+qMp/80ToUbgaluMmTXN8EpER7iUL6yrUmpNuuFt+Jx9YWW1+CEod9PlHIuQhAkbQ10FQN7u2IjC7zoixnBx8lmD4X8S0zR1MqMl7FxXoQ+vilajS+Xxtze0tg3kBS61EZNLV01wXm0ujxZpL+ewqnX4OIvb4SDRjbhzcFsofLmvjAA251ldse8k60+5DwvV7NOSqA29p6BKIlu7kiMydwc6WnE1gnRNIuFLazeiSpwK0FXSD3kRdo7KF3DFDmZ+T3zpndb/e6a1Q5CrQkLpw8Ko9UHq +qa19Ccy qcIyJ33CZQJEnagWxblhKksunwwoe4JopAgjXKeLA/XCmh//N0gmfRpKvHSVXapCQwvoT3ov85eOq9E4ov8EA75x4IKpGENveLbsPq+DpayY3QzBS7kaaHbNct4989+NN585G/8wqVleGluI1SDT2aRZVnEihzWWTdj8tyybP05UpJoehGOPN4ggmtsLFR3wRTIR1sW7zUHuxq8whkwwYFW4UYTcQ2aWcYE3IBh3QlVwnpPWlJS4IiIIpRJV+XvOLvt5wklsuCacDcebNbNoNPtuK8TKow4WXhUIV6XLlteffem+SH7/joluCW4pJWTGnkoEdivBwF86v94E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.002856, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kaiyang Zhao In preparation for exporting the number of pages scanned for each alloc type Signed-off-by: Kaiyang Zhao --- include/linux/mmzone.h | 1 + mm/vmscan.c | 13 +++++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index a4889c9d4055..abc9f1623c82 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1288,6 +1288,7 @@ typedef struct pglist_data { struct task_struct *kswapd; /* Protected by kswapd_lock */ int kswapd_order; enum zone_type kswapd_highest_zoneidx; + gfp_t kswapd_gfp; int kswapd_failures; /* Number of 'reclaimed == 0' runs */ diff --git a/mm/vmscan.c b/mm/vmscan.c index aa21da983804..ed0f47e2e810 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -7330,7 +7330,7 @@ clear_reclaim_active(pg_data_t *pgdat, int highest_zoneidx) * or lower is eligible for reclaim until at least one usable zone is * balanced. */ -static int balance_pgdat(pg_data_t *pgdat, int order, int highest_zoneidx) +static int balance_pgdat(pg_data_t *pgdat, int order, int highest_zoneidx, gfp_t gfp_mask) { int i; unsigned long nr_soft_reclaimed; @@ -7345,6 +7345,8 @@ static int balance_pgdat(pg_data_t *pgdat, int order, int highest_zoneidx) .order = order, .may_unmap = 1, }; + if (is_migrate_movable(gfp_migratetype(gfp_mask))) + sc.gfp_mask |= __GFP_MOVABLE; set_task_reclaim_state(current, &sc.reclaim_state); psi_memstall_enter(&pflags); @@ -7659,6 +7661,7 @@ static int kswapd(void *p) pg_data_t *pgdat = (pg_data_t *)p; struct task_struct *tsk = current; const struct cpumask *cpumask = cpumask_of_node(pgdat->node_id); + gfp_t gfp_mask; if (!cpumask_empty(cpumask)) set_cpus_allowed_ptr(tsk, cpumask); @@ -7680,6 +7683,7 @@ static int kswapd(void *p) WRITE_ONCE(pgdat->kswapd_order, 0); WRITE_ONCE(pgdat->kswapd_highest_zoneidx, MAX_NR_ZONES); + WRITE_ONCE(pgdat->kswapd_gfp, 0); atomic_set(&pgdat->nr_writeback_throttled, 0); for ( ; ; ) { bool ret; @@ -7687,6 +7691,7 @@ static int kswapd(void *p) alloc_order = reclaim_order = READ_ONCE(pgdat->kswapd_order); highest_zoneidx = kswapd_highest_zoneidx(pgdat, highest_zoneidx); + gfp_mask = READ_ONCE(pgdat->kswapd_gfp); kswapd_try_sleep: kswapd_try_to_sleep(pgdat, alloc_order, reclaim_order, @@ -7696,8 +7701,10 @@ static int kswapd(void *p) alloc_order = READ_ONCE(pgdat->kswapd_order); highest_zoneidx = kswapd_highest_zoneidx(pgdat, highest_zoneidx); + gfp_mask = READ_ONCE(pgdat->kswapd_gfp); WRITE_ONCE(pgdat->kswapd_order, 0); WRITE_ONCE(pgdat->kswapd_highest_zoneidx, MAX_NR_ZONES); + WRITE_ONCE(pgdat->kswapd_gfp, 0); ret = try_to_freeze(); if (kthread_should_stop()) @@ -7721,7 +7728,7 @@ static int kswapd(void *p) trace_mm_vmscan_kswapd_wake(pgdat->node_id, highest_zoneidx, alloc_order); reclaim_order = balance_pgdat(pgdat, alloc_order, - highest_zoneidx); + highest_zoneidx, gfp_mask); if (reclaim_order < alloc_order) goto kswapd_try_sleep; } @@ -7759,6 +7766,8 @@ void wakeup_kswapd(struct zone *zone, gfp_t gfp_flags, int order, if (READ_ONCE(pgdat->kswapd_order) < order) WRITE_ONCE(pgdat->kswapd_order, order); + WRITE_ONCE(pgdat->kswapd_gfp, gfp_flags); + if (!waitqueue_active(&pgdat->kswapd_wait)) return; From patchwork Wed Mar 20 02:42:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kaiyang Zhao X-Patchwork-Id: 13597225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E90EC54E68 for ; Wed, 20 Mar 2024 02:42:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FA0D6B0095; Tue, 19 Mar 2024 22:42:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BF046B0096; Tue, 19 Mar 2024 22:42:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA13E6B0098; Tue, 19 Mar 2024 22:42:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BC56C6B0095 for ; Tue, 19 Mar 2024 22:42:28 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 97E76A014A for ; Wed, 20 Mar 2024 02:42:28 +0000 (UTC) X-FDA: 81915868776.30.0599411 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by imf19.hostedemail.com (Postfix) with ESMTP id F0CA21A0009 for ; Wed, 20 Mar 2024 02:42:26 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=Oy3gckHl; spf=pass (imf19.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.219.44 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu; dmarc=pass (policy=none) header.from=cs.cmu.edu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710902547; a=rsa-sha256; cv=none; b=OHkBp0KETIFilZlutSASNS4IkhWPn/RIB771pF8eyWr+XT7v1b+6uO3LQnzloNBJyVnYjS DurRe93kH+D/12YUQnnVo7eGM9Oux1QQdHAmqzGJl+OosBgKQht7/HwVkRPXiqW88PV/X2 7Y6wxG7z8X0tNy6gamslsDzDU+fpbDI= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=cs.cmu.edu header.s=google-2021 header.b=Oy3gckHl; spf=pass (imf19.hostedemail.com: domain of kaiyang2@andrew.cmu.edu designates 209.85.219.44 as permitted sender) smtp.mailfrom=kaiyang2@andrew.cmu.edu; dmarc=pass (policy=none) header.from=cs.cmu.edu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710902547; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DIGL+dvsKsYG7Tx7310j/b/qOHwOOB9JaG0dOT6QmVU=; b=JNraL8T40xazXoeM2w/fpmvVnQ95Bq1EkgOGD8UfoFZkyj/GLpcsqRPmz6j+S1Hcq3CzKn OinqFq+oaOGEDbYaXaU9xqzOT9m9edrKNqOr6/gnoAHywg4yZ3t/lPbpxdetwOJKe3f1dA sGS24FPofHHCXxDJdBKHc36GYs3nCNk= Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-690bd329df2so42751676d6.2 for ; Tue, 19 Mar 2024 19:42:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.cmu.edu; s=google-2021; t=1710902546; x=1711507346; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=DIGL+dvsKsYG7Tx7310j/b/qOHwOOB9JaG0dOT6QmVU=; b=Oy3gckHlkIklzRmaLI9e0KRgsJiYxXX7rQP6gHCtZVlyDw7S6DQ9P3hugKXR57ow2B 9pweHVtI1VASkEVvbWjx9ehAyP9AxMqgODj5jSW8DrNXJJEP0iTgXWXuipmEeg4uV787 TnemiJxT5lMcQ8WemyO2qc7bYD7jciCTZVXxlV474Fdl7fa6J0EofJHqiFXHqhz7lddt xBeplZ0LuSGNiXzYKsJMWaxeC37bxrRy+ed1qRHA7r2FQ/clHqRijQ+PA6hUBgo4CLIL wzXYtDI402ul/1sT+HCmT+uE8zZWzOEAaQ4DBxRNSQPWWiqJhD2O1YHfY3g32TJ40LTU iMQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710902546; x=1711507346; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=DIGL+dvsKsYG7Tx7310j/b/qOHwOOB9JaG0dOT6QmVU=; b=HwMzuLhP3F6c33xC2F1Kqc1BrPLv/gDaOSSwheSEOcnAXNNutoE2DbqP8J7H5Er56m JlIReA3gIRlrJ9pk/zm72ZXk0uyCZk8gIzeo10QlljjYW7Mz23HR3FWAOc3+VijiQden 5MBTT4uTVUepAzPLbtfNu0qWN5LhF3Dc2MqtmbnTXHNHK/tc66JMNlQah7VOkic2BxDE a32RPLQXm7plmhj3yreGFtcUyefAuJe2/7mSfjiSLlAiNhjNNJG1hp71aIPzvm3bUTwk TNAOlPyWMhnzfQU7oU27izHsFmbnuRwm+3yx/k9mSHOLRLpfWPxcWaBFdNb9rffcYWiL OCSg== X-Gm-Message-State: AOJu0YwRbgI/1FpTkwgLZstC0s09fG6zH5lH9X5GEhX/cqcCliNOV895 nnesKQDRfbg4JaaHLt9MRHlmv3YZ5R5o72cxgQGAearlOGBeO/qedXSOGpKlvofWmu7rwR5Jstt DWSUmblcKtTCVZL3WVTtN4nP0TEIA/9SmZcuK8qUFrvl8gzdcdcMPg4ZiY6MvopDXZW4a7WizXS 06LJSo5vi42QaNkpx8WJZKGaDHysR823dZemw= X-Google-Smtp-Source: AGHT+IF1l0N2egM3aSC98LzaXNROU1+O+6Y4xqmw+dsS00Fx32Wnx7e/F2IaRaJEP8IIHWZ1DYSFqw== X-Received: by 2002:a0c:dc14:0:b0:696:22ae:eb67 with SMTP id s20-20020a0cdc14000000b0069622aeeb67mr4027527qvk.33.1710902545749; Tue, 19 Mar 2024 19:42:25 -0700 (PDT) Received: from localhost (pool-74-98-221-57.pitbpa.fios.verizon.net. [74.98.221.57]) by smtp.gmail.com with UTF8SMTPSA id g15-20020a0562140acf00b0069150ebcc30sm7204402qvi.76.2024.03.19.19.42.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Mar 2024 19:42:25 -0700 (PDT) From: kaiyang2@cs.cmu.edu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Kaiyang Zhao , hannes@cmpxchg.org, ziy@nvidia.com, dskarlat@cs.cmu.edu Subject: [RFC PATCH 7/7] exports the number of pages scanned on behalf of movable/unmovable allocations Date: Wed, 20 Mar 2024 02:42:18 +0000 Message-Id: <20240320024218.203491-8-kaiyang2@cs.cmu.edu> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> References: <20240320024218.203491-1-kaiyang2@cs.cmu.edu> Reply-To: Kaiyang Zhao MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: F0CA21A0009 X-Stat-Signature: e5cn314sa5yowkzof4qtarztuozm3jz9 X-Rspam-User: X-HE-Tag: 1710902546-989719 X-HE-Meta: U2FsdGVkX1/pIRxn2I3xZdsdby5bgtJ6j6mgXe2kBTJyTF/EbE47pKM1qxnsyppf3nhH3YuM+H4OMJN+A2yO1gTbOASljRSqPdxr2VydBaS0sni4KM6p/CFUpvnIGiK1gtrggnnEKn76NLVXYX2b1mhnjDsOMD9m58HQc8eROzUJW5KWognvodr0/2nwAgOx1qRjE4IIh6X5QRdkKnRBEySlEuYQPcPwGljvcBywLf1ko7eoyY4BlorEcK09qEYIHnzQZOD2l7JR+l44pSm74NtQ0SHiSMRnwqTa+IUtLT0+z+iiozkqrGRVXYIbpEg1Dc0uEpTLcBgzIKbgyj26sU/OaJf0H8l5Vu6TD7yM6mekdWfJdKfYEJmIoX6pi3zRobfDLUU4CHfYvXWPhSIvy9mqp85LaIj/sLH132qub7BxAOFTYVDVlQwGKcgD49+X9L6iT9UsITGuo9hkI5hm1ppW8ywk6e6R+EeyepmzQK++x6xgF9jUzPPcrHjsOC69DIJoOoo0I4K0Q30rXP855yDnbyrvjrtU8q1NMCikAei+PTRpIIX0SPMfVPM6ZI0U+bnOb1yNoix3WPAq1R0oOHGbVmJE5KsHG3fWV0f9weA/JZKJPnOq4DS1LDO9i0aPPl6mUGNcT473QYWdyQfth8dAoMUbhKA83M4voEOYI1Ax5fJRH0EWAuIPjNEJE20w771x/FvuPWs2aOkbE+w+tKmyYQ06oIHlBJhY4kKhRvtFoRamACc3h3WV5bw3pVWfn4uqiYC2zNxA2U93UQXE0KDDzm3yrNrXL0LZQIvyvQ+84qWXJhAofxyMMmZHY/cO3r3eF7XLeGfzYQeQ0GgEzMfCb2ZfKaAEfmaeEirNvVgK/WItWl0VWPLhwIg+hhOzO+l+WpomZCTwoeVbNL49MBoITV2nWLg1+J3ImSynE0/si2J0uF1VC82tb/Cud7digtq9LPmY25gQaKk5Pcs K66hGedX VpqBWkoiyiTXH0HwNVQCO5db/pYBxVNo9x2mViyQIiANY6n+eFXQlJqQFod8W/ikzhCvDP5VPDvVyU3rpUBUlYtNoo9aiH3lv5EOkf7Owxa8Nzk8vrljNL7iBYcioPNekLFLNfT4fVtW+MX13SX3c1GOj0fsRvQzaRDJH0wZ0EDVFMCj2xTO8/RT/NHtRP9I7Pgnn4rnqtwMSvDf0dec7fGkckAJ54PKP6iGtbFHCxnFOBIcqhRWhGqbtnUjpmPZqtKhQOdexyAquRwQ/FjlukJoi2pshSqcHJObZmGc0402SgPm6nYbmPiQSrBzHeMrXfLj9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.002132, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kaiyang Zhao exports the number of pages scanned on behalf of movable/unmovable allocations in vmstat Signed-off-by: Kaiyang Zhao --- include/linux/vm_event_item.h | 2 ++ mm/vmscan.c | 11 +++++++++++ mm/vmstat.c | 2 ++ 3 files changed, 15 insertions(+) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index c9183117c8f7..dcfff56c6d29 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -50,6 +50,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, PGSCAN_DIRECT_THROTTLE, PGSCAN_ANON, PGSCAN_FILE, + PGSCAN_MOVABLE, /* number of pages scanned on behalf of a movable allocation */ + PGSCAN_UNMOVABLE, PGSTEAL_ANON, PGSTEAL_FILE, #ifdef CONFIG_NUMA diff --git a/mm/vmscan.c b/mm/vmscan.c index ed0f47e2e810..4eadf0254918 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -904,6 +904,12 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, cond_resched(); } + /* Arbitrarily consider 16 pages scanned */ + if (is_migrate_movable(gfp_migratetype(shrinkctl->gfp_mask))) + count_vm_events(PGSCAN_MOVABLE, 16); + else + count_vm_events(PGSCAN_UNMOVABLE, 16); + /* * The deferred work is increased by any new work (delta) that wasn't * done, decreased by old deferred work that was done now. @@ -2580,6 +2586,11 @@ static unsigned long shrink_inactive_list(unsigned long nr_to_scan, __count_memcg_events(lruvec_memcg(lruvec), item, nr_scanned); __count_vm_events(PGSCAN_ANON + file, nr_scanned); + if (is_migrate_movable(gfp_migratetype(sc->gfp_mask))) + __count_vm_events(PGSCAN_MOVABLE, nr_scanned); + else + __count_vm_events(PGSCAN_UNMOVABLE, nr_scanned); + spin_unlock_irq(&lruvec->lru_lock); if (nr_taken == 0) diff --git a/mm/vmstat.c b/mm/vmstat.c index 444740605f2f..56062d53a36c 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1281,6 +1281,8 @@ const char * const vmstat_text[] = { "pgscan_direct_throttle", "pgscan_anon", "pgscan_file", + "pgscan_by_movable", + "pgscan_by_unmovable", "pgsteal_anon", "pgsteal_file",