From patchwork Wed Feb 1 19:50:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 13124914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41831C05027 for ; Wed, 1 Feb 2023 19:52:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 425D46B0072; Wed, 1 Feb 2023 14:52:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3ADC26B0073; Wed, 1 Feb 2023 14:52:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29CB96B0075; Wed, 1 Feb 2023 14:52:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 13F806B0072 for ; Wed, 1 Feb 2023 14:52:35 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DABC91A0FFD for ; Wed, 1 Feb 2023 19:52:34 +0000 (UTC) X-FDA: 80419770228.10.72DF84F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id F13BAC0010 for ; Wed, 1 Feb 2023 19:52:32 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="PEbTBY/j"; spf=pass (imf28.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675281153; a=rsa-sha256; cv=none; b=2XBAugFpC12wtZ84OWWtBzJA+RDhpp41E71FSB5yYJ/GTeYHIYS9jlEe1/M5VbdTbVtaLd VWFpJlHyordylr1YrRe93sDERpdF6an0/d6NUgRt57kJYgZ7rA7D6P4eGPQL1NRZd0OsGP bzBlKKtKwkxpBUYG+D0XMImHQ1pAxxY= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="PEbTBY/j"; spf=pass (imf28.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675281153; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:references:dkim-signature; bh=7C1hmy1E7+0MQkK/mZzmUZ/gKoFVIBd7d2YZY/3tmcQ=; b=KkNnEsBFmNFDJQA15PVWNWqhs/5CfOMhY2m7ogogHb67KxItkSwLf7KlW9Itjw9AGPudC0 /F4zO1Kg8tS2dUcVd43ta/1HiTL4G9HNV4QW+rc54dgWU6jsdcEPKfXr3Ps7gq7bjOs2Z1 sVLmwFot0KYE79lWRDPyEvQNZ/R81rU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675281152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=7C1hmy1E7+0MQkK/mZzmUZ/gKoFVIBd7d2YZY/3tmcQ=; b=PEbTBY/jpgG10Z/vEHePGkDX6YvPTB8g+NLWmDmgyiuIuW8y86FtvjsqLm+4/FQaiIU+Fx ORw7PQUAE+MfSMYbp5D9WmZC8TG2ZTZ/SfP1efvPiYEOfZAqzocAAScJf7mcgJjESm+wCc n+Nv68YxS0oc6EaMtJgGG/B1kDv5GGo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-511-mZt9PK5VNHS2IdT6gYfx_w-1; Wed, 01 Feb 2023 14:52:31 -0500 X-MC-Unique: mZt9PK5VNHS2IdT6gYfx_w-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4A1D73C02558; Wed, 1 Feb 2023 19:52:30 +0000 (UTC) Received: from tpad.localdomain (ovpn-112-2.gru2.redhat.com [10.97.112.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 182731121339; Wed, 1 Feb 2023 19:52:30 +0000 (UTC) Received: by tpad.localdomain (Postfix, from userid 1000) id 0D1C1403C42BF; Wed, 1 Feb 2023 16:51:49 -0300 (-03) Message-ID: <20230201195104.411744803@redhat.com> User-Agent: quilt/0.67 Date: Wed, 01 Feb 2023 16:50:14 -0300 From: Marcelo Tosatti To: Christoph Lameter Cc: Aaron Tomlin , Frederic Weisbecker , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Marcelo Tosatti Subject: [PATCH 1/5] mm/vmstat: remove remote node draining References: <20230201195013.881721887@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspam-User: X-Rspamd-Queue-Id: F13BAC0010 X-Rspamd-Server: rspam01 X-Stat-Signature: 54pdaarqk1zfftfmwkxotwie9q7d41u4 X-HE-Tag: 1675281152-611328 X-HE-Meta: U2FsdGVkX1/+TKOv7XoqhKselN7e6iU8PLjv70cjr9TswMzJCtrJlCBH4nXVZP1WxkfI0jOx9/enfSYKslVvLKltpk11++Jd+tUBpoczhe3MqCpfl7Hwh4WxElrMOly2Jx1qeHQXXWkzj1gZh/bv4vlT92rRnL+ioQVDgRdNDizZT20hjLC/lKN8uqrZnPIWfFlHztijJm+42t26lNoRPu7J+ATRTQ64+cTL/H+OK6J5WHrevz8L9/Rqc1cEWicyotTW4xmz4yJq7gfyteqYA8SbU7LZVpMcSyEJIJkC5VWEXBAaRToqQxGUncwq/APLWKzQUXQezDnbLyeXWnHxgCbxZrnQvqhY9yYv/09dPyK5N5q/x1vn5J8voy6UHG7szn29ehjEFcZ7s7JRNm9OaPnTdiO1xLwiiEW/P5oSls6wOX0ah5OeyVx6ap5UhGyLI3dzbApS+MAQQIE8wPaRoWNztSm8rzDtluzcnge8/0BY077OQUoF2BYLzwIm7of8akhibVntM5Ju2PTZzmhyaabSzzz6bP3SH9Z2/oF6H+N+PZoM0/k2Wt+6x8W1jD/hpdUet+mqYAFPGpRpZCdYbQu6GxnvvJPLro0p1dbIGGsnoIoHHBOj1/1bnL9HfD77CqUos+yhbPdGyuGuJq0XTLJ2WokgcR7Y0b2dL8WQKKQIKdrw8fXtYMl4VIrEx8jZTXhPDZZ0VtNu6jFjMyKoZ1vJ20LP/3TVul/3yeS7MCXpEVoRyLEONHLUo1m12tUaKwS4Io3Ie9YXHEvYhzdIt+q+1W0twsc6MM+M1SIkaQp/vhtdU32PfB0YRHMIlKJky/Z7e9hvgX1dqQdMoQ0mxyVtsWuvQsIMa1JZ7czyqlvEaWfmXkSv/h19RLwZoUzg7LJsBGJtdxpc/x7GiKhHGqnTesiyRqVWYrIBTNBcqQnQpS51ZZSHa+r+pbZ+UZm0pWlkNqSGHfsRV5LjvaL 3npqAmKQ Xf4qUMKBXWR4lhftrlb3VjmvgMJNP5fH7RKkdIcvAU4m/ou/xdPGpzGZ2Sh6v5s7FsTYeu5Htnr8CmqefWvRWTKe0dYGGt0vsziCjoltDh87FsM2C6yFnQhawMkJmqeUaS81SsJN9puAdD8pCLSG+J0hjLk4JjU6bhUbFBEpdiiOxQO1iLB0ZUZc8cRCRl7o8H91k1SrLzLJ2pmfK60ujmi9+sB5ggr9wQ+aS6cC1tGh56Oc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Draining of pages from the local pcp for a remote zone was necessary since: "Note that remote node draining is a somewhat esoteric feature that is required on large NUMA systems because otherwise significant portions of system memory can become trapped in pcp queues. The number of pcp is determined by the number of processors and nodes in a system. A system with 4 processors and 2 nodes has 8 pcps which is okay. But a system with 1024 processors and 512 nodes has 512k pcps with a high potential for large amount of memory being caught in them." Since commit 443c2accd1b6679a1320167f8f56eed6536b806e ("mm/page_alloc: remotely drain per-cpu lists"), drain_all_pages() is able to remotely free those pages when necessary. Signed-off-by: Marcelo Tosatti Index: linux-vmstat-remote/include/linux/mmzone.h =================================================================== --- linux-vmstat-remote.orig/include/linux/mmzone.h +++ linux-vmstat-remote/include/linux/mmzone.h @@ -577,9 +577,6 @@ struct per_cpu_pages { int high; /* high watermark, emptying needed */ int batch; /* chunk size for buddy add/remove */ short free_factor; /* batch scaling factor during free */ -#ifdef CONFIG_NUMA - short expire; /* When 0, remote pagesets are drained */ -#endif /* Lists of pages, one per migrate type stored on the pcp-lists */ struct list_head lists[NR_PCP_LISTS]; Index: linux-vmstat-remote/mm/vmstat.c =================================================================== --- linux-vmstat-remote.orig/mm/vmstat.c +++ linux-vmstat-remote/mm/vmstat.c @@ -803,7 +803,7 @@ static int fold_diff(int *zone_diff, int * * The function returns the number of global counters updated. */ -static int refresh_cpu_vm_stats(bool do_pagesets) +static int refresh_cpu_vm_stats(void) { struct pglist_data *pgdat; struct zone *zone; @@ -814,9 +814,6 @@ static int refresh_cpu_vm_stats(bool do_ for_each_populated_zone(zone) { struct per_cpu_zonestat __percpu *pzstats = zone->per_cpu_zonestats; -#ifdef CONFIG_NUMA - struct per_cpu_pages __percpu *pcp = zone->per_cpu_pageset; -#endif for (i = 0; i < NR_VM_ZONE_STAT_ITEMS; i++) { int v; @@ -826,44 +823,8 @@ static int refresh_cpu_vm_stats(bool do_ atomic_long_add(v, &zone->vm_stat[i]); global_zone_diff[i] += v; -#ifdef CONFIG_NUMA - /* 3 seconds idle till flush */ - __this_cpu_write(pcp->expire, 3); -#endif } } -#ifdef CONFIG_NUMA - - if (do_pagesets) { - cond_resched(); - /* - * Deal with draining the remote pageset of this - * processor - * - * Check if there are pages remaining in this pageset - * if not then there is nothing to expire. - */ - if (!__this_cpu_read(pcp->expire) || - !__this_cpu_read(pcp->count)) - continue; - - /* - * We never drain zones local to this processor. - */ - if (zone_to_nid(zone) == numa_node_id()) { - __this_cpu_write(pcp->expire, 0); - continue; - } - - if (__this_cpu_dec_return(pcp->expire)) - continue; - - if (__this_cpu_read(pcp->count)) { - drain_zone_pages(zone, this_cpu_ptr(pcp)); - changes++; - } - } -#endif } for_each_online_pgdat(pgdat) { @@ -1864,7 +1825,7 @@ int sysctl_stat_interval __read_mostly = #ifdef CONFIG_PROC_FS static void refresh_vm_stats(struct work_struct *work) { - refresh_cpu_vm_stats(true); + refresh_cpu_vm_stats(); } int vmstat_refresh(struct ctl_table *table, int write, @@ -1928,7 +1889,7 @@ int vmstat_refresh(struct ctl_table *tab static void vmstat_update(struct work_struct *w) { - if (refresh_cpu_vm_stats(true)) { + if (refresh_cpu_vm_stats()) { /* * Counters were updated so we expect more updates * to occur in the future. Keep on running the @@ -1991,7 +1952,7 @@ void quiet_vmstat(void) * it would be too expensive from this path. * vmstat_shepherd will take care about that for us. */ - refresh_cpu_vm_stats(false); + refresh_cpu_vm_stats(); } /*