From patchwork Mon Aug 21 20:54:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13359805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65E44EE4996 for ; Mon, 21 Aug 2023 20:55:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5C22940010; Mon, 21 Aug 2023 16:55:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D0CC08E0012; Mon, 21 Aug 2023 16:55:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD43C940010; Mon, 21 Aug 2023 16:55:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AB0D48E0012 for ; Mon, 21 Aug 2023 16:55:04 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7E62DC0123 for ; Mon, 21 Aug 2023 20:55:04 +0000 (UTC) X-FDA: 81149316528.19.A152125 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf17.hostedemail.com (Postfix) with ESMTP id CF6264001A for ; Mon, 21 Aug 2023 20:55:02 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=VapQsrLT; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3Jc_jZAoKCIA2swv2elqihksskpi.gsqpmry1-qqozego.svk@flex--yosryahmed.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3Jc_jZAoKCIA2swv2elqihksskpi.gsqpmry1-qqozego.svk@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692651302; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=kxBzNRuKT3FK2xpnd0zI9fHxTniHJWyags4nIBlfqGo=; b=pDPBX81E68RYZrm7E7qEJy5ds4KU8CNYPu30gyQ607A5EayFWTaKyFYJuZi/ymCa/L/aoI PFblPlESYtXkMWeaUKSDTc95LV+tayhq9zpU6GUgySJajLaDWkQBlTAIlofeQ7+VPuP+jU LNbdNiP9K8PZmlpkITYhIKWS6rUbo6A= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=VapQsrLT; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3Jc_jZAoKCIA2swv2elqihksskpi.gsqpmry1-qqozego.svk@flex--yosryahmed.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3Jc_jZAoKCIA2swv2elqihksskpi.gsqpmry1-qqozego.svk@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692651302; a=rsa-sha256; cv=none; b=v8fJTIfZEloxS3uWogcPDpN5QB9iSBJLgkVfmr0H6tvSwBtctoajxoaZZE7uVOS05Hq6RH zm9kKKBwH60ooID7JKTzQmswQ255BIVfkMaHF+v4MHEL+XfY7SrzYC+Hha02DlgRhZO+It xrZ5qdRnsNAwbJWpRzFrd6KKwaNpry0= Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-563b4d68e07so5023030a12.1 for ; Mon, 21 Aug 2023 13:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1692651301; x=1693256101; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=kxBzNRuKT3FK2xpnd0zI9fHxTniHJWyags4nIBlfqGo=; b=VapQsrLTsIuBdCvmJlnkDnAHLl9uwXYyl4pfelPDhgZfBCaf5UuJw/4Df69B1daF5W gCChcu7YIx6NYtBnd3VU2xjT4LtJH6C/uxHDF7ZH/XP0VUHjmQ+px5zJGjW8u6Pyo3Oj yj/4S4uWg7n3+4ebepXr/CillPdeCpQ5UBCWFWdu+mHPXoiOzfNDuksE7H66dECQgKT6 m1AWJKvfJ8wt7vsYfZxBy4KRUdv08ei+xhliPGh0s3uythMQplNFuk0BbdjYyAumUH0L lIyQdhxBjiYhgCOEDtvGnrQH3rbPBra4M16LsQzP1POusk9aDfzM4/jaIedXoVa+MnbM DV9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692651301; x=1693256101; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=kxBzNRuKT3FK2xpnd0zI9fHxTniHJWyags4nIBlfqGo=; b=d08GWTUPmDP+2y8SCCfY8+lBrUAuNLhq2Fe4KXLiUkWFciNlN6sPxAYnNXVbICCmL3 qdYq+5Rw4sfoCMN40U/atomJPegyOrNo8XYNTY2q5FWiMFwc+uYCPiIii6MTxC7uKS0/ p1IT3KXJGpPTj6+e1vLvitemJFrZYZlMc/VYg1aLBgelnnuWBoL9WtwrDDFad9XgY1Mq /CbYOjbxDov7toAesDEDIC0CUcTrruV3RiU80ntKQIe7EkgN9U92H9KR77Wy/Gj/LCNl OnjeF+EtM/pIBGOBonOO/ZjqhRA1scDTG/rw31u8laPYbfM/7wk7CDDftEjaIbhCMT2M L1yA== X-Gm-Message-State: AOJu0YwdKrjylHCvYWrC1cuJmGLnqb9TcCOog8NHKGXBoo0zk+mGyPz1 2xFoi/zJUncVnng3rO3QjaPcp65s/dYfVPX+ X-Google-Smtp-Source: AGHT+IGm4+gLgX0J8HWzAgZ1L2jlPh7tsMAE98z+t4zMYWWyBe2ouAbZ7MlyET/FFgHamToA8Hug4lBE2hC2cTf8 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90b:4ac1:b0:269:84ec:7140 with SMTP id mh1-20020a17090b4ac100b0026984ec7140mr1536395pjb.2.1692651301416; Mon, 21 Aug 2023 13:55:01 -0700 (PDT) Date: Mon, 21 Aug 2023 20:54:55 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.rc1.204.g551eb34607-goog Message-ID: <20230821205458.1764662-1-yosryahmed@google.com> Subject: [PATCH 0/3] memcg: non-unified flushing for userspace stats From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: CF6264001A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 48xwhhd9of9kxhf3uad37daai5gy37ni X-HE-Tag: 1692651302-819005 X-HE-Meta: U2FsdGVkX1/5hbz42RpGI4uhCg/m8gRbxZW6nuImCWccI2uEj9K4f1JaVhRXsR/WjFidgLY7Za25H29B/w7sGt4o+VyBokBJq8SAeixN2P9w0Ub6YNtBWJyq5YE9DRCNCvQ0Bo6n2OGvNbCdMZ0iJs/xIPPbyoN51bQQrJdUA9sLS5ngIvm45KzbRYT/uzptFHnuJfu3nUiJ2jHlHI/V9nmqG4oztUUrFk8SPG7USvFmpbxweWMtdMzIp3xYJwEdbIIZ8GSdeBBDRUwyD8QUU9Hh+hJfKp0n6VplIHFAHEHJ6G8Ip3SfGqgaeiPuEEjn8ukjZwxPBxAhBUqFZCdRzawRerX5kXkHPCKh/Q43a11tDziy4HGSNJfrmxoPlhggA855mO/59JWno83ZEC7DnXzfPpOkAw56sA6zrSrxbC1MST77XCjOWgq+MJH5Xbi/0u8NZbZJ0V7XWZb3X/QTUL9J+mNi843qB8mZVCp558imMUCR3g/CHPA31S5271ZV/xHginoJwwamlpLM/82Ge3CPbIH6upJXoMSYmDx7C2pQXZd7qxCsFxpQmEG3nzEuRZBVYmpylNHqBSAYmB/qfZS7dNkmLqIErHBC+VbokpaYPn157F276tlNPlogONUHUJ8JmWpkMRvA3vdztHcDiSsCRMbVxx+Rglh/RNO/sX4Q0GgdsNqe6xcHjGNWwQYIxSFgYgl579NPWfGtrz30E8kQfWElNE5zZkCxrACNy56v/9naJDmhEoilWULAYRsY0G3Q7dNSlViGZBsE8AadEVM82OlMz4wU88r0gd2tn7p/pnLws5IcLPR7iUKBSjkm4bcDKf8xe/G87ckWGDMrGzIKt6TgTfvDpSFC6vkVokqVyuYf3n9hc86ZzA0VQAoJOCxKbOzXmu/wxqpJPqZgNgNcOiou0aqto8PURSX3juqlDc+/gegAu3GiLHG2Ev1m7UGShyWxEchsUSWInxm WZPi2uil npCMzbTSzFYLGWmckzP3+Vl9pMEhhiAwx/5oOX56l/O1oo8A/34McTK/BLysc6HsOn4B0VaZw4YZgZBi1K6mG1jO4tETJ+5WZ0dP0zCCIYspnSe6SJ8t32lS+OVzz8Ljw3TtrqlEgwZJzGZS3noAAvNNr3GB03fYg8sj1m8ERRh0syMgIXGfrzcsPJVtb+kiZq+wtJtrn2lhCL9c95Gi+nhjZ11NqtR49BoUa0CyGZwY2OZvkh3ICLEK/2OUjPOEM98rruiBqX3c2HlcGp0Pkfo42PHX33vo6wfECzEMLcoR76fjot8tb+W2/NlXZj0jwjqgImyf6BlJPMKL5046iUzYPyierdYHy9MZxukC/YRPztN6qS377HN3g9QzNz8nBAzqmc1cCVhFkM934QPaieFqGHbobzrUIu8DFhiiPeDD0aNdcY34VGTtil5QKOBdrpfqisUR3EebTGkFxODkwxZsWhZcJwMbnaxiqj07Cy5IUc4/zi73q11+IgvGf5qF2XvqynftQrj94rYyw825WcL57kvpulKAqQI24arjO4tIxMYQEOIj2AOYGYI+xxPfK38NT3FLCFvoryn0pYNfbZtvjz1pr/0Ws4Uh1lzM55DvFLw+UhHoMULTAemraL9Nb8B5n457UmsV7nol1Tf9UWYXace+6qqs0FVsM3BXJyxbSUvmBtc2WYjSN67Xlc+dZYngdw5f33I38cJfVDZyuGqLgYx7cMbcX0HPm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Most memcg flushing contexts using "unified" flushing, where only one flusher is allowed at a time (others skip), and all flushers need to flush the entire tree. This works well with high concurrency, which mostly comes from in-kernel flushers (e.g. reclaim, refault, ..). For userspace reads, unified flushing leads to non-deterministic stats staleness and reading cost. This series clarifies and documents the differences between unified and non-unified flushing (patches 1 & 2), then opts userspace reads out of unified flushing (patch 3). This patch series is a follow up on the discussion in [1]. That was a patch that proposed that userspace reads wait for ongoing unified flushers to complete before returning. There were concerns about the latency that this introduces to userspace reads, especially with ongoing reports of expensive stat reads even with unified flushing. Hence, this series follows a different approach, by opting userspace reads out of unified flushing completely. The cost of userspace reads are now determinstic, and depend on the size of the subtree being read. This should fix both the *sometimes* expensive reads (due to flushing the entire tree) and occasional staless (due to skipping flushing). I attempted to remove unified flushing completely, but noticed that in-kernel flushers with high concurrency (e.g. hundreds of concurrent reclaimers). This sort of concurrency is not expected from userspace reads. More details about testing and some numbers in the last patch's changelog. [1]https://lore.kernel.org/lkml/20230809045810.1659356-1-yosryahmed@google.com/ [2]https://lore.kernel.org/lkml/CABWYdi0c6__rh-K7dcM_pkf9BJdTRtAU08M43KO9ME4-dsgfoQ@mail.gmail.com/ Yosry Ahmed (3): mm: memcg: properly name and document unified stats flushing mm: memcg: add a helper for non-unified stats flushing mm: memcg: use non-unified stats flushing for userspace reads include/linux/memcontrol.h | 8 ++--- mm/memcontrol.c | 74 +++++++++++++++++++++++++++----------- mm/vmscan.c | 2 +- mm/workingset.c | 4 +-- 4 files changed, 60 insertions(+), 28 deletions(-)