From patchwork Mon Jun 6 22:20:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12871132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1DD1DC433EF for ; Mon, 6 Jun 2022 22:22:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Mime-Version: Message-Id:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=gTcAvRdoDT39Wc5u0ClVB5mxTPeVUxJ9UIJesHWWNX8=; b=jbt 9F9NyHh6BQP46BC93g037MbPNC52AI1X3y25tcBHa7jhQxJ425RlugGhaJt0eXbLMh1/lXbVOHyCu MABfVs8C6+WrZUlPa1+ZYv++Pr88QqhySTaN1N5T/uhlQA+tdB3dzkXliV17sSMkSC9P7MWflGq+9 E9Ve0hFx203bSi3xviwq0q44AC8mnWShkCskl1K43D7TDqHseyZVFTE66nXIn9X6lOL8rzs4FIwi+ N1JVjpGAOwKeqZ4imSpJUhgYiKHSYuNZ9slJhFVlv3bIS6lJ+lpEL7+pj1BuaKaswe/lv4B/UrA39 IP1E6SYcM3rT73d7W7dQbs270e/B8oA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nyL61-002yne-25; Mon, 06 Jun 2022 22:21:13 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nyL5x-002ymT-3D for linux-arm-kernel@lists.infradead.org; Mon, 06 Jun 2022 22:21:10 +0000 Received: by mail-pf1-x44a.google.com with SMTP id z5-20020aa79f85000000b0051baa4e9fb8so8414492pfr.7 for ; Mon, 06 Jun 2022 15:21:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=IJS83ZKmvay92IEqxRhRKAGkk5kusGJyLJThB250L6U=; b=OBqMy3wK2eu42u4CbtznTgnyseS6gsaJLr5FDK08zli4P+8VmL18aLuhw+bSnMdM7k iFz03jncbwfAA7r/AGTwPC55fSnuiaen+bze1yfaaUyFA+CNkxQfCc05giglU0XkUEVo t0wTaD5v0fh6UuTsvLua6zwa4GLFp9MUBqwGr5TCmjvLaGNwvrJnSC+2h7QKCddI9v2O 1AvKbLWw/YHC21qUQMpix/9u1/qvIqsCc78g0RrjeITNLQzJr0ZmEpnbP1xHYGLnKYVq HF3gC2h07Gg4ZKzkzeDiB+DLw++z+7xHgEUcgkjKGWNrPJUIIGcun4GT14NaNl+uGKgC 0Sxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=IJS83ZKmvay92IEqxRhRKAGkk5kusGJyLJThB250L6U=; b=eZC6BVjDwAx5bIRwmDyQo+85ZCqpxeclZiG7G9gjFvo/5xncVv6YOjdgjU8ljNSEcN ZzLAjrx1FFehq5N2TCcGF+tCFdXE6UleqbmerfBLSiWE2Sj++FzdEMbqpytY/uuaUEzH cFV8LdsK96HBGXGAQ31l/H2autBHrx2pueT27xjhquaPVuZx5n5xqHuZTBtxUq+mk6uo emuXBk//lFhCWiE5O9iCDLKBRZC1WHgnx0AUXiOe8qrMWRWKS0wJxD8c2DHksfY6Z7Ti hQRb/QCHPewskndybKHLp50eoxCXLNpU5Y0x92TsjJPHtNUsPhohqrTzefnhFQP2f2b6 5Xwg== X-Gm-Message-State: AOAM5331QwnyeRugCFT443UMUjnCUtXVFoQ80yXloO5HZ1hN0rs4rLP+ cd9xw6fzN6rkHYnx+UYAS2xf5E0EMaZ/HJnG X-Google-Smtp-Source: ABdhPJx1wdZontdtDHAURSxbRB7De+X97i6gu5k06GDDv+Ku0jwJHaJ3ZGwcu97YUcHYEb+wa/3Y2TiM5NS+scjm X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:d584:b0:167:6ab9:b094 with SMTP id k4-20020a170902d58400b001676ab9b094mr12694803plh.16.1654554067166; Mon, 06 Jun 2022 15:21:07 -0700 (PDT) Date: Mon, 6 Jun 2022 22:20:54 +0000 Message-Id: <20220606222058.86688-1-yosryahmed@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.36.1.255.ge46751e96f-goog Subject: [PATCH v5 0/4] KVM: mm: count KVM mmu usage in memory stats From: Yosry Ahmed To: Tejun Heo , Johannes Weiner , Zefan Li , Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Oliver Upton Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220606_152109_191055_A3720779 X-CRM114-Status: GOOD ( 20.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We keep track of several kernel memory stats (total kernel memory, page tables, stack, vmalloc, etc) on multiple levels (global, per-node, per-memcg, etc). These stats give insights to users to how much memory is used by the kernel and for what purposes. Currently, memory used by kvm mmu is not accounted in any of those kernel memory stats. This patch series accounts the memory pages used by KVM for page tables in those stats in a new NR_SECONDARY_PAGETABLE stat. This stat can be later extended to account for other types of secondary pages tables (e.g. iommu page tables). KVM has a decent number of large allocations that aren't for page tables, but for most of them, the number/size of those allocations scales linearly with either the number of vCPUs or the amount of memory assigned to the VM. KVM's secondary page table allocations do not scale linearly, especially when nested virtualization is in use. From a KVM perspective, NR_SECONDARY_PAGETABLE will scale with KVM's per-VM pages_{4k,2m,1g} stats unless the guest is doing something bizarre (e.g. accessing only 4kb chunks of 2mb pages so that KVM is forced to allocate a large number of page tables even though the guest isn't accessing that much memory). However, someone would need to either understand how KVM works to make that connection, or know (or be told) to go look at KVM's stats if they're running VMs to better decipher the stats. Also, having NR_PAGETABLE side-by-side with NR_SECONDARY_PAGETABLE is informative. For example, when backing a VM with THP vs. HugeTLB, NR_SECONDARY_PAGETABLE is roughly the same, but NR_PAGETABLE is an order of magnitude higher with THP. So having this stat will at the very least prove to be useful for understanding tradeoffs between VM backing types, and likely even steer folks towards potential optimizations. --- Chnages in V5: - Updated cover letter to explain more the rationale behind the change (Thanks to contributions by Sean Christopherson). - Removed extraneous + in arm64 patch (Oliver Upton, Marc Zyngier). - Shortened secondary_pagetables to sec_pagetables (Shakeel Butt). - Removed dependency on other patchsets (applies to queue branch). Changes in V4: - Changed accounting hooks in arm64 to only account s2 page tables and refactored them to a much cleaner form, based on recommendations from Oliver Upton and Marc Zyngier. - Dropped patches for mips and riscv. I am not interested in those archs anyway and don't have the resources to test them. I posted them for completeness but it doesn't seem like anyone was interested. Changes in V3: - Added NR_SECONDARY_PAGETABLE instead of piggybacking on NR_PAGETABLE stats. Changes in V2: - Added accounting stats for other archs than x86. - Changed locations in the code where x86 KVM page table stats were accounted based on suggestions from Sean Christopherson. --- Yosry Ahmed (4): mm: add NR_SECONDARY_PAGETABLE to count secondary page table uses. KVM: mmu: add a helper to account memory used by KVM MMU. KVM: x86/mmu: count KVM mmu usage in secondary pagetable stats. KVM: arm64/mmu: count KVM s2 mmu usage in secondary pagetable stats Documentation/admin-guide/cgroup-v2.rst | 5 ++++ Documentation/filesystems/proc.rst | 4 +++ arch/arm64/kvm/mmu.c | 35 ++++++++++++++++++++++--- arch/x86/kvm/mmu/mmu.c | 16 +++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 12 +++++++++ drivers/base/node.c | 2 ++ fs/proc/meminfo.c | 2 ++ include/linux/kvm_host.h | 9 +++++++ include/linux/mmzone.h | 1 + mm/memcontrol.c | 1 + mm/page_alloc.c | 6 ++++- mm/vmstat.c | 1 + 12 files changed, 87 insertions(+), 7 deletions(-)