From patchwork Tue Jan 2 15:38:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13509130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99F29C46CD2 for ; Tue, 2 Jan 2024 15:38:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11D6D6B0296; Tue, 2 Jan 2024 10:38:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CF546B029A; Tue, 2 Jan 2024 10:38:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E88CA6B029B; Tue, 2 Jan 2024 10:38:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D05296B0296 for ; Tue, 2 Jan 2024 10:38:49 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A7E231C1078 for ; Tue, 2 Jan 2024 15:38:49 +0000 (UTC) X-FDA: 81634778778.29.A31B7C0 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id E441414001A for ; Tue, 2 Jan 2024 15:38:45 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704209926; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=9vACLO0HhGruQj9RpiBzlla0ek8TFI2xuU+hAw6BNz8=; b=aC0tvTKjQCYWiNWxucvRnxF6EbXU/rlAmx+e9gVPzkUR80pFIiP01PgI+V8QNn1tctEvkq e9DZevY9F7eFOrIWCvI7BA7d+0zT9OAA/vLnJHHscztQ/xBuzZvVEOVii4DYehXxWUwAe4 sI5fFgXMLeg9IL1PvE/KM72GnF+wvlA= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704209926; a=rsa-sha256; cv=none; b=VzKiq0XWTPlyS4RceQp4NsZlrjVoPUyqaUB2Jbhi8cgivEC4rfhSrfZ3puVQzFM0KN+jXL 0gqxW3DUgFR+UpkC9JYT7DRAGFzpJL4occYzFVwY9oesnBo6/TFhOrnqghffgFVX7WbdVB L95/mr8XKLYBB6IKUpIkfYIy+DlwVEs= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 11FCDC15; Tue, 2 Jan 2024 07:39:31 -0800 (PST) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 344A03F64C; Tue, 2 Jan 2024 07:38:43 -0800 (PST) From: Ryan Roberts To: Andrew Morton , Zenghui Yu , Matthew Wilcox , David Hildenbrand , Kefeng Wang , John Hubbard , Zi Yan , Barry Song <21cnbao@gmail.com>, Alistair Popple Cc: Ryan Roberts , linux-mm@kvack.org Subject: [RFC PATCH v1] tools/mm: Add thpmaps script to dump THP usage info Date: Tue, 2 Jan 2024 15:38:28 +0000 Message-Id: <20240102153828.1002295-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: E441414001A X-Stat-Signature: wsc3j3zuxwijpjq6ye3681a6suyc5hi9 X-HE-Tag: 1704209925-324690 X-HE-Meta: U2FsdGVkX1/caCnNHCYP4b9J63pwuWmXO1oAl8lKlN+/Z1Vv92sNk6EwHfGk3jabnJXL/HcTB2ZyC8iX+hpwTPD2SjZUwE8NiwkAljF1q6pwjsm8rihUpMvg1YfEBtvZWG1YMyg6B0+NlXhkkOotEbNhQvkDMJfgriZ9o0nctAjhbSugGRutOP360NzDS5HxaupFoX6YAvELiXH9l9C2jmHA/3tAd6guxU2CCaZYsPYPYyLKj0iu/2/nFZWT0rSkrmB2zkaJFd6y7nNSB4AIVlWS4ha5+BphdUeYwiHDnXACCrVPTHDkv8LwF2fHZRh63t5obE4eBT0c5IUgkE5b4RsUH0+/NfqvDeOGW/Hmi554TZsPj63BFntQ5N5BHiZ6K6JaXOlx9pEBKR+TJSdXJbmA7ghKAMKuCZfjlrYw9g31VsnqquCEX9nWRJZkbFYB3pKfE+1k0QnHLMGYjZumSHr9qcxHk3vbNRzCpNn72E58ujfoFrBHqgdSkMDMOAetQnSEHA7BxoPC8g3FIRSgzGUjgr0vBuLXefk2qhMQp/hBGJRcOSiFfCnlsmVSejx9SC3BwFBc3GIPHHj3gZl07mhZ1NGPk8jAmgYv6VhndrABceGJgM7cyxvi04k5LcWrQath0l09qZNmm6gIbiv4DlleOvmGB0acq9zIyLDnv6a1W43wT1dwDmj8t5o9sR0djNho2zr3uAdjMhZAWPa1weBK47ywQE5gmZ5YO9cMh5bYM1kR9jiMBPVgtRk4pk0xsiHIQDgmmQv6rV+8h8DyeIndm1fOqjQ3uNbjPSQxesu5TcFE847RXzoSfvn8AY7j6fEEJq1Q/PkObXksCVgqN8xDaEuoTuKqbwLo+Vfl3TFHm7AKR578/98uPPqOmJ8LNSkff5KiOEtep20+jR2QD05uU6/aWYGPmfPlcQY+rMs/TxFHAVXJ/S5cj1B8iA1U3FjFq8CZDuqsPiaEqHO Cwrd8NmV rPmHsuq+UHqUBAVBdcoT1uv6rfwdQRHmrlPu/DoJEcGavocTTAEgQjsRbh4qHBb7B9DhxP3DSuJaVY6kbDIg2lC8zUZoD7LCBLb8wkidu5DsE1E8O+14k2o2HFQjqbFTo2c3gokrc2PIa5kfg01kVq7FTkqPaeMi58MbC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With the proliferation of large folios for file-backed memory, and more recently the introduction of multi-size THP for anonymous memory, it is becoming useful to be able to see exactly how large folios are mapped into processes. For some architectures (e.g. arm64), if most memory is mapped using contpte-sized and -aligned blocks, TLB usage can be optimized so it's useful to see where these requirements are and are not being met. thpmaps is a Python utility that reads /proc//smaps, /proc//pagemap and /proc/kpageflags to print information about how transparent huge pages (both file and anon) are mapped to a specified process or cgroup. It aims to help users debug and optimize their workloads. In future we may wish to introduce stats directly into the kernel (e.g. smaps or similar), but for now this provides a short term solution without the need to introduce any new ABI. Run with help option for a full listing of the arguments: # thpmaps --help --8<-- usage: thpmaps [-h] [--pid pid] [--cgroup path] [--summary] [--cont size[KMG]] [--inc-smaps] [--inc-empty] [--periodic sleep_ms] Prints information about how transparent huge pages are mapped to a specified process or cgroup. Shows statistics for fully-mapped THPs of every size, mapped both naturally aligned and unaligned for both file and anonymous memory. See [anon|file]-thp-[aligned|unaligned]-kB keys. Shows statistics for mapped pages that belong to a THP but which are not fully mapped. See [anon|file]-thp-partial keys. Optionally shows statistics for naturally aligned, contiguous blocks of memory of a specified size (when --cont is provided). See [anon|file]-cont- aligned-kB keys. Statistics are shown in kB and as a percentage of either total anon or file memory as appropriate. options: -h, --help show this help message and exit --pid pid Process id of the target process. Exactly one of --pid and --cgroup must be provided. --cgroup path Path to the target cgroup in sysfs. Iterates over every pid in the cgroup. Exactly one of --pid and --cgroup must be provided. --summary Sum the per-vma statistics to provide a summary over the whole process or cgroup. --cont size[KMG] Adds anon and file stats for naturally aligned, contiguously mapped blocks of the specified size. May be issued multiple times to track multiple sized blocks. Useful to infer e.g. arm64 contpte and hpa mappings. Size must be a power-of-2 number of pages. --inc-smaps Include all numerical, additive /proc//smaps stats in the output. --inc-empty Show all statistics including those whose value is 0. --periodic sleep_ms Run in a loop, polling every sleep_ms milliseconds. Requires root privilege to access pagemap and kpageflags. --8<-- Example command to summarise fully and partially mapped THPs and 64K contiguous blocks over all VMAs in a single process (--inc-empty forces printing stats that are 0): # ./thpmaps --pid 10837 --cont 64K --summary --inc-empty --8<-- anon-thp-aligned-16kB: 16 kB ( 0%) anon-thp-aligned-32kB: 0 kB ( 0%) anon-thp-aligned-64kB: 4194304 kB (100%) anon-thp-aligned-128kB: 0 kB ( 0%) anon-thp-aligned-256kB: 0 kB ( 0%) anon-thp-aligned-512kB: 0 kB ( 0%) anon-thp-aligned-1024kB: 0 kB ( 0%) anon-thp-aligned-2048kB: 0 kB ( 0%) anon-thp-unaligned-16kB: 0 kB ( 0%) anon-thp-unaligned-32kB: 0 kB ( 0%) anon-thp-unaligned-64kB: 0 kB ( 0%) anon-thp-unaligned-128kB: 0 kB ( 0%) anon-thp-unaligned-256kB: 0 kB ( 0%) anon-thp-unaligned-512kB: 0 kB ( 0%) anon-thp-unaligned-1024kB: 0 kB ( 0%) anon-thp-unaligned-2048kB: 0 kB ( 0%) anon-thp-partial: 0 kB ( 0%) file-thp-aligned-16kB: 16 kB ( 1%) file-thp-aligned-32kB: 64 kB ( 5%) file-thp-aligned-64kB: 640 kB (50%) file-thp-aligned-128kB: 128 kB (10%) file-thp-aligned-256kB: 0 kB ( 0%) file-thp-aligned-512kB: 0 kB ( 0%) file-thp-aligned-1024kB: 0 kB ( 0%) file-thp-aligned-2048kB: 0 kB ( 0%) file-thp-unaligned-16kB: 16 kB ( 1%) file-thp-unaligned-32kB: 32 kB ( 3%) file-thp-unaligned-64kB: 64 kB ( 5%) file-thp-unaligned-128kB: 0 kB ( 0%) file-thp-unaligned-256kB: 0 kB ( 0%) file-thp-unaligned-512kB: 0 kB ( 0%) file-thp-unaligned-1024kB: 0 kB ( 0%) file-thp-unaligned-2048kB: 0 kB ( 0%) file-thp-partial: 12 kB ( 1%) anon-cont-aligned-64kB: 4194304 kB (100%) file-cont-aligned-64kB: 768 kB (61%) --8<-- Signed-off-by: Ryan Roberts Tested-by: Barry Song --- I've found this very useful for debugging, and I know others have requested a way to check if mTHP and contpte is working, so thought this might a good short term solution until we figure out how best to add stats in the kernel? Thanks, Ryan tools/mm/Makefile | 9 +- tools/mm/thpmaps | 573 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 578 insertions(+), 4 deletions(-) create mode 100755 tools/mm/thpmaps -- 2.25.1 diff --git a/tools/mm/Makefile b/tools/mm/Makefile index 1c5606cc3334..7bb03606b9ea 100644 --- a/tools/mm/Makefile +++ b/tools/mm/Makefile @@ -3,7 +3,8 @@ # include ../scripts/Makefile.include -TARGETS=page-types slabinfo page_owner_sort +BUILD_TARGETS=page-types slabinfo page_owner_sort +INSTALL_TARGETS = $(BUILD_TARGETS) thpmaps LIB_DIR = ../lib/api LIBS = $(LIB_DIR)/libapi.a @@ -11,9 +12,9 @@ LIBS = $(LIB_DIR)/libapi.a CFLAGS += -Wall -Wextra -I../lib/ -pthread LDFLAGS += $(LIBS) -pthread -all: $(TARGETS) +all: $(BUILD_TARGETS) -$(TARGETS): $(LIBS) +$(BUILD_TARGETS): $(LIBS) $(LIBS): make -C $(LIB_DIR) @@ -29,4 +30,4 @@ sbindir ?= /usr/sbin install: all install -d $(DESTDIR)$(sbindir) - install -m 755 -p $(TARGETS) $(DESTDIR)$(sbindir) + install -m 755 -p $(INSTALL_TARGETS) $(DESTDIR)$(sbindir) diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps new file mode 100755 index 000000000000..af9b19f63eb4 --- /dev/null +++ b/tools/mm/thpmaps @@ -0,0 +1,573 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-only +# Copyright (C) 2024 ARM Ltd. +# +# Utility providing smaps-like output detailing transparent hugepage usage. +# For more info, run: +# ./thpmaps --help +# +# Requires numpy: +# pip3 install numpy + + +import argparse +import collections +import math +import os +import re +import resource +import shutil +import sys +import time +import numpy as np + + +with open('/sys/kernel/mm/transparent_hugepage/hpage_pmd_size') as f: + PAGE_SIZE = resource.getpagesize() + PAGE_SHIFT = int(math.log2(PAGE_SIZE)) + PMD_SIZE = int(f.read()) + PMD_ORDER = int(math.log2(PMD_SIZE / PAGE_SIZE)) + + +def align_forward(v, a): + return (v + (a - 1)) & ~(a - 1) + + +def align_offset(v, a): + return v & (a - 1) + + +def nrkb(nr): + # Convert number of pages to KB. + return (nr << PAGE_SHIFT) >> 10 + + +def odkb(order): + # Convert page order to KB. + return nrkb(1 << order) + + +def cont_ranges_all(arrs): + # Given a list of arrays, find the ranges for which values are monotonically + # incrementing in all arrays. + assert(len(arrs) > 0) + sz = len(arrs[0]) + for arr in arrs: + assert(arr.shape == (sz,)) + r = np.full(sz, 2) + d = np.diff(arrs[0]) == 1 + for dd in [np.diff(arr) == 1 for arr in arrs[1:]]: + d &= dd + r[1:] -= d + r[:-1] -= d + return [np.repeat(arr, r).reshape(-1, 2) for arr in arrs] + + +class ArgException(Exception): + pass + + +class FileIOException(Exception): + pass + + +class BinArrayFile: + # Base class used to read /proc//pagemap and /proc/kpageflags into a + # numpy array. Use inherrited class in a with clause to ensure file is + # closed when it goes out of scope. + def __init__(self, filename, element_size): + self.element_size = element_size + self.filename = filename + self.fd = os.open(self.filename, os.O_RDONLY) + + def cleanup(self): + os.close(self.fd) + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + self.cleanup() + + def _readin(self, offset, buffer): + length = os.preadv(self.fd, (buffer,), offset) + if len(buffer) != length: + raise FileIOException('error: {} failed to read {} bytes at {:x}' + .format(self.filename, len(buffer), offset)) + + def _toarray(self, buf): + assert(self.element_size == 8) + return np.frombuffer(buf, dtype=np.uint64) + + def getv(self, vec): + sz = 0 + for region in vec: + sz += int(region[1] - region[0] + 1) * self.element_size + buf = bytearray(sz) + view = memoryview(buf) + pos = 0 + for region in vec: + offset = int(region[0]) * self.element_size + length = int(region[1] - region[0] + 1) * self.element_size + self._readin(offset, view[pos:pos+length]) + pos += length + return self._toarray(buf) + + def get(self, index, nr=1): + offset = index * self.element_size + length = nr * self.element_size + buf = bytearray(length) + self._readin(offset, buf) + return self._toarray(buf) + + +PM_PAGE_PRESENT = 1 << 63 +PM_PFN_MASK = (1 << 55) - 1 + +class PageMap(BinArrayFile): + # Read ranges of a given pid's pagemap into a numpy array. + def __init__(self, pid='self'): + super().__init__(f'/proc/{pid}/pagemap', 8) + + +KPF_ANON = 1 << 12 +KPF_COMPOUND_HEAD = 1 << 15 +KPF_COMPOUND_TAIL = 1 << 16 + +class KPageFlags(BinArrayFile): + # Read ranges of /proc/kpageflags into a numpy array. + def __init__(self): + super().__init__(f'/proc/kpageflags', 8) + + +VMA = collections.namedtuple('VMA', [ + 'name', + 'start', + 'end', + 'read', + 'write', + 'execute', + 'private', + 'pgoff', + 'major', + 'minor', + 'inode', + 'stats', +]) + +class VMAList: + # A container for VMAs, parsed from /proc//smaps. Iterate over the + # instance to receive VMAs. + head_regex = re.compile(r"^([\da-f]+)-([\da-f]+) ([r-])([w-])([x-])([ps]) ([\da-f]+) ([\da-f]+):([\da-f]+) ([\da-f]+)\s*(.*)$") + kb_item_regex = re.compile(r"(\w+):\s*(\d+)\s*kB") + + def __init__(self, pid='self'): + def is_vma(line): + return self.head_regex.search(line) != None + + def get_vma(line): + m = self.head_regex.match(line) + if m is None: + return None + return VMA( + name=m.group(11), + start=int(m.group(1), 16), + end=int(m.group(2), 16), + read=m.group(3) == 'r', + write=m.group(4) == 'w', + execute=m.group(5) == 'x', + private=m.group(6) == 'p', + pgoff=int(m.group(7), 16), + major=int(m.group(8), 16), + minor=int(m.group(9), 16), + inode=int(m.group(10), 16), + stats={}, + ) + + def get_value(line): + # Currently only handle the KB stats because they are summed for + # --summary. Core code doesn't know how to combine other stats. + exclude = ['KernelPageSize', 'MMUPageSize'] + m = self.kb_item_regex.search(line) + if m: + param = m.group(1) + if param not in exclude: + value = int(m.group(2)) + return param, value + return None, None + + def parse_smaps(file): + vmas = [] + i = 0 + + line = file.readline() + + while True: + if not line: + break + line = line.strip() + + i += 1 + + vma = get_vma(line) + if vma is None: + raise FileIOException(f'error: could not parse line {i}: "{line}"') + + while True: + line = file.readline() + if not line: + break + line = line.strip() + if is_vma(line): + break + + i += 1 + + param, value = get_value(line) + if param: + vma.stats[param] = {'type': None, 'value': value} + + vmas.append(vma) + + return vmas + + with open(f'/proc/{pid}/smaps', 'r') as file: + self.vmas = parse_smaps(file) + + def __iter__(self): + yield from self.vmas + + +def thp_parse(max_order, kpageflags, vfns, pfns, anons, heads): + # Given 4 same-sized arrays representing a range within a page table backed + # by THPs (vfns: virtual frame numbers, pfns: physical frame numbers, anons: + # True if page is anonymous, heads: True if page is head of a THP), return a + # dictionary of statistics describing the mapped THPs. + stats = { + 'file': { + 'partial': 0, + 'aligned': [0] * (max_order + 1), + 'unaligned': [0] * (max_order + 1), + }, + 'anon': { + 'partial': 0, + 'aligned': [0] * (max_order + 1), + 'unaligned': [0] * (max_order + 1), + }, + } + + indexes = np.arange(len(vfns), dtype=np.uint64) + ranges = cont_ranges_all([indexes, vfns, pfns]) + for rindex, rpfn in zip(ranges[0], ranges[2]): + index_next = int(rindex[0]) + index_end = int(rindex[1]) + 1 + pfn_end = int(rpfn[1]) + 1 + + folios = indexes[index_next:index_end][heads[index_next:index_end]] + + # Account pages for any partially mapped THP at the front. In that case, + # the first page of the range is a tail. + nr = (int(folios[0]) if len(folios) else index_end) - index_next + stats['anon' if anons[index_next] else 'file']['partial'] += nr + + # Account pages for any partially mapped THP at the back. In that case, + # the next page after the range is a tail. + if len(folios): + flags = int(kpageflags.get(pfn_end)[0]) + if flags & KPF_COMPOUND_TAIL: + nr = index_end - int(folios[-1]) + folios = folios[:-1] + index_end -= nr + stats['anon' if anons[index_end - 1] else 'file']['partial'] += nr + + # Account fully mapped THPs in the middle of the range. + if len(folios): + folio_nrs = np.append(np.diff(folios), np.uint64(index_end - folios[-1])) + folio_orders = np.log2(folio_nrs).astype(np.uint64) + for index, order in zip(folios, folio_orders): + index = int(index) + order = int(order) + nr = 1 << order + vfn = int(vfns[index]) + align = 'aligned' if align_forward(vfn, nr) == vfn else 'unaligned' + anon = 'anon' if anons[index] else 'file' + stats[anon][align][order] += nr + + rstats = {} + + def flatten_sub(type, subtype, stats): + for od, nr in enumerate(stats[2:], 2): + rstats[f"{type}-thp-{subtype}-{odkb(od)}kB"] = {'type': type, 'value': nrkb(nr)} + + def flatten_type(type, stats): + flatten_sub(type, 'aligned', stats['aligned']) + flatten_sub(type, 'unaligned', stats['unaligned']) + rstats[f"{type}-thp-partial"] = {'type': type, 'value': nrkb(stats['partial'])} + + flatten_type('anon', stats['anon']) + flatten_type('file', stats['file']) + + return rstats + + +def cont_parse(order, vfns, pfns, anons, heads): + # Given 4 same-sized arrays representing a range within a page table backed + # by THPs (vfns: virtual frame numbers, pfns: physical frame numbers, anons: + # True if page is anonymous, heads: True if page is head of a THP), return a + # dictionary of statistics describing the contiguous blocks. + nr_cont = 1 << order + nr_anon = 0 + nr_file = 0 + + ranges = cont_ranges_all([np.arange(len(vfns), dtype=np.uint64), vfns, pfns]) + for rindex, rvfn, rpfn in zip(*ranges): + index_next = int(rindex[0]) + index_end = int(rindex[1]) + 1 + vfn_start = int(rvfn[0]) + pfn_start = int(rpfn[0]) + + if align_offset(pfn_start, nr_cont) != align_offset(vfn_start, nr_cont): + continue + + off = align_forward(vfn_start, nr_cont) - vfn_start + index_next += off + + while index_next + nr_cont <= index_end: + folio_boundary = heads[index_next+1:index_next+nr_cont].any() + if not folio_boundary: + if anons[index_next]: + nr_anon += nr_cont + else: + nr_file += nr_cont + index_next += nr_cont + + return { + f"anon-cont-aligned-{nrkb(nr_cont)}kB": {'type': 'anon', 'value': nrkb(nr_anon)}, + f"file-cont-aligned-{nrkb(nr_cont)}kB": {'type': 'file', 'value': nrkb(nr_file)}, + } + + +def vma_print(vma, pid): + # Prints a VMA instance in a format similar to smaps. The main difference is + # that the pid is included as the first value. + print("{:08x} {:016x}-{:016x} {}{}{}{} {:08x} {:02x}:{:02x} {:08x} {}" + .format( + pid, vma.start, vma.end, + 'r' if vma.read else '-', 'w' if vma.write else '-', + 'x' if vma.execute else '-', 'p' if vma.private else 's', + vma.pgoff, vma.major, vma.minor, vma.inode, vma.name + )) + + +def stats_print(stats, tot_anon, tot_file, inc_empty): + # Print a statistics dictionary. + label_field = 32 + for label, stat in stats.items(): + type = stat['type'] + value = stat['value'] + if value or inc_empty: + pad = max(0, label_field - len(label) - 1) + if type == 'anon': + percent = f' ({value / tot_anon:3.0%})' + elif type == 'file': + percent = f' ({value / tot_file:3.0%})' + else: + percent = '' + print(f"{label}:{' ' * pad}{value:8} kB{percent}") + + +def vma_parse(vma, pagemap, kpageflags, contorders): + # Generate thp and cont statistics for a single VMA. + start = vma.start >> PAGE_SHIFT + end = vma.end >> PAGE_SHIFT + + pmes = pagemap.get(start, end - start) + present = pmes & PM_PAGE_PRESENT != 0 + pfns = pmes & PM_PFN_MASK + pfns = pfns[present] + vfns = np.arange(start, end, dtype=np.uint64) + vfns = vfns[present] + + flags = kpageflags.getv(cont_ranges_all([pfns])[0]) + anons = flags & KPF_ANON != 0 + heads = flags & KPF_COMPOUND_HEAD != 0 + tails = flags & KPF_COMPOUND_TAIL != 0 + thps = heads | tails + + tot_anon = np.count_nonzero(anons) + tot_file = np.size(anons) - tot_anon + tot_anon = nrkb(tot_anon) + tot_file = nrkb(tot_file) + + vfns = vfns[thps] + pfns = pfns[thps] + anons = anons[thps] + heads = heads[thps] + + thpstats = thp_parse(PMD_ORDER, kpageflags, vfns, pfns, anons, heads) + contstats = [cont_parse(order, vfns, pfns, anons, heads) for order in contorders] + + return { + **thpstats, + **{k: v for s in contstats for k, v in s.items()} + }, tot_anon, tot_file + + +def do_main(args): + pids = set() + summary = {} + summary_anon = 0 + summary_file = 0 + + if args.cgroup: + with open(f'{args.cgroup}/cgroup.procs') as pidfile: + for line in pidfile.readlines(): + pids.add(int(line.strip())) + else: + pids.add(args.pid) + + for pid in pids: + try: + with PageMap(pid) as pagemap: + with KPageFlags() as kpageflags: + for vma in VMAList(pid): + if (vma.read or vma.write or vma.execute) and vma.stats['Rss']['value'] > 0: + stats, vma_anon, vma_file = vma_parse(vma, pagemap, kpageflags, args.cont) + else: + stats = {} + vma_anon = 0 + vma_file = 0 + if args.inc_smaps: + stats = {**vma.stats, **stats} + if args.summary: + for k, v in stats.items(): + if k in summary: + assert(summary[k]['type'] == v['type']) + summary[k]['value'] += v['value'] + else: + summary[k] = v + summary_anon += vma_anon + summary_file += vma_file + else: + vma_print(vma, pid) + stats_print(stats, vma_anon, vma_file, args.inc_empty) + except FileNotFoundError: + if not args.cgroup: + raise + except ProcessLookupError: + if not args.cgroup: + raise + + if args.summary: + stats_print(summary, summary_anon, summary_file, args.inc_empty) + + +def main(): + def formatter(prog): + width = shutil.get_terminal_size().columns + width -= 2 + width = min(80, width) + return argparse.HelpFormatter(prog, width=width) + + def size2order(human): + units = {"K": 2**10, "M": 2**20, "G": 2**30} + unit = 1 + if human[-1] in units: + unit = units[human[-1]] + human = human[:-1] + try: + size = int(human) + except ValueError: + raise ArgException('error: --cont value must be integer size with optional KMG unit') + size *= unit + order = int(math.log2(size / PAGE_SIZE)) + if order < 1: + raise ArgException('error: --cont value must be size of at least 2 pages') + if (1 << order) * PAGE_SIZE != size: + raise ArgException('error: --cont value must be size of power-of-2 pages') + return order + + parser = argparse.ArgumentParser(formatter_class=formatter, + description="""Prints information about how transparent huge pages are + mapped to a specified process or cgroup. + + Shows statistics for fully-mapped THPs of every size, mapped + both naturally aligned and unaligned for both file and + anonymous memory. See + [anon|file]-thp-[aligned|unaligned]-kB keys. + + Shows statistics for mapped pages that belong to a THP but + which are not fully mapped. See [anon|file]-thp-partial + keys. + + Optionally shows statistics for naturally aligned, + contiguous blocks of memory of a specified size (when --cont + is provided). See [anon|file]-cont-aligned-kB keys. + + Statistics are shown in kB and as a percentage of either + total anon or file memory as appropriate.""", + epilog="""Requires root privilege to access pagemap and kpageflags.""") + + parser.add_argument('--pid', + metavar='pid', required=False, type=int, + help="""Process id of the target process. Exactly one of --pid and + --cgroup must be provided.""") + + parser.add_argument('--cgroup', + metavar='path', required=False, + help="""Path to the target cgroup in sysfs. Iterates over every pid in + the cgroup. Exactly one of --pid and --cgroup must be provided.""") + + parser.add_argument('--summary', + required=False, default=False, action='store_true', + help="""Sum the per-vma statistics to provide a summary over the whole + process or cgroup.""") + + parser.add_argument('--cont', + metavar='size[KMG]', required=False, default=[], action='append', + help="""Adds anon and file stats for naturally aligned, contiguously + mapped blocks of the specified size. May be issued multiple times to + track multiple sized blocks. Useful to infer e.g. arm64 contpte and + hpa mappings. Size must be a power-of-2 number of pages.""") + + parser.add_argument('--inc-smaps', + required=False, default=False, action='store_true', + help="""Include all numerical, additive /proc//smaps stats in the + output.""") + + parser.add_argument('--inc-empty', + required=False, default=False, action='store_true', + help="""Show all statistics including those whose value is 0.""") + + parser.add_argument('--periodic', + metavar='sleep_ms', required=False, type=int, + help="""Run in a loop, polling every sleep_ms milliseconds.""") + + args = parser.parse_args() + + try: + if (args.pid and args.cgroup) or \ + (not args.pid and not args.cgroup): + raise ArgException("error: Exactly one of --pid and --cgroup must be provided.") + + args.cont = [size2order(cont) for cont in args.cont] + except ArgException as e: + parser.print_usage() + raise + + if args.periodic: + while True: + do_main(args) + print() + time.sleep(args.periodic / 1000) + else: + do_main(args) + + +if __name__ == "__main__": + try: + main() + except Exception as e: + prog = os.path.basename(sys.argv[0]) + print(f'{prog}: {e}') + exit(1)