From patchwork Tue Dec 26 20:01:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 13505026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E98BC46CD3 for ; Tue, 26 Dec 2023 20:02:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4EB726B0072; Tue, 26 Dec 2023 15:02:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 472116B0075; Tue, 26 Dec 2023 15:02:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 306096B0074; Tue, 26 Dec 2023 15:02:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1EDDF6B0071 for ; Tue, 26 Dec 2023 15:02:12 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DA95A8054D for ; Tue, 26 Dec 2023 20:02:11 +0000 (UTC) X-FDA: 81610040862.26.C718F5B Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by imf09.hostedemail.com (Postfix) with ESMTP id 29C1F14001B for ; Tue, 26 Dec 2023 20:02:09 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=kqDzpsP0; spf=pass (imf09.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.45 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703620929; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=O3PECSz7M6LwJTPdrkLY+55pPQo2vTN8jkKqtc7LNKw=; b=tdyW9EOcYTN1CfErnnQZBcqrYoonN67pb6GThQe62KN2gwQ7ra37QVQOpHBqmsdZZlxCN9 qNJKY2DVlWiDW2L15DkMUaBo+th+1wiEIvB8i4oVs13XdMvi0SvKq7Bn+jSb+JF5kc9Tlg dx7jir52Nnzq9ASoIaMuJtK5bHpPil0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703620929; a=rsa-sha256; cv=none; b=wBZ6w3i0lgF09TX9822cYZq1daTfrisDordMdP+50APRZLvcjtgQd1B/G18606fCOm7rt0 uAbdIEtZfKbMbXtXdSIclg02JK7FyO1hTooAJEZzdI78ZR9TAmfCkBZ9dDgvlmANHoagjK 7Z77rzWfWh4L8jAeoW4AwZC0Sl1TX+I= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=soleen.com header.s=google header.b=kqDzpsP0; spf=pass (imf09.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.219.45 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com; dmarc=none Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-67f8a5ed1a0so31781306d6.2 for ; Tue, 26 Dec 2023 12:02:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1703620928; x=1704225728; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=O3PECSz7M6LwJTPdrkLY+55pPQo2vTN8jkKqtc7LNKw=; b=kqDzpsP05niz+jKjIOmbVhT8gL4ulPjhqWnTP6luIl15HvFkBvvaL6UBcSxDOyJb/V LBpsYMH5GoO9c7FVtIE090I8daY9ByPrp5OW2S/ML6DxpzDlZ48e/ZYEkcbgr/oZXvbT Oqb+BKl0LH0e9BmkIS26OK3V6YC/4z2X8I4mvx6U2vt5geEZYdIwe7qdfX8niM7N98BI SdfUARN+vqeBF/WxwMekFgWOiEqN9BtDZ2VgWiIh6ZnSP4JHH8TeldQ2ii7V1W117Trq xDiYAgdhKiJnDCkmnSXBPk4EZ6jalUIWjHY/hLOr9rnMUJoYo0partkExec6PC+c1KEL wUZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703620928; x=1704225728; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=O3PECSz7M6LwJTPdrkLY+55pPQo2vTN8jkKqtc7LNKw=; b=iAAAnCXwwxQR+KVD+TJ7mWTtbBYvvORdYtrBa2rAyDKyaM2JpyKmXd6QNdtJBIgGz/ fo8BB0PtYHObJAZeHoldCsOxjBzgk4s+bcw2baBwQGjMk78qNPddKOPyVmahmRRhAyWr aKH0H9wICc2r3r5NZRl99PVJ65ZX4oFM+yrK6JuQE7M3frXeutLUvs5B8Evb7ywASQ7w oQf2tT4EB0+HwHpg7hnJI1orvsT2FfdBg6FX4dP3TE4Vja4SPSxipS3b3KI+q0mCy9Ap tiWyWquZtAUR0mkhIfwY0T0ES7uO5NC9ZTU55cG0Iem68Rw+Sxde3lXjuyp6X9CBwIIJ Y01g== X-Gm-Message-State: AOJu0Yz7XuFExKD3sEujsiXkFkFi01fuVzF6gKO6znfaaLTWggqu1V/x 3Hm7e4WvWBZQ0io2ra2TBBVS8hpUyQA++A== X-Google-Smtp-Source: AGHT+IEQhfs2w29Y4oEHKMdn2lx6fbgiHsN/7DqKMKEEg4mMRcO6NOtJWlf/527QYsbumDRKw7qTlA== X-Received: by 2002:a05:6214:769:b0:67f:3d0e:783f with SMTP id f9-20020a056214076900b0067f3d0e783fmr13773008qvz.54.1703620928161; Tue, 26 Dec 2023 12:02:08 -0800 (PST) Received: from soleen.c.googlers.com.com (55.87.194.35.bc.googleusercontent.com. [35.194.87.55]) by smtp.gmail.com with ESMTPSA id t5-20020a0cf985000000b0067f696f412esm4894539qvn.112.2023.12.26.12.02.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 12:02:07 -0800 (PST) From: Pasha Tatashin To: akpm@linux-foundation.org, alim.akhtar@samsung.com, alyssa@rosenzweig.io, asahi@lists.linux.dev, baolu.lu@linux.intel.com, bhelgaas@google.com, cgroups@vger.kernel.org, corbet@lwn.net, david@redhat.com, dwmw2@infradead.org, hannes@cmpxchg.org, heiko@sntech.de, iommu@lists.linux.dev, jernej.skrabec@gmail.com, jonathanh@nvidia.com, joro@8bytes.org, krzysztof.kozlowski@linaro.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, lizefan.x@bytedance.com, marcan@marcan.st, mhiramat@kernel.org, m.szyprowski@samsung.com, pasha.tatashin@soleen.com, paulmck@kernel.org, rdunlap@infradead.org, robin.murphy@arm.com, samuel@sholland.org, suravee.suthikulpanit@amd.com, sven@svenpeter.dev, thierry.reding@gmail.com, tj@kernel.org, tomas.mudrunka@gmail.com, vdumpa@nvidia.com, wens@csie.org, will@kernel.org, yu-cheng.yu@intel.com, rientjes@google.com Subject: [PATCH v3 00/10] IOMMU memory observability Date: Tue, 26 Dec 2023 20:01:55 +0000 Message-ID: <20231226200205.562565-1-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog MIME-Version: 1.0 X-Stat-Signature: c9segwkbwn1xd6km1gwjihihsiieo1ye X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 29C1F14001B X-Rspam-User: X-HE-Tag: 1703620929-629165 X-HE-Meta: U2FsdGVkX1/SUzDKHpvuDM+vWPwe3kdZnfY+MZ23QCAWKwv1D4a8ZfXCq8GbjVGepTqGHnPoRwdUboDD9rcYoxP/ZosXsJq+V6w6z+7w1VB8BpuFcd9vSYr1wgKq0m7hVmGWQpP4Mmh+/xz5vJwW6Pj5NqsYwP3l0MBxWKX//CD48V7wByZkCqADawZXQBbBV7uEONa0QaC0wanwQk9GMqYFOtt2cgY8rNZYXcaRxH+KGchxqJvupm+Gr1/IslPEIA8FZq/aYgWn7vhOSVGB8dcirjvYdH078EzT0mEY0YftjzxBIHQlhim/okFxO2qo1ZErZ3iggz9bhbFv84sSGwvSOyOeFhRx+BaaIiU6ttPZV848I5sPZwkwMZHXTg0nt9L9Q6WRUF4qRYoRIuGcoDbZLBiKMbViu2X4ZRPBO9bV/3bZY390UZCdPt3gP5+jHTqvSb2Rv+VFzpR9WiNE51RnsMwhnHvgs0Nz7lNYQutUsjpI6+zhYClu/G3GE1A9sHiizFoHyNztffmBSZhLIhEgSmhiPR0j0skCFUpdUtYEiPZUnik+FXv3cE71tQmHbBMJTu+181BhYnBI8BCMBfVW36ciJlCxdltgFott5nw22oSaA+nDEY/VTlSpb4FZyLm4d9n+bN6n/XENNET7YxKiv9P7ZJ1v0oIqV4q+uiHoc7mH8HFlxGvLbTWnab9/z3EJ2Xp8EiSuWwI2pd/B5xVvCJhYb9Kp4tA1/8SEL5Nb425V27o+PuaPzzM50PphKvK3MVcylmeTZ/4M2wGVbN+v2jj9prA2Yh8mpHs5XztR8CIKDma+yQQWtXl03Xh56fz1v0FTHXMBp6HgT5b3wWV9ybYwiHAWFrB9FGFbRgqkUf5ubI4OKmhoNQmHHs9Uwel0sh3D+EesVA7g+XnMc/TWn8tWK+94v3uY2O8rVCpxoYyjNH5MHtrOtmyEeygqfD9tp7RsUiCwI2iqZHO HEqtpD7T IDCKO8xWxqECNm57jth0OkS7XNjnmM143xQ1svgXjpGXwtvNHvAiOmED3oYeaMoKpBZW7DIdQs1xfS0a5o8k2vHxkNwPRtuTVUzne7NpLKAsWdQgTkmCkOcJZaQHWMupNhWSsM3I5hIgGsPE9aKc1ySm/TM7fItnG6vHd0zH7eILpW1MjR8lOdps5RXgrjX9a+b+sk4TWxXBNt8g4IRiaDGheTgS/asNApSEhs9DI5pmCnqN2CuvdsoT++6c4CwKuZvNcUuOhzaI+3hZqnJkkMFavNplRD/ck6ihzJR4QC1zS8p/AHxHJEud0HM2GoUCcc6oaXhsaX9HQor6yCBhBsZhQWadI+6kni2Ya2t2YLggL18N8pK8PCXjhneX0DpNRkapt//6UFUihPPaJ5KBGnkEWRebMgSzcY6lP0U3jkucuFDF2CqozyaxmpN4D/LPROmKklKD8RahXS6HnEeAecKA6Ajt86tSK47/FVXNDQDwzXvj+TZ4ZESFupWT5Xxiv4K3/8vlVyhRvv7bfjKuQes+8AVJoJa/DhQ+e45s/IhxpPKxQ/sbOyt3eC+CMUERIkp7O8fiNZY/nz20l0m4V1fkObchWX2LxXd4nLMP9Hqpx1nQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: ---------------------------------------------------------------------- Changelog ---------------------------------------------------------------------- v3: - Sync with v6.7-rc7 - Addressed comments from David Rientjes: s/pages/page/, added unlikely() into the branches, expanded comment for iommu_free_pages_list(). - Added Acked-bys: David Rientjes v2: - Added Reviewed-by Janne Grunau - Sync with 6.7.0-rc3, 3b47bc037bd44f142ac09848e8d3ecccc726be99 - Separated form the series patches: vhost-vdpa: account iommu allocations https://lore.kernel.org/all/20231130200447.2319543-1-pasha.tatashin@soleen.com vfio: account iommu allocations https://lore.kernel.org/all/20231130200900.2320829-1-pasha.tatashin@soleen.com as suggested by Jason Gunthorpe - Fixed SPARC build issue detected by kernel test robot - Drop the following patches as they do account iommu page tables: iommu/dma: use page allocation function provided by iommu-pages.h iommu/fsl: use page allocation function provided by iommu-pages.h iommu/iommufd: use page allocation function provided by iommu-pages.h as suggested by Robin Murphy. These patches are not related to IOMMU page tables. We might need to do a separate work to support DMA observability. - Remove support iommu/io-pgtable-arm-v7s as the 2nd level pages are under a page size, thanks Robin Murphy for pointing this out. ---------------------------------------------------------------------- Description ---------------------------------------------------------------------- IOMMU subsystem may contain state that is in gigabytes. Majority of that state is iommu page tables. Yet, there is currently, no way to observe how much memory is actually used by the iommu subsystem. This patch series solves this problem by adding both observability to all pages that are allocated by IOMMU, and also accountability, so admins can limit the amount if via cgroups. The system-wide observability is using /proc/meminfo: SecPageTables: 438176 kB Contains IOMMU and KVM memory. Per-node observability: /sys/devices/system/node/nodeN/meminfo Node N SecPageTables: 422204 kB Contains IOMMU and KVM memory memory in the given NUMA node. Per-node IOMMU only observability: /sys/devices/system/node/nodeN/vmstat nr_iommu_pages 105555 Contains number of pages IOMMU allocated in the given node. Accountability: using sec_pagetables cgroup-v2 memory.stat entry. With the change, iova_stress[1] stops as limit is reached: # ./iova_stress iova space: 0T free memory: 497G iova space: 1T free memory: 495G iova space: 2T free memory: 493G iova space: 3T free memory: 491G stops as limit is reached. This series encorporates suggestions that came from the discussion at LPC [2]. ---------------------------------------------------------------------- [1] https://github.com/soleen/iova_stress [2] https://lpc.events/event/17/contributions/1466 ---------------------------------------------------------------------- Previous versions v1: https://lore.kernel.org/all/20231128204938.1453583-1-pasha.tatashin@soleen.com v2: https://lore.kernel.org/linux-mm/20231130201504.2322355-1-pasha.tatashin@soleen.com ---------------------------------------------------------------------- Pasha Tatashin (10): iommu/vt-d: add wrapper functions for page allocations iommu/amd: use page allocation function provided by iommu-pages.h iommu/io-pgtable-arm: use page allocation function provided by iommu-pages.h iommu/io-pgtable-dart: use page allocation function provided by iommu-pages.h iommu/exynos: use page allocation function provided by iommu-pages.h iommu/rockchip: use page allocation function provided by iommu-pages.h iommu/sun50i: use page allocation function provided by iommu-pages.h iommu/tegra-smmu: use page allocation function provided by iommu-pages.h iommu: observability of the IOMMU allocations iommu: account IOMMU allocated memory Documentation/admin-guide/cgroup-v2.rst | 2 +- Documentation/filesystems/proc.rst | 4 +- drivers/iommu/amd/amd_iommu.h | 8 - drivers/iommu/amd/init.c | 91 +++++---- drivers/iommu/amd/io_pgtable.c | 13 +- drivers/iommu/amd/io_pgtable_v2.c | 20 +- drivers/iommu/amd/iommu.c | 13 +- drivers/iommu/exynos-iommu.c | 14 +- drivers/iommu/intel/dmar.c | 10 +- drivers/iommu/intel/iommu.c | 47 ++--- drivers/iommu/intel/iommu.h | 2 - drivers/iommu/intel/irq_remapping.c | 10 +- drivers/iommu/intel/pasid.c | 12 +- drivers/iommu/intel/svm.c | 7 +- drivers/iommu/io-pgtable-arm.c | 7 +- drivers/iommu/io-pgtable-dart.c | 37 ++-- drivers/iommu/iommu-pages.h | 236 ++++++++++++++++++++++++ drivers/iommu/rockchip-iommu.c | 14 +- drivers/iommu/sun50i-iommu.c | 7 +- drivers/iommu/tegra-smmu.c | 18 +- include/linux/mmzone.h | 5 +- mm/vmstat.c | 3 + 22 files changed, 395 insertions(+), 185 deletions(-) create mode 100644 drivers/iommu/iommu-pages.h Tested-by: Bagas Sanjaya