From patchwork Thu Jun 27 15:47:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maarten Lankhorst X-Patchwork-Id: 13714714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0231DC30653 for ; Thu, 27 Jun 2024 15:48:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 57C186B008C; Thu, 27 Jun 2024 11:48:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 476D26B0096; Thu, 27 Jun 2024 11:48:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 203FF6B008C; Thu, 27 Jun 2024 11:48:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EB2DC6B008A for ; Thu, 27 Jun 2024 11:48:06 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9292D120849 for ; Thu, 27 Jun 2024 15:48:06 +0000 (UTC) X-FDA: 82277099772.18.4CA14FC Received: from mblankhorst.nl (lankhorst.se [141.105.120.124]) by imf18.hostedemail.com (Postfix) with ESMTP id 732491C0014 for ; Thu, 27 Jun 2024 15:48:04 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf18.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719503272; a=rsa-sha256; cv=none; b=gERVgTwrJSZykI8Gcx+CYRd2NJzTkxwjboUMjROk9F92jtS2TDnYzKrEquBUYgtmsUCR2/ Fsus8K+OuT7aPKitggeBW2z2n31adK6V4zT6dVOKAcJ0JcWeEvSL1nGBqOGXZLa68zEAS0 w7psHDI3+65REo9KfsciChTj2/1qLTk= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf18.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719503272; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ilFQkRNZg4JP3i5XiY9uml6zuzXeFxrKRQ2c7RYg/tI=; b=Oa5moPhjlgmimt3ytuhHV/Z/dxBhPhPWpxCoX9v0y1qEw2mAbof6usHNFF7pIk1daJLlbZ rCGjECr1VZNNQPaqYP5Oo4Aby4+6p7deg7i44NHsvQMzEUoWHvgw13pXa09R8SRSWt4LXo MlnVqWZjPSDA3JHk+mPC4T5LRP+HYGI= From: Maarten Lankhorst To: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song Cc: Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org, Maarten Lankhorst Subject: [RFC PATCH 1/6] mm/page_counter: Move calculating protection values to page_counter Date: Thu, 27 Jun 2024 17:47:20 +0200 Message-ID: <20240627154754.74828-2-maarten.lankhorst@linux.intel.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 732491C0014 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ndjj4md3ckqbcr9xdpxaradmbionicof X-HE-Tag: 1719503284-122533 X-HE-Meta: U2FsdGVkX1+9adFYz+skTNMlo2/keM2Gu2KiZ2YoUCVrr/oPc0fGj320dCIS+TatCV5cQjErhpYbDx0qxl7N5fMhIg6jUKXFdXgY2E2quTl7dobmpbvZlBKjlAUSSCG900YKKrWaADuxMIO5SoCtIbel1eKxx+dc+5n93rNi1nWiLSZ12wI4m6Ww+3gXdkVBbGMgFIrCaLhs58neYAc7oKM29vPxVJpEjTRJjfpaAGoNEb6M0MaRRqgZ/6aKmXSyDvhKt1ITp7cXyKByGZ4HqjhsCQu852Il6Qdai6nkN/ER+dvHnzDqgt76UoATdEl1q5FZ3P++ZMUg22BiArP4brZUEro+CayN9LXyRUesgxrXF+DTpRlLV1uhYB4F+cv52SLfsAnmeF28v4aG+rbzmfPmpDla/kDDrE8VEBVLe6Ks7nmBGw5wDiYb6y83YvgteNUB74NruGHXOudzZx0kLyKcRCQiBhpwp6Dsxt/f/cYFgQqoftTaxl49jJIqLNs1bG8ROegL1B3aQPN2OFyq/IlTwGvDw0666YRVhLAsb9oYUYquowwb383sjgh6i46pVVTjRRttAiPTZxNFMpeJJTdQ267819mLmnlJIVdXjwlJ5xSyhQZ8LA8H+UZJbSuxw6wUH78eujMPGQdfjpJUcCLHrtWg1UEDkL/vgRZBII2LyhQ/dR0hKVY8bt9evPQ32d9TV1ljJiiBIVph0iclRervqv2NjLmIsP0HyJGsbUD+fPAGJv7XfkKJO+BHEyUGkar0RWQe8bQPKCrpcA8mmMY1ZGm/Co1bO76Yap2RcYc1bCk7AaisCrI5ax76tbxWPJknXk61zjam+ndtbpOjK+HMRJzD5fRTZA+Nm6Uq+DsEvBK5+XiKXnwnF03qQ89iZrojiqGYksGIxB0rBIZERHCJRTnKfg/GO1YjeoOENF3xZbJMI47I3W0B5AMa7kxVWNkYq6wGD/ESxn80KqT /9e3kKMI Ru8epzrRHjgglqvftKUXVS83QLV5ZhEF35Nv6lM9qVsnkS+ekYzsVm5EIfdigLTKkGrNeY2mtSaCEJ4Sp6YZwVSAy4QHLgilD+MURnGdXxWqhwzrhuXz0RmosGRYdAmE+mbwf/sJG5VicCcS9g+RXOd5uOG/0iZMIG7wDOMdY6tFhvhPheCGIMkWl2Q3cQy8JfpQ6B0mJhZ0eSjQDzXdbyLW4CtGueUeeGnxTgWH4duqMC0jyi+OrnnoRWLT4Fo1RiTbaqcvfL93g8nReOyfhDj8rakVG9hN7or4x X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: It's a lot of math, and there is nothing memcontrol specific about it. This makes it easier to use inside of the drm cgroup controller. Signed-off-by: Maarten Lankhorst Acked-by: Roman Gushchin Acked-by: Shakeel Butt --- include/linux/page_counter.h | 4 + mm/memcontrol.c | 154 +------------------------------ mm/page_counter.c | 173 +++++++++++++++++++++++++++++++++++ 3 files changed, 180 insertions(+), 151 deletions(-) diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h index 8cd858d912c4..904c52f97284 100644 --- a/include/linux/page_counter.h +++ b/include/linux/page_counter.h @@ -81,4 +81,8 @@ static inline void page_counter_reset_watermark(struct page_counter *counter) counter->watermark = page_counter_read(counter); } +void page_counter_calculate_protection(struct page_counter *root, + struct page_counter *counter, + bool recursive_protection); + #endif /* _LINUX_PAGE_COUNTER_H */ diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 71fe2a95b8bd..9454e1a3120e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7316,122 +7316,6 @@ struct cgroup_subsys memory_cgrp_subsys = { .early_init = 0, }; -/* - * This function calculates an individual cgroup's effective - * protection which is derived from its own memory.min/low, its - * parent's and siblings' settings, as well as the actual memory - * distribution in the tree. - * - * The following rules apply to the effective protection values: - * - * 1. At the first level of reclaim, effective protection is equal to - * the declared protection in memory.min and memory.low. - * - * 2. To enable safe delegation of the protection configuration, at - * subsequent levels the effective protection is capped to the - * parent's effective protection. - * - * 3. To make complex and dynamic subtrees easier to configure, the - * user is allowed to overcommit the declared protection at a given - * level. If that is the case, the parent's effective protection is - * distributed to the children in proportion to how much protection - * they have declared and how much of it they are utilizing. - * - * This makes distribution proportional, but also work-conserving: - * if one cgroup claims much more protection than it uses memory, - * the unused remainder is available to its siblings. - * - * 4. Conversely, when the declared protection is undercommitted at a - * given level, the distribution of the larger parental protection - * budget is NOT proportional. A cgroup's protection from a sibling - * is capped to its own memory.min/low setting. - * - * 5. However, to allow protecting recursive subtrees from each other - * without having to declare each individual cgroup's fixed share - * of the ancestor's claim to protection, any unutilized - - * "floating" - protection from up the tree is distributed in - * proportion to each cgroup's *usage*. This makes the protection - * neutral wrt sibling cgroups and lets them compete freely over - * the shared parental protection budget, but it protects the - * subtree as a whole from neighboring subtrees. - * - * Note that 4. and 5. are not in conflict: 4. is about protecting - * against immediate siblings whereas 5. is about protecting against - * neighboring subtrees. - */ -static unsigned long effective_protection(unsigned long usage, - unsigned long parent_usage, - unsigned long setting, - unsigned long parent_effective, - unsigned long siblings_protected) -{ - unsigned long protected; - unsigned long ep; - - protected = min(usage, setting); - /* - * If all cgroups at this level combined claim and use more - * protection than what the parent affords them, distribute - * shares in proportion to utilization. - * - * We are using actual utilization rather than the statically - * claimed protection in order to be work-conserving: claimed - * but unused protection is available to siblings that would - * otherwise get a smaller chunk than what they claimed. - */ - if (siblings_protected > parent_effective) - return protected * parent_effective / siblings_protected; - - /* - * Ok, utilized protection of all children is within what the - * parent affords them, so we know whatever this child claims - * and utilizes is effectively protected. - * - * If there is unprotected usage beyond this value, reclaim - * will apply pressure in proportion to that amount. - * - * If there is unutilized protection, the cgroup will be fully - * shielded from reclaim, but we do return a smaller value for - * protection than what the group could enjoy in theory. This - * is okay. With the overcommit distribution above, effective - * protection is always dependent on how memory is actually - * consumed among the siblings anyway. - */ - ep = protected; - - /* - * If the children aren't claiming (all of) the protection - * afforded to them by the parent, distribute the remainder in - * proportion to the (unprotected) memory of each cgroup. That - * way, cgroups that aren't explicitly prioritized wrt each - * other compete freely over the allowance, but they are - * collectively protected from neighboring trees. - * - * We're using unprotected memory for the weight so that if - * some cgroups DO claim explicit protection, we don't protect - * the same bytes twice. - * - * Check both usage and parent_usage against the respective - * protected values. One should imply the other, but they - * aren't read atomically - make sure the division is sane. - */ - if (!(cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT)) - return ep; - if (parent_effective > siblings_protected && - parent_usage > siblings_protected && - usage > protected) { - unsigned long unclaimed; - - unclaimed = parent_effective - siblings_protected; - unclaimed *= usage - protected; - unclaimed /= parent_usage - siblings_protected; - - ep += unclaimed; - } - - return ep; -} - /** * mem_cgroup_calculate_protection - check if memory consumption is in the normal range * @root: the top ancestor of the sub-tree being checked @@ -7443,8 +7327,8 @@ static unsigned long effective_protection(unsigned long usage, void mem_cgroup_calculate_protection(struct mem_cgroup *root, struct mem_cgroup *memcg) { - unsigned long usage, parent_usage; - struct mem_cgroup *parent; + bool recursive_protection = + cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_RECURSIVE_PROT; if (mem_cgroup_disabled()) return; @@ -7452,39 +7336,7 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root, if (!root) root = root_mem_cgroup; - /* - * Effective values of the reclaim targets are ignored so they - * can be stale. Have a look at mem_cgroup_protection for more - * details. - * TODO: calculation should be more robust so that we do not need - * that special casing. - */ - if (memcg == root) - return; - - usage = page_counter_read(&memcg->memory); - if (!usage) - return; - - parent = parent_mem_cgroup(memcg); - - if (parent == root) { - memcg->memory.emin = READ_ONCE(memcg->memory.min); - memcg->memory.elow = READ_ONCE(memcg->memory.low); - return; - } - - parent_usage = page_counter_read(&parent->memory); - - WRITE_ONCE(memcg->memory.emin, effective_protection(usage, parent_usage, - READ_ONCE(memcg->memory.min), - READ_ONCE(parent->memory.emin), - atomic_long_read(&parent->memory.children_min_usage))); - - WRITE_ONCE(memcg->memory.elow, effective_protection(usage, parent_usage, - READ_ONCE(memcg->memory.low), - READ_ONCE(parent->memory.elow), - atomic_long_read(&parent->memory.children_low_usage))); + page_counter_calculate_protection(&root->memory, &memcg->memory, recursive_protection); } static int charge_memcg(struct folio *folio, struct mem_cgroup *memcg, diff --git a/mm/page_counter.c b/mm/page_counter.c index db20d6452b71..8ee49cbf71be 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -262,3 +262,176 @@ int page_counter_memparse(const char *buf, const char *max, return 0; } + + +/* + * This function calculates an individual page counter's effective + * protection which is derived from its own memory.min/low, its + * parent's and siblings' settings, as well as the actual memory + * distribution in the tree. + * + * The following rules apply to the effective protection values: + * + * 1. At the first level of reclaim, effective protection is equal to + * the declared protection in memory.min and memory.low. + * + * 2. To enable safe delegation of the protection configuration, at + * subsequent levels the effective protection is capped to the + * parent's effective protection. + * + * 3. To make complex and dynamic subtrees easier to configure, the + * user is allowed to overcommit the declared protection at a given + * level. If that is the case, the parent's effective protection is + * distributed to the children in proportion to how much protection + * they have declared and how much of it they are utilizing. + * + * This makes distribution proportional, but also work-conserving: + * if one counter claims much more protection than it uses memory, + * the unused remainder is available to its siblings. + * + * 4. Conversely, when the declared protection is undercommitted at a + * given level, the distribution of the larger parental protection + * budget is NOT proportional. A counter's protection from a sibling + * is capped to its own memory.min/low setting. + * + * 5. However, to allow protecting recursive subtrees from each other + * without having to declare each individual counter's fixed share + * of the ancestor's claim to protection, any unutilized - + * "floating" - protection from up the tree is distributed in + * proportion to each counter's *usage*. This makes the protection + * neutral wrt sibling cgroups and lets them compete freely over + * the shared parental protection budget, but it protects the + * subtree as a whole from neighboring subtrees. + * + * Note that 4. and 5. are not in conflict: 4. is about protecting + * against immediate siblings whereas 5. is about protecting against + * neighboring subtrees. + */ +static unsigned long effective_protection(unsigned long usage, + unsigned long parent_usage, + unsigned long setting, + unsigned long parent_effective, + unsigned long siblings_protected, + bool recursive_protection) +{ + unsigned long protected; + unsigned long ep; + + protected = min(usage, setting); + /* + * If all cgroups at this level combined claim and use more + * protection than what the parent affords them, distribute + * shares in proportion to utilization. + * + * We are using actual utilization rather than the statically + * claimed protection in order to be work-conserving: claimed + * but unused protection is available to siblings that would + * otherwise get a smaller chunk than what they claimed. + */ + if (siblings_protected > parent_effective) + return protected * parent_effective / siblings_protected; + + /* + * Ok, utilized protection of all children is within what the + * parent affords them, so we know whatever this child claims + * and utilizes is effectively protected. + * + * If there is unprotected usage beyond this value, reclaim + * will apply pressure in proportion to that amount. + * + * If there is unutilized protection, the cgroup will be fully + * shielded from reclaim, but we do return a smaller value for + * protection than what the group could enjoy in theory. This + * is okay. With the overcommit distribution above, effective + * protection is always dependent on how memory is actually + * consumed among the siblings anyway. + */ + ep = protected; + + /* + * If the children aren't claiming (all of) the protection + * afforded to them by the parent, distribute the remainder in + * proportion to the (unprotected) memory of each cgroup. That + * way, cgroups that aren't explicitly prioritized wrt each + * other compete freely over the allowance, but they are + * collectively protected from neighboring trees. + * + * We're using unprotected memory for the weight so that if + * some cgroups DO claim explicit protection, we don't protect + * the same bytes twice. + * + * Check both usage and parent_usage against the respective + * protected values. One should imply the other, but they + * aren't read atomically - make sure the division is sane. + */ + if (!recursive_protection) + return ep; + + if (parent_effective > siblings_protected && + parent_usage > siblings_protected && + usage > protected) { + unsigned long unclaimed; + + unclaimed = parent_effective - siblings_protected; + unclaimed *= usage - protected; + unclaimed /= parent_usage - siblings_protected; + + ep += unclaimed; + } + + return ep; +} + + +/** + * page_counter_calculate_protection - check if memory consumption is in the normal range + * @root: the top ancestor of the sub-tree being checked + * @memcg: the memory cgroup to check + * @recursive_protection: Whether to use memory_recursiveprot behavior. + * + * Calculates elow/emin thresholds for given page_counter. + * + * WARNING: This function is not stateless! It can only be used as part + * of a top-down tree iteration, not for isolated queries. + */ +void page_counter_calculate_protection(struct page_counter *root, + struct page_counter *counter, + bool recursive_protection) +{ + unsigned long usage, parent_usage; + struct page_counter *parent = counter->parent; + + /* + * Effective values of the reclaim targets are ignored so they + * can be stale. Have a look at mem_cgroup_protection for more + * details. + * TODO: calculation should be more robust so that we do not need + * that special casing. + */ + if (root == counter) + return; + + usage = page_counter_read(counter); + if (!usage) + return; + + if (parent == root) { + counter->emin = READ_ONCE(counter->min); + counter->elow = READ_ONCE(counter->low); + return; + } + + parent_usage = page_counter_read(parent); + + WRITE_ONCE(counter->emin, effective_protection(usage, parent_usage, + READ_ONCE(counter->min), + READ_ONCE(parent->emin), + atomic_long_read(&parent->children_min_usage), + recursive_protection)); + + WRITE_ONCE(counter->elow, effective_protection(usage, parent_usage, + READ_ONCE(counter->low), + READ_ONCE(parent->elow), + atomic_long_read(&parent->children_low_usage), + recursive_protection)); +} From patchwork Thu Jun 27 15:47:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Maarten Lankhorst X-Patchwork-Id: 13714715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D53CC2BD09 for ; Thu, 27 Jun 2024 15:48:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D53EF6B008A; Thu, 27 Jun 2024 11:48:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D00B86B0095; Thu, 27 Jun 2024 11:48:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADE2F6B0096; Thu, 27 Jun 2024 11:48:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 84F9A6B008A for ; Thu, 27 Jun 2024 11:48:08 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2A8AA120A37 for ; Thu, 27 Jun 2024 15:48:08 +0000 (UTC) X-FDA: 82277099856.29.C05C084 Received: from mblankhorst.nl (lankhorst.se [141.105.120.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 44E7920021 for ; Thu, 27 Jun 2024 15:48:06 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=none (imf03.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719503277; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D/Zv1QVLxFoAYgPV2UV1YUyjgDl2dg3RLmKtg9uHEd4=; b=xhKYTGf9QUyu7IlYPCB1MB7GaziBi7Gvmn/YF0e9+2H2CPXrMlimxa5gzIj11WacPH3roM THoyS2de9y756nGKdRsu//At1fJNE55f/hFUXVY3dqvP082BM+XuF25hwNYMOF8OLUXICW xowTnaR5cDHa26k/wGvaR1gQubQkMWI= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=none (imf03.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719503277; a=rsa-sha256; cv=none; b=qbzY9FZCayRyYVQIsp9EIBiodA5rj8bERkhU+dO4pH6sv7pvsT9awJeM8QrUIkkhHRpf0/ LyOk62IR0Lg8zL6cSNKWCrSzBhswKfOuQnxBnCx1IwX+XrnbHdRlo59Qibe+Dz3fiS4AMt lcwEeKnYgSBepyUi3tHZl/U8QsXD8GA= From: Maarten Lankhorst To: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Jonathan Corbet , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann Cc: Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org Subject: [RFC PATCH 2/6] drm/cgroup: Add memory accounting DRM cgroup Date: Thu, 27 Jun 2024 17:47:21 +0200 Message-ID: <20240627154754.74828-3-maarten.lankhorst@linux.intel.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 44E7920021 X-Stat-Signature: 1j3mqspysw8frr75dpihexgw4rkjtubg X-Rspam-User: X-HE-Tag: 1719503286-465222 X-HE-Meta: U2FsdGVkX1+adLD6iOPml2YB1V54J/7YHXsEtDwdOhiop1E1sTsEyCotnmLWGwMskHImsjP+lAaXMAo5tEhADFXXsQo6W0SKguw43H9bPsYBW5Ifo0Vswvxaef2AmMwvtYaFZQul7SAzZDVQFMLm8g735ksNY4Y6HlryNf9x/p5Q7aq6R/tmgp9rEkGOeJQZZRepLH13gKi8hohlQs+IRFB44S0hFrZFUiXztnGvIA5X1CJrBrmSVw4ebMSiaLWc9XxY1epuXxrtABVRfEK1v5ZfQhKvURFfFQ+MdoxtDYIIfPxUHfLhr5zbQR5TXzzq9oXz7YR748pzNKAlnhdV1Du++NNJ/c6PGICIMT1KZsPwKc5xVlTCdmUOMfmcHyG6j1hG2MQBKCdkj0dGwgTdl4p+0+dUmZ8bRa3XYCyDz8NHXqC58cUCjPk6J2gd5Jw8kTy/m94RnOSKso04Wk5Udco0V/uFfSFFzCVMXa2YZeP2g+AIriXgjERjKliFiX9l6W1yaCDQg8LZCvuk3GC83skPKToMpV5DZVH+NozyikJg0M4oTrzPsbSxicC7bWURPOO8l2iI4o2/DDlWVPGavq6gvagW74rNcpo5sSsPyZ9zK7mURk29sAn0BqRvxeaNsyGuT9hALDgj7CzxnvSANVFgTHlOKEjcQh4qebns9aJdqNV/nQMgRz2mi/TodscRBkz0MyJTTaSnXSQ4BSD3mv3lt6NDGc+NCRQ1+l0Ma5eoS1vamdUxl0aM0fTCnFh4eLSRDngYjN1eg6fA/QnUpz6KdrjTUhBUGpb89Gw74IzZcSw/Yh1qwvbxnPtT436Fbkpd0FcOC5tiVVCwGbBFRYb2gQqcAYBECekecJ3Ant5t8swdPIgeUuP1tVOpFu58P9IIrgxxwkIieO+5dt+YFHtFJ/USTohTjlyx/5tFYBvLbTRLgzLLm4ER6enyccHsn73dwFi2RG2UiN7RkRc LK7SsMMP sfTRv8Noj4nZtfWbJ8jGbGs5+4H3QpHFOu63yKxQbeB7cYBw5dA7RAaSe8swVHQa+KCWHFgL8Ehy0toREPjkyl1ZIj5ZWK7Ym/BS+bKk/avWyDsfzA4TihIG7R6jrvDCAmQC8eT8WPEwYmp5Bie+tuBFXyygm2XV9vt9Q8Z/MxaWfx8ekUTOqp+b5kJZe3GJJ8zEZCQEGb+S0iPpb7/BeJL4jZB2KheIf+L/Ed2lf39dtsUmFLSeICzng8K5HTcgQoNLJPpXVLa508dSEgwS+e0P4UpG32oId+SjBy/NNylTESCo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The initial version was based roughly on the rdma and misc cgroup controllers, with a lot of the accounting code borrowed from rdma. The current version is a complete rewrite with page counter; it uses the same min/low/max semantics as the memory cgroup as a result. There's a small mismatch as TTM uses u64, and page_counter long pages. In practice it's not a problem. 32-bits systems don't really come with >=4GB cards and as long as we're consistently wrong with units, it's fine. The device page size may not be in the same units as kernel page size, and each region might also have a different page size (VRAM vs GART for example). The interface is simple: - populate drmcgroup_device->regions[..] name and size for each active region, set num_regions accordingly. - Call drm(m)cg_register_device() - Use drmcg_try_charge to check if you can allocate a chunk of memory, use drmcg_uncharge when freeing it. This may return an error code, or -EAGAIN when the cgroup limit is reached. In that case a reference to the limiting pool is returned. - The limiting cs can be used as compare function for drmcs_evict_valuable. - After having evicted enough, drop reference to limiting cs with drmcs_pool_put. This API allows you to limit device resources with cgroups. You can see the supported cards in /sys/fs/cgroup/drm.capacity You need to echo +drm to cgroup.subtree_control, and then you can partition memory. Signed-off-by: Maarten Lankhorst Co-developed-by: Friedrich Vock --- Documentation/admin-guide/cgroup-v2.rst | 51 ++ Documentation/gpu/drm-compute.rst | 54 ++ include/linux/cgroup_drm.h | 115 ++++ include/linux/cgroup_subsys.h | 4 + init/Kconfig | 7 + kernel/cgroup/Makefile | 1 + kernel/cgroup/drm.c | 813 ++++++++++++++++++++++++ 7 files changed, 1045 insertions(+) create mode 100644 Documentation/gpu/drm-compute.rst create mode 100644 include/linux/cgroup_drm.h create mode 100644 kernel/cgroup/drm.c diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 8fbb0519d556..edf63c3edb41 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2553,6 +2553,57 @@ RDMA Interface Files mlx4_0 hca_handle=1 hca_object=20 ocrdma1 hca_handle=1 hca_object=23 +DRM +---- + +The "drm" controller regulates the distribution and accounting of +DRM resources, currently only memory regions. Because each memory +region may have its own page size, which does not have to be equal +to the system page size. the units are in bytes. + +DRM Interface Files +~~~~~~~~~~~~~~~~~~~~ + + drm.max, drm.min, drm.low + A readwrite nested-keyed file that exists for all the cgroups + except root that describes current configured resource limit + for a DRM device. + + Lines are keyed by device name and are not ordered. + Each line contains space separated resource name and its configured + limit that can be distributed. + + The following nested keys are defined. + + ========== ======================================================= + region.* Maximum amount of bytes that allocatable in this region + ========== ======================================================= + + An example for xe follows:: + + 0000:03:00.0 region.vram0=1073741824 region.stolen=max + + The semantics are the same as for the memory cgroup controller, and are + calculated in the same way. + + drm.capacity + A read-only file that describes maximum region capacity. + It only exists on the root cgroup. Not all memory can be + allocated by cgroups, as the kernel reserves some for + internal use. + + An example for xe follows:: + + 0000:03:00.0 region.vram0=8514437120 region.stolen=67108864 + + drm.current + A read-only file that describes current resource usage. + It exists for all the cgroup except root. + + An example for xe follows:: + + 0000:03:00.0 region.vram0=12550144 region.stolen=8650752 + HugeTLB ------- diff --git a/Documentation/gpu/drm-compute.rst b/Documentation/gpu/drm-compute.rst new file mode 100644 index 000000000000..116270976ef7 --- /dev/null +++ b/Documentation/gpu/drm-compute.rst @@ -0,0 +1,54 @@ +================================== +Long running workloads and compute +================================== + +Long running workloads (compute) are workloads that will not complete in 10 +seconds. (The time let the user wait before he reaches for the power button). +This means that other techniques need to be used to manage those workloads, +that cannot use fences. + +Some hardware may schedule compute jobs, and have no way to pre-empt them, or +have their memory swapped out from them. Or they simply want their workload +not to be preempted or swapped out at all. + +This means that it differs from what is described in driver-api/dma-buf.rst. + +As with normal compute jobs, dma-fence may not be used at all. In this case, +not even to force preemption. The driver with is simply forced to unmap a BO +from the long compute job's address space on unbind immediately, not even +waiting for the workload to complete. Effectively this terminates the workload +when there is no hardware support to recover. + +Since this is undesirable, there need to be mitigations to prevent a workload +from being terminated. There are several possible approach, all with their +advantages and drawbacks. + +The first approach you will likely try is to pin all buffers used by compute. +This guarantees that the job will run uninterrupted, but also allows a very +denial of service attack by pinning as much memory as possible, hogging the +all GPU memory, and possibly a huge chunk of CPU memory. + +A second approach that will work slightly better on its own is adding an option +not to evict when creating a new job (any kind). If all of userspace opts in +to this flag, it would prevent cooperating userspace from forced terminating +older compute jobs to start a new one. + +If job preemption and recoverable pagefaults are not available, those are the +only approaches possible. So even with those, you want a separate way of +controlling resources. The standard kernel way of doing so is cgroups. + +This creates a third option, using cgroups to prevent eviction. Both GPU and +driver-allocated CPU memory would be accounted to the correct cgroup, and +eviction would be made cgroup aware. This allows the GPU to be partitioned +into cgroups, that will allow jobs to run next to each other without +interference. + +The interface to the cgroup would be similar to the current CPU memory +interface, with similar semantics for min/low/high/max, if eviction can +be made cgroup aware. For now only max is implemented. + +What should be noted is that each memory region (tiled memory for example) +should have its own accounting, using $card key0 = value0 key1 = value1. + +The key is set to the regionid set by the driver, for example "tile0". +For the value of $card, we use drmGetUnique(). diff --git a/include/linux/cgroup_drm.h b/include/linux/cgroup_drm.h new file mode 100644 index 000000000000..bbc466f9f368 --- /dev/null +++ b/include/linux/cgroup_drm.h @@ -0,0 +1,115 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright © 2023 Intel Corporation + */ + +#ifndef _CGROUP_DRM_H +#define _CGROUP_DRM_H + +#include +#include + +#include + +struct drm_device; +struct drm_file; + +struct drmcgroup_pool_state; + +/* + * Use 8 as max, because of N^2 lookup when setting things, can be bumped if needed + * Identical to TTM_NUM_MEM_TYPES to allow simplifying that code. + */ +#define DRMCG_MAX_REGIONS 8 + +/* Public definition of cgroup device, should not be modified after _register() */ +struct drmcgroup_device { + struct { + u64 size; + const char *name; + } regions[DRMCG_MAX_REGIONS]; + + int num_regions; + + /* used by cgroups, do not use */ + void *priv; +}; + +#if IS_ENABLED(CONFIG_CGROUP_DRM) +int drmcg_register_device(struct drm_device *dev, + struct drmcgroup_device *drm_cg); +void drmcg_unregister_device(struct drmcgroup_device *cgdev); +int drmcg_try_charge(struct drmcgroup_pool_state **drmcs, + struct drmcgroup_pool_state **limitcs, + struct drmcgroup_device *cgdev, + u32 index, u64 size); +void drmcg_uncharge(struct drmcgroup_pool_state *drmcs, + struct drmcgroup_device *cgdev, + u32 index, u64 size); + +bool drmcs_evict_valuable(struct drmcgroup_pool_state *limitcs, + struct drmcgroup_device *dev, + int index, + struct drmcgroup_pool_state *testcs, + bool ignore_low, + bool *hit_low); + +void drmcs_pool_put(struct drmcgroup_pool_state *drmcs); +#else +static inline int +drmcg_register_device(struct drm_device *dev, + struct drmcgroup_device *drm_cg) +{ + return 0; +} + +static inline void drmcg_unregister_device(struct drmcgroup_device *cgdev) +{ +} + +static inline int drmcg_try_charge(struct drmcgroup_pool_state **drmcs, + struct drmcgroup_pool_state **limitcs, + struct drmcgroup_device *cgdev, + u32 index, u64 size) +{ + *drmcs = NULL; + if (limitcs) + *limitcs = NULL; + return 0; +} + +static inline void drmcg_uncharge(struct drmcgroup_pool_state *drmcs, + struct drmcgroup_device *cgdev, + u32 index, u64 size) +{ } + +static inline bool drmcs_evict_valuable(struct drmcgroup_pool_state *limitcs, + struct drmcgroup_device *dev, int index, + struct drmcgroup_pool_state *testcs, + bool ignore_low, bool *hit_low) +{ + return true; +} + +static inline void drmcs_pool_put(struct drmcgroup_pool_state *drmcs) +{ } + +#endif + +static inline void drmmcg_unregister_device(struct drm_device *dev, void *arg) +{ + drmcg_unregister_device(arg); +} + +/* + * This needs to be done as inline, because cgroup lives in the core + * kernel and it cannot call drm calls directly + */ +static inline int drmmcg_register_device(struct drm_device *dev, + struct drmcgroup_device *cgdev) +{ + return drmcg_register_device(dev, cgdev) ?: + drmm_add_action_or_reset(dev, drmmcg_unregister_device, cgdev); +} + +#endif /* _CGROUP_DRM_H */ diff --git a/include/linux/cgroup_subsys.h b/include/linux/cgroup_subsys.h index 445235487230..49460494a010 100644 --- a/include/linux/cgroup_subsys.h +++ b/include/linux/cgroup_subsys.h @@ -65,6 +65,10 @@ SUBSYS(rdma) SUBSYS(misc) #endif +#if IS_ENABLED(CONFIG_CGROUP_DRM) +SUBSYS(drm) +#endif + /* * The following subsystems are not supported on the default hierarchy. */ diff --git a/init/Kconfig b/init/Kconfig index febdea2afc3b..0acb42c96579 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1091,6 +1091,13 @@ config CGROUP_RDMA Attaching processes with active RDMA resources to the cgroup hierarchy is allowed even if can cross the hierarchy's limit. +config CGROUP_DRM + bool "DRM controller" + help + Provides the DRM subsystem controller. + + ... + config CGROUP_FREEZER bool "Freezer controller" help diff --git a/kernel/cgroup/Makefile b/kernel/cgroup/Makefile index 12f8457ad1f9..849bd2917477 100644 --- a/kernel/cgroup/Makefile +++ b/kernel/cgroup/Makefile @@ -6,4 +6,5 @@ obj-$(CONFIG_CGROUP_PIDS) += pids.o obj-$(CONFIG_CGROUP_RDMA) += rdma.o obj-$(CONFIG_CPUSETS) += cpuset.o obj-$(CONFIG_CGROUP_MISC) += misc.o +obj-$(CONFIG_CGROUP_DRM) += drm.o obj-$(CONFIG_CGROUP_DEBUG) += debug.o diff --git a/kernel/cgroup/drm.c b/kernel/cgroup/drm.c new file mode 100644 index 000000000000..26d64070e74f --- /dev/null +++ b/kernel/cgroup/drm.c @@ -0,0 +1,813 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2023-2024 Intel + * Partially based on the rdma and misc controllers, which bear the following copyrights: + * + * Copyright 2020 Google LLC + * Copyright (C) 2016 Parav Pandit + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +struct drmcg_device { + spinlock_t lock; + struct kref ref; + struct rcu_head rcu; + + /* Protected by RCU and global spinlock */ + struct list_head dev_node; + + /* Protected by global spinlock only */ + struct list_head pools; + + /* Copy of the struct passed by device, to prevent lifetime issues */ + struct drmcgroup_device base; + + /* Name describing the card, set by drmcg_register_device */ + const char *name; + + /* Whether the device is unregistered by its caller. + * No new pools should be added to the device afterwards. + */ + bool unregistered; +}; + +struct drmcgroup_state { + struct cgroup_subsys_state css; + + struct list_head pools; +}; + +struct drmcgroup_pool_state { + struct drmcg_device *device; + struct drmcgroup_state *cs; + + /* css node, RCU protected against device teardown */ + struct list_head css_node; + + /* dev node, no RCU protection required */ + struct list_head dev_node; + + int num_res, inited; + struct rcu_head rcu; + + struct drmcgroup_pool_res { + struct page_counter cnt; + } resources[]; +}; + +/* + * 3 operations require locking protection: + * - Registering and unregistering device to/from list, requires global lock. + * - Adding a drmcgroup_pool_state to a CSS, removing when CSS is freed. + * - Adding a drmcgroup_pool_state to a device list. + * + * Since for the most common operations RCU provides enough protection, I + * do not think more granular locking makes sense. Most protection is offered + * by RCU and the lockless operating page_counter. + */ +static DEFINE_SPINLOCK(drmcg_lock); +static LIST_HEAD(drmcg_devices); + +static inline struct drmcgroup_state * +css_to_drmcs(struct cgroup_subsys_state *css) +{ + return container_of(css, struct drmcgroup_state, css); +} + +static inline struct drmcgroup_state *get_current_drmcg(void) +{ + return css_to_drmcs(task_get_css(current, drm_cgrp_id)); +} + +static struct drmcgroup_state *parent_drmcg(struct drmcgroup_state *cg) +{ + return cg->css.parent ? css_to_drmcs(cg->css.parent) : NULL; +} + +static void free_cg_pool(struct drmcgroup_pool_state *pool) +{ + list_del(&pool->dev_node); + kfree(pool); +} + +static void +set_resource_min(struct drmcgroup_pool_state *pool, int i, u64 val) +{ + page_counter_set_min(&pool->resources[i].cnt, val); +} + +static void +set_resource_low(struct drmcgroup_pool_state *pool, int i, u64 val) +{ + page_counter_set_low(&pool->resources[i].cnt, val); +} + +static void +set_resource_max(struct drmcgroup_pool_state *pool, int i, u64 val) +{ + page_counter_set_max(&pool->resources[i].cnt, val); +} + +static u64 get_resource_low(struct drmcgroup_pool_state *pool, int idx) +{ + return pool ? READ_ONCE(pool->resources[idx].cnt.low) : 0; +} + +static u64 get_resource_min(struct drmcgroup_pool_state *pool, int idx) +{ + return pool ? READ_ONCE(pool->resources[idx].cnt.min) : 0; +} + +static u64 get_resource_max(struct drmcgroup_pool_state *pool, int idx) +{ + return pool ? READ_ONCE(pool->resources[idx].cnt.max) : PAGE_COUNTER_MAX; +} + +static u64 get_resource_current(struct drmcgroup_pool_state *pool, int idx) +{ + return pool ? page_counter_read(&pool->resources[idx].cnt) : 0; +} + +static void reset_all_resource_limits(struct drmcgroup_pool_state *rpool) +{ + int i; + + for (i = 0; i < rpool->num_res; i++) { + set_resource_min(rpool, i, 0); + set_resource_low(rpool, i, 0); + set_resource_max(rpool, i, PAGE_COUNTER_MAX); + } +} + +static void drmcs_offline(struct cgroup_subsys_state *css) +{ + struct drmcgroup_state *drmcs = css_to_drmcs(css); + struct drmcgroup_pool_state *pool; + + rcu_read_lock(); + list_for_each_entry_rcu(pool, &drmcs->pools, css_node) + reset_all_resource_limits(pool); + rcu_read_unlock(); +} + +static void drmcs_free(struct cgroup_subsys_state *css) +{ + struct drmcgroup_state *drmcs = css_to_drmcs(css); + struct drmcgroup_pool_state *pool, *next; + + spin_lock(&drmcg_lock); + list_for_each_entry_safe(pool, next, &drmcs->pools, css_node) { + /* + *The pool is dead and all references are 0, + * no need for RCU protection with list_del_rcu or freeing. + */ + list_del(&pool->css_node); + free_cg_pool(pool); + } + spin_unlock(&drmcg_lock); + + kfree(drmcs); +} + +static struct cgroup_subsys_state * +drmcs_alloc(struct cgroup_subsys_state *parent_css) +{ + struct drmcgroup_state *drmcs = kzalloc(sizeof(*drmcs), GFP_KERNEL); + if (!drmcs) + return ERR_PTR(-ENOMEM); + + INIT_LIST_HEAD(&drmcs->pools); + return &drmcs->css; +} + +static struct drmcgroup_pool_state * +find_cg_pool_locked(struct drmcgroup_state *drmcs, struct drmcg_device *dev) +{ + struct drmcgroup_pool_state *pool; + + list_for_each_entry_rcu(pool, &drmcs->pools, css_node, spin_locked(&drmcg_lock)) + if (pool->device == dev) + return pool; + + return NULL; +} + +static struct drmcgroup_pool_state *pool_parent(struct drmcgroup_pool_state *pool) +{ + if (!pool->resources[0].cnt.parent) + return NULL; + + return container_of(pool->resources[0].cnt.parent, typeof(*pool), resources[0].cnt); +} + +bool drmcs_evict_valuable(struct drmcgroup_pool_state *limit, + struct drmcgroup_device *dev, + int index, + struct drmcgroup_pool_state *test, + bool ignore_low, + bool *hit_low) +{ + struct drmcgroup_pool_state *pool = test; + struct page_counter *climit, *ctest; + u64 used, min, low; + + /* Can always evict from current pool, despite limits */ + if (limit == test) + return true; + + if (limit) { + if (!parent_drmcg(limit->cs)) + return true; + + for (pool = test; pool && limit != pool; pool = pool_parent(pool)) + {} + + if (!pool) + return false; + } else { + /* + * If there is no cgroup limiting memory usage, use the root + * cgroup instead for limit calculations. + */ + for (limit = test; pool_parent(limit); limit = pool_parent(limit)) + {} + } + + climit = &limit->resources[index].cnt; + ctest = &test->resources[index].cnt; + + page_counter_calculate_protection(climit, ctest, true); + + used = page_counter_read(ctest); + min = READ_ONCE(ctest->emin); + + if (used <= min) + return false; + + if (!ignore_low) { + low = READ_ONCE(ctest->elow); + if (used > low) + return true; + + *hit_low = true; + return false; + } + return true; +} +EXPORT_SYMBOL_GPL(drmcs_evict_valuable); + +static struct drmcgroup_pool_state * +alloc_pool_single(struct drmcgroup_state *drmcs, struct drmcg_device *dev, + struct drmcgroup_pool_state **allocpool) +{ + struct drmcgroup_state *parent = parent_drmcg(drmcs); + struct drmcgroup_pool_state *pool, *ppool = NULL; + int i; + + if (!*allocpool) { + pool = kzalloc(offsetof(struct drmcgroup_pool_state, resources[dev->base.num_regions]), GFP_NOWAIT); + if (!pool) + return ERR_PTR(-ENOMEM); + } else { + pool = *allocpool; + *allocpool = NULL; + } + + pool->device = dev; + pool->num_res = dev->base.num_regions; + pool->cs = drmcs; + + if (parent) + ppool = find_cg_pool_locked(parent, dev); + + for (i = 0; i < pool->num_res; i++) + page_counter_init(&pool->resources[i].cnt, ppool ? &ppool->resources[i].cnt : NULL); + reset_all_resource_limits(pool); + + list_add_tail_rcu(&pool->css_node, &drmcs->pools); + list_add_tail(&pool->dev_node, &dev->pools); + + if (!parent) + pool->inited = true; + else + pool->inited = ppool ? ppool->inited : false; + return pool; +} + +static struct drmcgroup_pool_state * +get_cg_pool_locked(struct drmcgroup_state *drmcs, struct drmcg_device *dev, + struct drmcgroup_pool_state **allocpool) +{ + struct drmcgroup_pool_state *pool, *ppool, *retpool; + struct drmcgroup_state *p, *pp; + int i; + + /* + * Recursively create pool, we may not initialize yet on + * recursion, this is done as a separate step. + */ + for (p = drmcs; p; p = parent_drmcg(p)) { + pool = find_cg_pool_locked(p, dev); + if (!pool) + pool = alloc_pool_single(p, dev, allocpool); + + if (IS_ERR(pool)) + return pool; + + if (p == drmcs && pool->inited) + return pool; + + if (pool->inited) + break; + } + + retpool = pool = find_cg_pool_locked(drmcs, dev); + for (p = drmcs, pp = parent_drmcg(drmcs); pp; p = pp, pp = parent_drmcg(p)) { + if (pool->inited) + break; + + /* ppool was created if it didn't exist by above loop. */ + ppool = find_cg_pool_locked(pp, dev); + + /* Fix up parent links, mark as inited. */ + for (i = 0; i < pool->num_res; i++) + pool->resources[i].cnt.parent = &ppool->resources[i].cnt; + pool->inited = true; + + pool = ppool; + } + + return retpool; +} + +static void drmcg_free_rcu(struct rcu_head *rcu) +{ + struct drmcg_device *dev = container_of(rcu, typeof(*dev), rcu); + struct drmcgroup_pool_state *pool, *next; + + list_for_each_entry_safe(pool, next, &dev->pools, dev_node) + free_cg_pool(pool); + kfree(dev->name); + kfree(dev); +} + +static void drmcg_free_device(struct kref *ref) +{ + struct drmcg_device *cgdev = container_of(ref, typeof(*cgdev), ref); + + call_rcu(&cgdev->rcu, drmcg_free_rcu); +} + +void drmcg_unregister_device(struct drmcgroup_device *cgdev) +{ + struct drmcg_device *dev; + struct list_head *entry; + + if (!cgdev || !cgdev->priv) + return; + + dev = cgdev->priv; + cgdev->priv = NULL; + + spin_lock(&drmcg_lock); + + /* Remove from global device list */ + list_del_rcu(&dev->dev_node); + + list_for_each_rcu(entry, &dev->pools) { + struct drmcgroup_pool_state *pool = + container_of(entry, typeof(*pool), dev_node); + + list_del_rcu(&pool->css_node); + } + + /* + * Ensure any RCU based lookups fail. Additionally, + * no new pools should be added to the dead device + * by get_cg_pool_unlocked. + */ + dev->unregistered = true; + spin_unlock(&drmcg_lock); + + kref_put(&dev->ref, drmcg_free_device); +} + +EXPORT_SYMBOL_GPL(drmcg_unregister_device); + +int drmcg_register_device(struct drm_device *drm_dev, + struct drmcgroup_device *cgdev) +{ + struct drmcg_device *dev; + char *name; + + cgdev->priv = NULL; + if (!cgdev->num_regions) + return 0; + + cgdev->priv = dev = kzalloc(sizeof (*dev), GFP_KERNEL); + if (!dev) + return -ENOMEM; + name = kstrdup(drm_dev->unique, GFP_KERNEL); + if (!name) { + kfree(dev); + cgdev->priv = NULL; + return -ENOMEM; + } + + INIT_LIST_HEAD(&dev->pools); + dev->name = name; + dev->base = *cgdev; + kref_init(&dev->ref); + + spin_lock(&drmcg_lock); + list_add_tail_rcu(&dev->dev_node, &drmcg_devices); + spin_unlock(&drmcg_lock); + + return 0; +} +EXPORT_SYMBOL_GPL(drmcg_register_device); + +static struct drmcg_device *drmcg_get_device(const char *name) +{ + struct drmcg_device *dev; + + list_for_each_entry_rcu(dev, &drmcg_devices, dev_node, spin_locked(&drmcg_lock)) + if (!strcmp(name, dev->name) && + kref_get_unless_zero(&dev->ref)) + return dev; + + return NULL; +} + +void drmcs_pool_put(struct drmcgroup_pool_state *pool) +{ + if (pool) + css_put(&pool->cs->css); +} +EXPORT_SYMBOL_GPL(drmcs_pool_put); + +static struct drmcgroup_pool_state * +get_cg_pool_unlocked(struct drmcgroup_state *cg, struct drmcg_device *dev) +{ + struct drmcgroup_pool_state *pool, *allocpool = NULL; + + /* fastpath lookup? */ + rcu_read_lock(); + pool = find_cg_pool_locked(cg, dev); + if (pool && !READ_ONCE(pool->inited)) + pool = NULL; + rcu_read_unlock(); + + while (!pool) { + spin_lock(&drmcg_lock); + if (!dev->unregistered) + pool = get_cg_pool_locked(cg, dev, &allocpool); + else + pool = ERR_PTR(-ENODEV); + spin_unlock(&drmcg_lock); + + if (pool == ERR_PTR(-ENOMEM)) { + pool = NULL; + if (WARN_ON(allocpool)) + continue; + + allocpool = kzalloc(offsetof(struct drmcgroup_pool_state, resources[dev->base.num_regions]), GFP_KERNEL); + if (allocpool) { + pool = NULL; + continue; + } + } + } + + kfree(allocpool); + return pool; +} + +void drmcg_uncharge(struct drmcgroup_pool_state *pool, + struct drmcgroup_device *cgdev, + u32 index, u64 size) +{ + if (index >= cgdev->num_regions || !pool) + return; + + page_counter_uncharge(&pool->resources[index].cnt, size); + css_put(&pool->cs->css); +} +EXPORT_SYMBOL_GPL(drmcg_uncharge); + +int drmcg_try_charge(struct drmcgroup_pool_state **drmcs, + struct drmcgroup_pool_state **limitcs, + struct drmcgroup_device *dev, + u32 index, u64 size) +{ + struct drmcg_device *cgdev = dev->priv; + struct drmcgroup_state *cg; + struct drmcgroup_pool_state *pool; + struct page_counter *fail; + int ret; + + *drmcs = NULL; + if (limitcs) + *limitcs = NULL; + + /* Early init or device unregistered */ + if (!cgdev) + return 0; + + if (index >= cgdev->base.num_regions) + return -EINVAL; + + /* + * hold on to css, as cgroup can be removed but resource + * accounting happens on css. + */ + cg = get_current_drmcg(); + + pool = get_cg_pool_unlocked(cg, cgdev); + if (IS_ERR(pool)) { + ret = PTR_ERR(pool); + goto err; + } + + if (!page_counter_try_charge(&pool->resources[index].cnt, size, &fail)) { + if (limitcs) { + *limitcs = container_of(fail, struct drmcgroup_pool_state, resources[index].cnt); + css_get(&(*limitcs)->cs->css); + } + ret = -EAGAIN; + goto err; + } + + /* On success, reference is transferred to *drmcs */ + *drmcs = pool; + return 0; + +err: + css_put(&cg->css); + return ret; +} +EXPORT_SYMBOL_GPL(drmcg_try_charge); + +static int drmcg_capacity_show(struct seq_file *sf, void *v) +{ + struct drmcg_device *dev; + int i; + + rcu_read_lock(); + list_for_each_entry_rcu(dev, &drmcg_devices, dev_node) { + seq_puts(sf, dev->name); + for (i = 0; i < dev->base.num_regions; i++) + seq_printf(sf, " region.%s=%lld", + dev->base.regions[i].name, + dev->base.regions[i].size); + seq_putc(sf, '\n'); + } + rcu_read_unlock(); + return 0; +} + +static s64 parse_resource(char *c, char **retname) +{ + substring_t argstr; + char *name, *value = c; + size_t len; + int ret; + u64 retval; + + name = strsep(&value, "="); + if (!name || !value) + return -EINVAL; + + /* Only support region setting for now */ + if (strncmp(name, "region.", 7)) + return -EINVAL; + else + name += 7; + + *retname = name; + len = strlen(value); + + argstr.from = value; + argstr.to = value + len; + + ret = match_u64(&argstr, &retval); + if (ret >= 0) { + if (retval > S64_MAX) + return -EINVAL; + if (retval > PAGE_COUNTER_MAX) + return PAGE_COUNTER_MAX; + return retval; + } + if (!strncmp(value, "max", len)) + return PAGE_COUNTER_MAX; + + /* Not u64 or max, error */ + return -EINVAL; +} + +static int drmcg_parse_limits(char *options, struct drmcg_device *dev, + u64 *new_limits, unsigned long *set_mask) +{ + char *c, *region; + + /* parse resource options */ + while ((c = strsep(&options, " \t")) != NULL) { + s64 limit; + int i; + + limit = parse_resource(c, ®ion); + if (limit < 0) + return limit; + + + for (i = 0; i < dev->base.num_regions; i++) + if (!strcmp(dev->base.regions[i].name, region)) + break; + + if (i == dev->base.num_regions) + return -EINVAL; + + new_limits[i] = limit; + *set_mask |= BIT(i); + } + return 0; +} + +static ssize_t drmcg_limit_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off, + void (*apply)(struct drmcgroup_pool_state *, int, u64)) +{ + struct drmcgroup_state *drmcs = css_to_drmcs(of_css(of)); + int err = 0; + + while (buf && !err) { + struct drmcgroup_pool_state *pool = NULL; + char *options, *dev_name; + unsigned long set_mask = 0; + struct drmcg_device *dev; + u64 new_limits[DRMCG_MAX_REGIONS]; + int i; + + options = buf; + buf = strchr(buf, '\n'); + if (buf) + *buf++ = '\0'; + + options = strstrip(options); + + /* eat empty lines */ + if (!options[0]) + continue; + + dev_name = strsep(&options, " \t"); + if (!dev_name[0]) + continue; + + rcu_read_lock(); + dev = drmcg_get_device(dev_name); + rcu_read_unlock(); + + if (!dev) + return -EINVAL; + + err = drmcg_parse_limits(options, dev, new_limits, &set_mask); + if (err < 0) + goto out_put; + + pool = get_cg_pool_unlocked(drmcs, dev); + if (IS_ERR(pool)) { + err = PTR_ERR(pool); + goto out_put; + } + + /* And commit */ + for_each_set_bit(i, &set_mask, DRMCG_MAX_REGIONS) + apply(pool, i, new_limits[i]); + +out_put: + kref_put(&dev->ref, drmcg_free_device); + } + + + return err ?: nbytes; +} + +static int drmcg_limit_show(struct seq_file *sf, void *v, + u64 (*fn)(struct drmcgroup_pool_state *, int)) +{ + struct drmcgroup_state *drmcs = css_to_drmcs(seq_css(sf)); + struct drmcg_device *dev; + + rcu_read_lock(); + list_for_each_entry_rcu(dev, &drmcg_devices, dev_node) { + struct drmcgroup_pool_state *pool = find_cg_pool_locked(drmcs, dev); + + seq_puts(sf, dev->name); + + for (int i = 0; i < dev->base.num_regions; i++) { + u64 val = fn(pool, i); + + if (val < PAGE_COUNTER_MAX) + seq_printf(sf, " region.%s=%lld", + dev->base.regions[i].name, val); + else + seq_printf(sf, " region.%s=max", dev->base.regions[i].name); + } + + seq_putc(sf, '\n'); + } + rcu_read_unlock(); + + css_put(&drmcs->css); + + return 0; +} + +static int drmcg_current_show(struct seq_file *sf, void *v) +{ + return drmcg_limit_show(sf, v, get_resource_current); +} + +static int drmcg_min_show(struct seq_file *sf, void *v) +{ + return drmcg_limit_show(sf, v, get_resource_min); +} + +static ssize_t drmcg_min_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + return drmcg_limit_write(of, buf, nbytes, off, set_resource_min); +} + +static int drmcg_low_show(struct seq_file *sf, void *v) +{ + return drmcg_limit_show(sf, v, get_resource_low); +} + +static ssize_t drmcg_low_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + return drmcg_limit_write(of, buf, nbytes, off, set_resource_low); +} + +static int drmcg_max_show(struct seq_file *sf, void *v) +{ + return drmcg_limit_show(sf, v, get_resource_max); +} + +static ssize_t drmcg_max_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + return drmcg_limit_write(of, buf, nbytes, off, set_resource_max); +} + +static struct cftype files[] = { + { + .name = "capacity", + .seq_show = drmcg_capacity_show, + .flags = CFTYPE_ONLY_ON_ROOT, + }, + { + .name = "current", + .seq_show = drmcg_current_show, + }, + { + .name = "min", + .write = drmcg_min_write, + .seq_show = drmcg_min_show, + .flags = CFTYPE_NOT_ON_ROOT, + }, + { + .name = "low", + .write = drmcg_low_write, + .seq_show = drmcg_low_show, + .flags = CFTYPE_NOT_ON_ROOT, + }, + { + .name = "max", + .write = drmcg_max_write, + .seq_show = drmcg_max_show, + .flags = CFTYPE_NOT_ON_ROOT, + }, + { } /* Zero entry terminates. */ +}; + +struct cgroup_subsys drm_cgrp_subsys = { + .css_alloc = drmcs_alloc, + .css_free = drmcs_free, + .css_offline = drmcs_offline, + .legacy_cftypes = files, + .dfl_cftypes = files, +}; From patchwork Thu Jun 27 15:47:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maarten Lankhorst X-Patchwork-Id: 13714716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41E09C30653 for ; Thu, 27 Jun 2024 15:48:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 53D406B0095; Thu, 27 Jun 2024 11:48:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C5CA6B0096; Thu, 27 Jun 2024 11:48:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C9C56B0099; Thu, 27 Jun 2024 11:48:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 011F06B0095 for ; Thu, 27 Jun 2024 11:48:09 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 839FC1A0AE6 for ; Thu, 27 Jun 2024 15:48:09 +0000 (UTC) X-FDA: 82277099898.18.0FA6609 Received: from mblankhorst.nl (lankhorst.se [141.105.120.124]) by imf16.hostedemail.com (Postfix) with ESMTP id CBCC7180021 for ; Thu, 27 Jun 2024 15:48:07 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; spf=none (imf16.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719503270; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MYCeKgjWukJyBeJ8r5qHndxiIkxkeOpXrRB+1H8rRJw=; b=c1knIy9wvzlYxZKxRCW7PkfoMxFHP1N25bMaWTEZ3tD2r0FXkF+256D9AdaH++29O4C8u1 XU1JCOom3cLLdgpL9dqV/+yjwP8MHYpkGHguaOKmnsmrv+naA1LUJXsRPJ4K8g5UoLtBcI 5Aqy+Q8kOCHNnhXKNgm+U31e80buVHM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719503270; a=rsa-sha256; cv=none; b=TNtUH2r1tEZEH1FFv3wKmGqVHnhsneINV1NV7yHJ4e0mc0B2UMOC5hqyJcHGwUNOlE4Whw Lca9/ScnOvFUS/THDAYHOLID5WIjz00WUwgqlS1SrdelqgQ2zJo5l26WKG/VDaxFb6iNRJ XH3ZjGvdt7qcgNaSBiNKPtBEbLgG638= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; spf=none (imf16.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none) From: Maarten Lankhorst To: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Christian Koenig , Huang Rui , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter Cc: Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 3/6] drm/ttm: Handle cgroup based eviction in TTM Date: Thu, 27 Jun 2024 17:47:22 +0200 Message-ID: <20240627154754.74828-4-maarten.lankhorst@linux.intel.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> MIME-Version: 1.0 X-Stat-Signature: bkbjzxbcwhc1nq67py8nao9hp7ueotdr X-Rspamd-Queue-Id: CBCC7180021 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719503287-464576 X-HE-Meta: U2FsdGVkX19bll0jQfSpUcbL0sfd6V+rV9qOo/5TxQLUUdMRKnD9BSFdLwTZEV5UkJTvAPNoWcodcFeLKPuixEA1qD3hakw+4U5LXNTJyzSOAzdR8S4nfmxecO+/Mny357ZlIGrJUAhuOrzU/VdGcDMY/6fhks1hra8aGf4Im1qg93tSzIl9SmC5CZxB84Xic9WEuepK3MaH4KzHtKon5lrxE5Aoh2L+rIOMnxZNHtVxgqdkRQXdI9CnmZv2nt0vMZQn0amQ3BJIPnzuSCKSMjXLsytb+biqISkiZhwtzOQZnAvPFg7Tlbko0/8gbgspOrB18yWzxe028MyciGcwwC90/dH2X6a3s0NBkGTPwii6NUbULaKkKzW4Y54X2W0fG4bS7bhUZB1eYknV+wwQe3WUFiuQ0EcT2YDathXbcPY+Lih8hItHnOU7OpeFoLR/0nSRMd1VbW/emfQjYN07JY4XrAGNmsCuYNXZzPfCcT8AAy9YQIbZ/jiY02BQ7VLJx4qgrlqH7wAcqrQkvDs/jQD190kMSudGfF/4btwKzxAXwILffy0vyaLKo3/OFdIhSOIfxwjckFhVQEY6+nnby4+xIEE6AcPaOTwVe4dfQc19WNyijY9AgpZWxWK7hlvR5T2ELKgKwhDv33b91+9G52zOl5kTlo2fnq9VDac9YFGdjPRaTYB0nVfisuDYCmPZKypZ0ruoPoTj01dqVa3e18QCnNCdaTDvZHRztRwvsaqhh8lGk7DiJFfeZnOwxnzeilVY2MuodgMtUrGxAgVjfuZ7CsxQiKqzn+hEBcyFX5kbZ23P6GXBEak/OX+lwf+Ja9OC1GOcA8WYrTO7W+i6mSlh1tTFX8+L+AX/VKAtH5HJnHTSQ75N9kiECRpchW9O/4W4ZAmU89w+ocT5Qnf5lYAD/82qDcmbylGN2zyR5hd57snT+MkOYstrMbaIRsrU52HRAz3HHymu4vjVfIk tZfYO2Kk KO0pF+aCcj9Gk+flLj4VEO/qoXDLc8DZOviztl9fRxNIvQTBRDeEL64yevEcI4iknkrKK5mrlQ2q+FW3Ezom8XdZjg2e0Z/HWZ19PCqDHB0QB71N63gFOFWY+kmYsMv7UJNNX2RCTxcVXyOrG3OUg2BJk5TZQDfXLTPLqavaWEka6Pvb1ss1TODx294gReKlI5G6XseIpB7vDWEu4oZf8ioEzUJC2rl7gxPicKt1oc6u+ytk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: cgroup resource allocation has to be handled in TTM, so -EAGAIN from cgroups can be converted into -ENOSPC, and the limitcg can be properly evicted in ttm code. When hitting a resource limit through -EAGAIN, the cgroup for which the limit is hit is also returned. This allows eviction to delete only from cgroups which are a subgroup of the current cgroup. The returned CSS is used to determine if eviction is valuable for a given resource, and allows TTM to only target specific resources to lower memory usage. Signed-off-by: Maarten Lankhorst Co-developed-by: Friedrich Vock --- drivers/gpu/drm/ttm/tests/ttm_bo_test.c | 18 ++++----- drivers/gpu/drm/ttm/tests/ttm_resource_test.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 38 ++++++++++++++++--- drivers/gpu/drm/ttm/ttm_resource.c | 28 ++++++++++++-- include/drm/ttm/ttm_bo.h | 3 +- include/drm/ttm/ttm_resource.h | 16 +++++++- 6 files changed, 84 insertions(+), 21 deletions(-) diff --git a/drivers/gpu/drm/ttm/tests/ttm_bo_test.c b/drivers/gpu/drm/ttm/tests/ttm_bo_test.c index 1f8a4f8adc92..e2adc336dda8 100644 --- a/drivers/gpu/drm/ttm/tests/ttm_bo_test.c +++ b/drivers/gpu/drm/ttm/tests/ttm_bo_test.c @@ -252,13 +252,13 @@ static void ttm_bo_unreserve_basic(struct kunit *test) bo = ttm_bo_kunit_init(test, test->priv, BO_SIZE); bo->priority = bo_prio; - err = ttm_resource_alloc(bo, place, &res1); + err = ttm_resource_alloc(bo, place, &res1, NULL); KUNIT_ASSERT_EQ(test, err, 0); bo->resource = res1; /* Add a dummy resource to populate LRU */ - ttm_resource_alloc(bo, place, &res2); + ttm_resource_alloc(bo, place, &res2, NULL); dma_resv_lock(bo->base.resv, NULL); ttm_bo_unreserve(bo); @@ -294,12 +294,12 @@ static void ttm_bo_unreserve_pinned(struct kunit *test) dma_resv_lock(bo->base.resv, NULL); ttm_bo_pin(bo); - err = ttm_resource_alloc(bo, place, &res1); + err = ttm_resource_alloc(bo, place, &res1, NULL); KUNIT_ASSERT_EQ(test, err, 0); bo->resource = res1; /* Add a dummy resource to the pinned list */ - err = ttm_resource_alloc(bo, place, &res2); + err = ttm_resource_alloc(bo, place, &res2, NULL); KUNIT_ASSERT_EQ(test, err, 0); KUNIT_ASSERT_EQ(test, list_is_last(&res2->lru, &priv->ttm_dev->pinned), 1); @@ -343,7 +343,7 @@ static void ttm_bo_unreserve_bulk(struct kunit *test) ttm_bo_set_bulk_move(bo1, &lru_bulk_move); dma_resv_unlock(bo1->base.resv); - err = ttm_resource_alloc(bo1, place, &res1); + err = ttm_resource_alloc(bo1, place, &res1, NULL); KUNIT_ASSERT_EQ(test, err, 0); bo1->resource = res1; @@ -351,7 +351,7 @@ static void ttm_bo_unreserve_bulk(struct kunit *test) ttm_bo_set_bulk_move(bo2, &lru_bulk_move); dma_resv_unlock(bo2->base.resv); - err = ttm_resource_alloc(bo2, place, &res2); + err = ttm_resource_alloc(bo2, place, &res2, NULL); KUNIT_ASSERT_EQ(test, err, 0); bo2->resource = res2; @@ -387,7 +387,7 @@ static void ttm_bo_put_basic(struct kunit *test) bo = ttm_bo_kunit_init(test, test->priv, BO_SIZE); bo->type = ttm_bo_type_device; - err = ttm_resource_alloc(bo, place, &res); + err = ttm_resource_alloc(bo, place, &res, NULL); KUNIT_ASSERT_EQ(test, err, 0); bo->resource = res; @@ -504,7 +504,7 @@ static void ttm_bo_pin_unpin_resource(struct kunit *test) bo = ttm_bo_kunit_init(test, test->priv, BO_SIZE); - err = ttm_resource_alloc(bo, place, &res); + err = ttm_resource_alloc(bo, place, &res, NULL); KUNIT_ASSERT_EQ(test, err, 0); bo->resource = res; @@ -555,7 +555,7 @@ static void ttm_bo_multiple_pin_one_unpin(struct kunit *test) bo = ttm_bo_kunit_init(test, test->priv, BO_SIZE); - err = ttm_resource_alloc(bo, place, &res); + err = ttm_resource_alloc(bo, place, &res, NULL); KUNIT_ASSERT_EQ(test, err, 0); bo->resource = res; diff --git a/drivers/gpu/drm/ttm/tests/ttm_resource_test.c b/drivers/gpu/drm/ttm/tests/ttm_resource_test.c index 029e1f094bb0..c7d3d86ff98b 100644 --- a/drivers/gpu/drm/ttm/tests/ttm_resource_test.c +++ b/drivers/gpu/drm/ttm/tests/ttm_resource_test.c @@ -302,7 +302,7 @@ static void ttm_sys_man_free_basic(struct kunit *test) res = kunit_kzalloc(test, sizeof(*res), GFP_KERNEL); KUNIT_ASSERT_NOT_NULL(test, res); - ttm_resource_alloc(bo, place, &res); + ttm_resource_alloc(bo, place, &res, NULL, NULL); man = ttm_manager_type(priv->devs->ttm_dev, mem_type); man->func->free(man, res); diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 6396dece0db1..6ca92b64f2fe 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include "ttm_module.h" @@ -594,18 +595,24 @@ int ttm_mem_evict_first(struct ttm_device *bdev, struct ttm_resource_manager *man, const struct ttm_place *place, struct ttm_operation_ctx *ctx, - struct ww_acquire_ctx *ticket) + struct ww_acquire_ctx *ticket, + struct drmcgroup_pool_state *limitcss) { struct ttm_buffer_object *bo = NULL, *busy_bo = NULL; struct ttm_resource_cursor cursor; struct ttm_resource *res; bool locked = false; int ret; + bool try_low = false, hit_low = false; spin_lock(&bdev->lru_lock); +retry: ttm_resource_manager_for_each_res(man, &cursor, res) { bool busy; + if (!drmcs_evict_valuable(limitcss, man->cgdev, man->cgidx, res->css, try_low, &hit_low)) + continue; + if (!ttm_bo_evict_swapout_allowable(res->bo, ctx, place, &locked, &busy)) { if (busy && !busy_bo && ticket != @@ -623,13 +630,25 @@ int ttm_mem_evict_first(struct ttm_device *bdev, } if (!bo) { + if (!ticket && !try_low && hit_low) + goto hit_low; + if (busy_bo && !ttm_bo_get_unless_zero(busy_bo)) busy_bo = NULL; + + if (!busy_bo && !try_low && hit_low) + goto hit_low; + spin_unlock(&bdev->lru_lock); ret = ttm_mem_evict_wait_busy(busy_bo, ctx, ticket); if (busy_bo) ttm_bo_put(busy_bo); return ret; + +hit_low: + busy_bo = NULL; + try_low = true; + goto retry; } if (bo->deleted) { @@ -769,14 +788,19 @@ static int ttm_bo_alloc_resource(struct ttm_buffer_object *bo, continue; do { - ret = ttm_resource_alloc(bo, place, res); - if (unlikely(ret && ret != -ENOSPC)) + struct drmcgroup_pool_state *limitcss = NULL; + + ret = ttm_resource_alloc(bo, place, res, force_space ? &limitcss : NULL); + if (unlikely(ret && ret != -ENOSPC && ret != -EAGAIN)) { + drmcs_pool_put(limitcss); return ret; + } if (likely(!ret) || !force_space) break; ret = ttm_mem_evict_first(bdev, man, place, ctx, - ticket); + ticket, limitcss); + drmcs_pool_put(limitcss); if (unlikely(ret == -EBUSY)) break; if (unlikely(ret)) @@ -1162,7 +1186,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, memset(&hop, 0, sizeof(hop)); place.mem_type = TTM_PL_SYSTEM; - ret = ttm_resource_alloc(bo, &place, &evict_mem); + ret = ttm_resource_alloc(bo, &place, &evict_mem, NULL); if (unlikely(ret)) goto out; @@ -1201,7 +1225,9 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, if (locked) dma_resv_unlock(bo->base.resv); ttm_bo_put(bo); - return ret == -EBUSY ? -ENOSPC : ret; + if (ret == -EAGAIN || ret == -EBUSY) + return -ENOSPC; + return ret; } void ttm_bo_tt_destroy(struct ttm_buffer_object *bo) diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 4a66b851b67d..1a8312afe247 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -229,15 +230,31 @@ EXPORT_SYMBOL(ttm_resource_fini); int ttm_resource_alloc(struct ttm_buffer_object *bo, const struct ttm_place *place, - struct ttm_resource **res_ptr) + struct ttm_resource **res_ptr, + struct drmcgroup_pool_state **limitcs) { struct ttm_resource_manager *man = ttm_manager_type(bo->bdev, place->mem_type); + struct drmcgroup_pool_state *drmcs = NULL; int ret; + if (man->cgdev) { + ret = drmcg_try_charge(&drmcs, limitcs, + man->cgdev, man->cgidx, + bo->base.size); + if (ret) + return ret; + } + ret = man->func->alloc(man, bo, place, res_ptr); - if (ret) + if (ret) { + if (drmcs) + drmcg_uncharge(drmcs, man->cgdev, man->cgidx, + bo->base.size); return ret; + } + + (*res_ptr)->css = drmcs; spin_lock(&bo->bdev->lru_lock); ttm_resource_add_bulk_move(*res_ptr, bo); @@ -249,6 +266,7 @@ EXPORT_SYMBOL_FOR_TESTS_ONLY(ttm_resource_alloc); void ttm_resource_free(struct ttm_buffer_object *bo, struct ttm_resource **res) { struct ttm_resource_manager *man; + struct drmcgroup_pool_state *css; if (!*res) return; @@ -256,9 +274,13 @@ void ttm_resource_free(struct ttm_buffer_object *bo, struct ttm_resource **res) spin_lock(&bo->bdev->lru_lock); ttm_resource_del_bulk_move(*res, bo); spin_unlock(&bo->bdev->lru_lock); + + css = (*res)->css; man = ttm_manager_type(bo->bdev, (*res)->mem_type); man->func->free(man, *res); *res = NULL; + if (man->cgdev) + drmcg_uncharge(css, man->cgdev, man->cgidx, bo->base.size); } EXPORT_SYMBOL(ttm_resource_free); @@ -401,7 +423,7 @@ int ttm_resource_manager_evict_all(struct ttm_device *bdev, while (!list_empty(&man->lru[i])) { spin_unlock(&bdev->lru_lock); ret = ttm_mem_evict_first(bdev, man, NULL, &ctx, - NULL); + NULL, NULL); if (ret) return ret; spin_lock(&bdev->lru_lock); diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h index 6ccf96c91f3a..709a30b6bfe6 100644 --- a/include/drm/ttm/ttm_bo.h +++ b/include/drm/ttm/ttm_bo.h @@ -386,7 +386,8 @@ int ttm_mem_evict_first(struct ttm_device *bdev, struct ttm_resource_manager *man, const struct ttm_place *place, struct ttm_operation_ctx *ctx, - struct ww_acquire_ctx *ticket); + struct ww_acquire_ctx *ticket, + struct drmcgroup_pool_state *limitcg); vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo, struct vm_fault *vmf); vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, diff --git a/include/drm/ttm/ttm_resource.h b/include/drm/ttm/ttm_resource.h index 69769355139f..0574ebb226cb 100644 --- a/include/drm/ttm/ttm_resource.h +++ b/include/drm/ttm/ttm_resource.h @@ -38,6 +38,7 @@ #define TTM_MAX_BO_PRIORITY 4U #define TTM_NUM_MEM_TYPES 8 +struct drmcgroup_device; struct ttm_device; struct ttm_resource_manager; struct ttm_resource; @@ -174,6 +175,15 @@ struct ttm_resource_manager { * bdev->lru_lock. */ uint64_t usage; + + /** + * @cgdev: drmcgroup_device used for memory accounting, if not NULL. + */ + struct drmcgroup_device *cgdev; + /** + * @cgidx: Resource index used by this resource manager for cgroup accounting + */ + u32 cgidx; }; /** @@ -202,6 +212,7 @@ struct ttm_bus_placement { * @placement: Placement flags. * @bus: Placement on io bus accessible to the CPU * @bo: weak reference to the BO, protected by ttm_device::lru_lock + * @css: cgroup state this resource is charged to * * Structure indicating the placement and space resources used by a * buffer object. @@ -214,6 +225,8 @@ struct ttm_resource { struct ttm_bus_placement bus; struct ttm_buffer_object *bo; + struct drmcgroup_pool_state *css; + /** * @lru: Least recently used list, see &ttm_resource_manager.lru */ @@ -362,7 +375,8 @@ void ttm_resource_fini(struct ttm_resource_manager *man, int ttm_resource_alloc(struct ttm_buffer_object *bo, const struct ttm_place *place, - struct ttm_resource **res); + struct ttm_resource **res, + struct drmcgroup_pool_state **limitcs); void ttm_resource_free(struct ttm_buffer_object *bo, struct ttm_resource **res); bool ttm_resource_intersects(struct ttm_device *bdev, struct ttm_resource *res, From patchwork Thu Jun 27 15:47:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maarten Lankhorst X-Patchwork-Id: 13714717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DD1BC2BD09 for ; Thu, 27 Jun 2024 15:48:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38D9D6B0096; Thu, 27 Jun 2024 11:48:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 33F296B0099; Thu, 27 Jun 2024 11:48:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 190FA6B009A; Thu, 27 Jun 2024 11:48:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DEF6E6B0096 for ; Thu, 27 Jun 2024 11:48:11 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 94377A0B10 for ; Thu, 27 Jun 2024 15:48:11 +0000 (UTC) X-FDA: 82277099982.04.BA49817 Received: from mblankhorst.nl (lankhorst.se [141.105.120.124]) by imf14.hostedemail.com (Postfix) with ESMTP id 09F7710000A for ; Thu, 27 Jun 2024 15:48:09 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=none (imf14.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719503281; a=rsa-sha256; cv=none; b=hq6XwT4kbSHdoA0scXpy+dgoqiA5/1NQ9rJDwNJUyeh6mvrRglFZOlR1yIEZHQpaATrq1V Q3lFCaphX8PpL0ih6GPMOn6gEN/7nvbo0Ehz1f8wbkb32932NxZ9Cz3HHN0ooh0V+j+lvH iqLAZtajE3VC6+HIdZj/5wXOzUUcHOU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=none (imf14.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719503281; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ofPiDQ1ehYffKAtKWPwTW8iiyJRRsBbx21eivUic3V8=; b=Ii5aubNqQOFrCAh3xh8bvY2Wa1CylyhveKH7cmLTkXzgptgvN/Y4Tqv7s5769kNsX2v8uC e7UTII6QCBvRJ+sfPFVA0Q2kSKjHbN6KgRO74aNZpKa+K88GxX30ERnX306rNAvC7iHnpd IavOs/748AGFaDufXoeJM/966At3wes= From: Maarten Lankhorst To: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Lucas De Marchi , =?utf-8?q?Thomas_Hellstr=C3=B6m?= , Rodrigo Vivi , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter Cc: Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 4/6] drm/xe: Implement cgroup for vram Date: Thu, 27 Jun 2024 17:47:23 +0200 Message-ID: <20240627154754.74828-5-maarten.lankhorst@linux.intel.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> MIME-Version: 1.0 X-Stat-Signature: p66sp5rw5477sjigj1t1da3dq6meptdb X-Rspamd-Queue-Id: 09F7710000A X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1719503289-161652 X-HE-Meta: U2FsdGVkX196PFbb2wA5g9TE0jOistHz8md37+RC9WaVukgAWmYMX8NnB8nSoqUyMrjXzZjO6T28uX5GcdZ3dD0iTSMBKZ+FtjtA/aCPCCuC5OY3w8NXbhHyHlMvjbp5ZIoLreThJAfmamL5MXwEGbAyVWIGWD9gwB2kZH/3jRSsCRdVMocqeHqSkY7LiD+y4RlXG+02HR8/O9lLcbG18Pc/Xki05OLTQmig7MBK4XBzMzTN+VYc6C73+uo9GeLG96f2vphSmOcLB8ahJZmio8k8u5+/7FLtnM13NPmcxFeqZyMdqYXcYvqh4wRXtkTK9eMtkW6n0C73+RYc75MOWjk8Z+SsZOSGx7ENTiCHkqwoKfwfc9T18AK0/+v4vrH9RfhWnlFMoLsQrT9Dslztz0xZTYBnUjGTXyuzRu+3oo1kZFwbnLzmFJEM9dK1CNJvPy1op9E1VKgHXdGHAq48/FgCc9v4QF8+w4d0yVC+4O8p6bxHnjex+9h42daMI9s9HdKFX3K6xHcPQnf0Nou4g0DlOC6VC/HyCjH3UK4GaR+6xWCOj29idGq2MI82XjXzqaRJ9CfXiplTaqn+Ghf1zBFmaX3TnELuFcblox/4skF1lPckkO2F/NyempovPLfy/wDP0K7+whMZ/hWZ/Pp/mwd+BFVRP+CgSAtcINz5ikBv7TnlACd7+Glxkl7juYhiC4DTpwXcofnnRFWp/KpWjIaQbQQfTRUYKNNbzUU+imwGKgvi6Kujms3iK5cbw9rbFSY94fmKDWITEC+qnswC3E5XYRDz1tm3AxdSKxgtTGpZo6uED0Xj0gBUrzqbB1d8/jv2mPJFdnwA9jzHrEkhpEdjopvNqtGBlXG9M7bpozmBYoMHYo5DYUBOGaxpna220yIc+J7Kpj/xdiyy15c1ukR9DqXN5BHp75GT6ltO9B1HMieir/+t3hFycty6l9dYanUIDdMRa/hJepE1Jnw d2ACrWiM tdbVulpECwwlGcK2RXJEqjQjosYClbVU9JtrVKuvDmVcat07ZnXcFz6rgdcTVOpA0ZS7Hp1mU9eRC98FCqk8ml0y16rXdRUrd4XKKQufB/PRy8fm3E1ODSwiss+TWpr8z8IgIAhU0Zjoz/vfmJcwgKd3EocL4ir0Y8IFLWHA6onVpW3iXGcsmQptIxVCl0xSJXk0aiNZXCOsXZfv4yVEntzXFNQ/dJBKoIGlFSLaiTIAPQ48/K0r0mOjDXi1IiT4yq60dnvfaxk2VFgk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add vram based cgroup eviction to Xe. Most hardware with VRAM uses TTM for its management, and can be similarly trivially enabled. Signed-off-by: Maarten Lankhorst --- drivers/gpu/drm/xe/xe_device.c | 4 ++++ drivers/gpu/drm/xe/xe_device_types.h | 4 ++++ drivers/gpu/drm/xe/xe_ttm_vram_mgr.c | 10 ++++++++++ 3 files changed, 18 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c index 5ef9b50a20d0..6e630d67b13a 100644 --- a/drivers/gpu/drm/xe/xe_device.c +++ b/drivers/gpu/drm/xe/xe_device.c @@ -616,6 +616,10 @@ int xe_device_probe(struct xe_device *xe) /* Allocate and map stolen after potential VRAM resize */ xe_ttm_stolen_mgr_init(xe); + err = drmmcg_register_device(&xe->drm, &xe->cg); + if (err) + goto err_irq_shutdown; + /* * Now that GT is initialized (TTM in particular), * we can try to init display, and inherit the initial fb. diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h index 2e62450d86e1..6b60837d9090 100644 --- a/drivers/gpu/drm/xe/xe_device_types.h +++ b/drivers/gpu/drm/xe/xe_device_types.h @@ -7,6 +7,7 @@ #define _XE_DEVICE_TYPES_H_ #include +#include #include #include @@ -216,6 +217,9 @@ struct xe_device { /** @devcoredump: device coredump */ struct xe_devcoredump devcoredump; + /** @cg: drm cgroup bookkeeping */ + struct drmcgroup_device cg; + /** @info: device info */ struct intel_device_info { /** @info.graphics_name: graphics IP name */ diff --git a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c index fe3779fdba2c..9a625b69fc52 100644 --- a/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c +++ b/drivers/gpu/drm/xe/xe_ttm_vram_mgr.c @@ -339,6 +339,16 @@ int __xe_ttm_vram_mgr_init(struct xe_device *xe, struct xe_ttm_vram_mgr *mgr, struct ttm_resource_manager *man = &mgr->manager; int err; + if (mem_type != XE_PL_STOLEN) { + int cgregion = xe->cg.num_regions++; + + xe->cg.regions[cgregion].size = size; + xe->cg.regions[cgregion].name = + mem_type == XE_PL_VRAM0 ? "vram0" : "vram1"; + man->cgdev = &xe->cg; + man->cgidx = cgregion; + } + man->func = &xe_ttm_vram_mgr_func; mgr->mem_type = mem_type; mutex_init(&mgr->lock); From patchwork Thu Jun 27 15:47:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maarten Lankhorst X-Patchwork-Id: 13714718 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC39CC2BD09 for ; Thu, 27 Jun 2024 15:48:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 319F06B0099; Thu, 27 Jun 2024 11:48:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A06A6B009A; Thu, 27 Jun 2024 11:48:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F6826B009C; Thu, 27 Jun 2024 11:48:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E65F76B0099 for ; Thu, 27 Jun 2024 11:48:12 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 85FD0A0B43 for ; Thu, 27 Jun 2024 15:48:12 +0000 (UTC) X-FDA: 82277100024.07.52DC62F Received: from mblankhorst.nl (lankhorst.se [141.105.120.124]) by imf21.hostedemail.com (Postfix) with ESMTP id EA7201C0012 for ; Thu, 27 Jun 2024 15:48:10 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf21.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719503278; a=rsa-sha256; cv=none; b=bx065BstrWAdVkkR5xQZcOXnzI4z0RR+CDbOoE7b0frFAdAqCaAzaILv002meO91RUX7WH YDmi3J1RRTCVGM5raAyvokjM8cyAGAomQB5DMPhDCYNl6zByOHzkBiKxwio/VPyf/vjtSI 2D5wsZz14lukyuiId3gqZdh5fW+zIk8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf21.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719503278; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ve8LLrR7QK2UWxet92F6IrCINGp8NU1IxhY7RHhdh7A=; b=E48VY38IO0d7CFk6GczPKNL3WNy3/E+DYVIBU9w1XCidqGu326esabYsjC7pmNbjJsQXha hq17MYvEPa7Jj/uMTKRCXWPtUIeU2L0+IdYbEdt5bxqwrIZMqhYR/jmr4V1uURDzfP6glk w3ci30KNG4qcjBA/ESnDPZXIcaff2Y4= From: Maarten Lankhorst To: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= , "Pan, Xinhui" , David Airlie , Daniel Vetter Cc: Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org, Maarten Lankhorst , amd-gfx@lists.freedesktop.org Subject: [RFC PATCH 5/6] drm/amdgpu: Add cgroups implementation Date: Thu, 27 Jun 2024 17:47:24 +0200 Message-ID: <20240627154754.74828-6-maarten.lankhorst@linux.intel.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: EA7201C0012 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7wwtnnagq48wjqw9ryih7mmtf5eazgz7 X-HE-Tag: 1719503290-653514 X-HE-Meta: U2FsdGVkX1+rsfKnTH7mm9VHivNDbdNO5JlmwEc59+k4qylC3XcTxs/A3/KX18K5cQ6MFIKzdC8kAWzTLXJd78/NEW61jI/GktAJUFus7sgZ25n3SN9f22DPFWYihvYyRZitgv8DpW5CTKQ3/kqCZPZU+ImVOIF0KQ3MEcidgws5UvfEZfewGQ15t2IkqgQnFTbm5rSN6ZPWZy42oB7SYQKs6gOtHjCSWg7MxOhk2KeyqlPukURgThztFWuGvQD95chR1r5leOCN3dlsxJmymmaWLi8kAl/NKNAofNpIUyBvgbdC9W+a3G9BgMUXq+nsJu/Z8J6iJXF/3jtynTyvEepbspKxW26j3TkLLT6rlPsiBtR48ggV0yI91VEENYBV5oQJtHCN3ySJOXCTZ0eH5rW0oFpKPxB/TSxJWeIfErDh1CC1HxroEtcmT2RBPssIAQ180bmek9r3w+imHbH4GwKd4yGz4V7xHae9r8X8f+niVxS0hN7GHA7SV1u62IaYtPv8dpjYn4dHnuRMD3QjjoFB4VtPswAE6LJHl5SdUy/Aarn6gJR6Yu6jfpjBFKbz9Dz7wIfDb4jNpHVftd+0YjBPBKbCj4e5QF5O89suY3sgfrUp2EobKbWL4d80ntI30rBMNtpp0g5hddKj3dVLpmh8dG3rgRoGlDOwpgTg+cOCThgdCyn0wuUz14kJdeI998C3w58yA20PsukNVK3UUMeiL2l0DduPWgc1Jbk9DMqeqDwhbtlYRLeAkhQNTkJRevoh47pm9sm8m6JhRDczlOhmYucmC/liq+S8FFgDKRAAXxqwchKD1zO/7kaG2/wZSPROAWsql33i0KS4cRHfDHUIbXY01mbKUzfJj0J+exvb2B07cSsYQvTo3i3B8ijVm9VRrf6bz2I/vaMoG7FY2rlBbQdFhIe6byywTMAdUFoP41PoWxlgZHn4VeF/u8a+7r/vwgGlrxydUKCkqpw VXYJ/iPd 2/l+N41G09VajK8xtOoD+mgCe/g74syzLcpbPyRnjP/7WuSxtDbwJwLeU4xO7Pviv7/0Q/nYOoIR/oyiTz7tQ8vbA52ZXSUkf8BN8mcsgv0l1HWuY/KDMZzKAX/zZyZ+bYLMenFFWb9T1+gBqP5ZaAd8fQ7D6Hc2d0w0L7sNDkUACwKUwMsiEkmILmihVJnT+vK8bYxm68ZSkTGZ2Zi2nAbTvhnnGIDWtm74biLQeibcCIGjjNHYPUVk5csz23tv17EGWnVgTv46Ad6s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Similar to xe, enable some simple management of VRAM only. Signed-off-by: Maarten Lankhorst --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 6 ++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 6 ++++++ 3 files changed, 14 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index f87d53e183c3..9ec55f05bfd5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -43,6 +43,7 @@ #include "amdgpu_ctx.h" #include +#include #include #include #include @@ -836,6 +837,7 @@ struct amdgpu_device { struct device *dev; struct pci_dev *pdev; struct drm_device ddev; + struct drmcgroup_device cg; #ifdef CONFIG_DRM_AMD_ACP struct amdgpu_acp acp; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index e785f128411d..1afe28016d40 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1856,6 +1856,12 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) return r; } + r = drmmcg_register_device(adev_to_drm(adev), &adev->cg); + if (r) { + DRM_ERROR("Failed initializing cgroup allocator.\n"); + return r; + } + /* Change the size here instead of the init above so only lpfn is affected */ amdgpu_ttm_set_buffer_funcs_status(adev, false); #ifdef CONFIG_64BIT diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 6c30eceec896..b207af0bff3a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -895,6 +895,12 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) DRM_INFO("Setup dummy vram mgr\n"); } + adev->cg.regions[0].size = adev->gmc.real_vram_size; + adev->cg.regions[0].name = "vram"; + adev->cg.num_regions++; + man->cgdev = &adev->cg; + man->cgidx = 0; + ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; From patchwork Thu Jun 27 15:47:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maarten Lankhorst X-Patchwork-Id: 13714719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABCB0C2BD09 for ; Thu, 27 Jun 2024 15:48:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 230DB6B009A; Thu, 27 Jun 2024 11:48:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1DF7B6B009C; Thu, 27 Jun 2024 11:48:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05A586B009D; Thu, 27 Jun 2024 11:48:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DC6F46B009A for ; Thu, 27 Jun 2024 11:48:13 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 68D3EA3D5C for ; Thu, 27 Jun 2024 15:48:13 +0000 (UTC) X-FDA: 82277100066.24.E43554F Received: from mblankhorst.nl (lankhorst.se [141.105.120.124]) by imf16.hostedemail.com (Postfix) with ESMTP id C37C518001D for ; Thu, 27 Jun 2024 15:48:11 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf16.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719503277; a=rsa-sha256; cv=none; b=cq48KOcvUvEyhD5NMHwKSkLN/TZEe9b1tYRUUDqciehDKeDePpibRSHMY2B++dIAU1fwAi HQMXKD6daT5UFP1HSrSRY96o88fDdITtV5cbAuHqzJg2PEUEfhE3F+ZamBGptZgR9t+KHQ jnjAx6D6Bq+5R1UrmxyR/vbyMkR88Qk= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf16.hostedemail.com: domain of mlankhorst@mblankhorst.nl has no SPF policy when checking 141.105.120.124) smtp.mailfrom=mlankhorst@mblankhorst.nl ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719503277; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=C3jab/nsujH/XyxuYZaarvB+XE0upxWZFBa65SXf6PA=; b=CpVlZu3TW/xCwZtyOjS9FZT8ucW6uX6yJt4QtUiZNiMm8w+CoahCGwdURQ0iKrk07Ho6o3 JFknsIhkI+MqH/v7H1jjGs4QRfpxdEohLuEAynzB+8fxnu6JhreD1tFmWymMPqwn+viXYM cwvovp6Mkr0L+78LJUErP1Mn+jEXP+w= From: Maarten Lankhorst To: intel-xe@lists.freedesktop.org, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tejun Heo , Zefan Li , Johannes Weiner , Andrew Morton , Lucas De Marchi , =?utf-8?q?Thomas_Hellstr=C3=B6m?= , Rodrigo Vivi , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Daniel Vetter Cc: Friedrich Vock , cgroups@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 6/6] drm/xe: Hack to test with mapped pages instead of vram. Date: Thu, 27 Jun 2024 17:47:25 +0200 Message-ID: <20240627154754.74828-7-maarten.lankhorst@linux.intel.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> References: <20240627154754.74828-1-maarten.lankhorst@linux.intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C37C518001D X-Stat-Signature: o4aorfkrfjaiqkj3mx47gw5r4r6en9rn X-Rspam-User: X-HE-Tag: 1719503291-41196 X-HE-Meta: U2FsdGVkX19uQc60N4p6ncRr2VDQQAkRWhirPKpZzdCB1EACUIrYzC8ffoPwmkSRM4CBIjg9X86aKfi3wkafnGLX8zjzLK+2gChG/yjsL4YVZPUsL0o6MjNevrcsX59gJ9wPzsQsEXW1yarpP8jUrf3DgyZMjARP0j/JfmyXwinuIBXzd7NXvvK0l4pYANMmKejufWrWNpqomXw/HVgb1VZhDk/h1zIBqWZlofDNfnRpYvTSZU1gLFdB36J+cTO/733BYGBlsQ+abedeDd11/r0hKOOQE7vX9ubRvxqpnqgXDsREmXCCJU7fZDJmVT8LXOdNk97me9VLjlXEdGzGXV/HjBmogPFJOsK6HJ1EGEOcaSeA1QF6zEiTJnsDO+nBA+nyGWkz23ts3xkuzbbiyZKaAKsH8lsXGrQGBZDU64UQEKM+jkEzdCIj0Q8wK3noO5AHcIxuCq2i0tMiaYE1x0MxvITNIYQSQfb46ujlf8mbY5G9GG5a9HOrG7odVZ4GAgOcPWbJLBGoTNPdk5gUOe5aaiMm9+w3o+hqQ/MF0BqlaK13L6heCTCV+sGg6gBe4UZNA3xiQl/V8iZ6zapgZ4hDQyuuZKDoHIcVcG/iSJP7zfnQ3KZinE7KYmP84R9Jk7t62V/ry1xwFd/dgxMx47YX27VaigWGpVRr1U6hhXaQsR6lGUVGu6DNIxodJQ/uucESfJqvwAs9xWuDiCwo5W6UBmUHQSUdt0JzhR8iIcwUZRg0DU8gN0dNoIkO7UKKSDrUFPBGnW0P9CkLLFpp6fR/ms8IEEDdJhHUzrvfeXzqBVAF3P3gssRzjtIBVn6R722PE8L1/ZjbwRf3DbyXDVPveRDWVILwU941kOUp/xxxzZHIkvmeYdrd2Rl4Fa7hiwhs8o55DQ9Z0hK4RoI+7BFINsKHCi2BN9JYOSS+WInhkI5rqzqQAUNF5+KnNS7WEEQvolwAPTjzUufcwLD KCzr+9e6 hCIw9aI9tXp37KGPxw0JhnE9V3yEfv8hywxTzh62CddMP2p5TSSg4NpaBKvu6YmncUSwC4siWzYrOcrA0Tx7jXUtUCFjdi3oHEGWNFAzEewFGQ0L+lo8PomJzeTb9H7zy7yY1tVQ/soYePlItr0R5Jl2NSAAixR/n/pz3TN43AXF0Lv7FRsatssLBauItdfoTArpd7f3l1/rprgsNvHiiMp9UKObdKElGWmrF0R94QzrqnH7g/KwMGyG04kqHbpRdXuIJRGsA/9vR1OI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We will probably want to make this a proper region in TTM for everything, so that we can charge VRAM twice, once for mapped in sysmem, once for mapped in vram. That way we don't need to deal with evict failing from lack of available memory in mapped. Signed-off-by: Maarten Lankhorst --- drivers/gpu/drm/xe/xe_ttm_sys_mgr.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_ttm_sys_mgr.c b/drivers/gpu/drm/xe/xe_ttm_sys_mgr.c index 9844a8edbfe1..20fa8ec8925e 100644 --- a/drivers/gpu/drm/xe/xe_ttm_sys_mgr.c +++ b/drivers/gpu/drm/xe/xe_ttm_sys_mgr.c @@ -101,6 +101,18 @@ static void ttm_sys_mgr_fini(struct drm_device *drm, void *arg) ttm_set_driver_manager(&xe->ttm, XE_PL_TT, NULL); } +static inline void apply_cg(struct xe_device *xe, + struct ttm_resource_manager *man, + u64 gtt_size) +{ + int cgregion = xe->cg.num_regions++; + + xe->cg.regions[cgregion].size = gtt_size; + xe->cg.regions[cgregion].name = "mapped"; + man->cgdev = &xe->cg; + man->cgidx = cgregion; + +} int xe_ttm_sys_mgr_init(struct xe_device *xe) { struct ttm_resource_manager *man = &xe->mem.sys_mgr; @@ -116,6 +128,8 @@ int xe_ttm_sys_mgr_init(struct xe_device *xe) man->func = &xe_ttm_sys_mgr_func; ttm_resource_manager_init(man, &xe->ttm, gtt_size >> PAGE_SHIFT); ttm_set_driver_manager(&xe->ttm, XE_PL_TT, man); + apply_cg(xe, man, gtt_size); + ttm_resource_manager_set_used(man, true); return drmm_add_action_or_reset(&xe->drm, ttm_sys_mgr_fini, xe); }