From patchwork Tue Oct 8 04:43:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13826535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71BA6CEF17A for ; Tue, 8 Oct 2024 14:00:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4DDDC6B009A; Tue, 8 Oct 2024 10:00:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 440206B009B; Tue, 8 Oct 2024 10:00:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F6A16B009C; Tue, 8 Oct 2024 10:00:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id ED5D56B009A for ; Tue, 8 Oct 2024 10:00:56 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C8BC216015F for ; Tue, 8 Oct 2024 14:00:55 +0000 (UTC) X-FDA: 82650596112.30.2BA0ACD Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf25.hostedemail.com (Postfix) with ESMTP id 9F016A000E for ; Tue, 8 Oct 2024 14:00:54 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=nyUtUPPZ; spf=pass (imf25.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.176 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728395919; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BnKzHoFsIxSKvnCi298qgtwKBJFTSXeEqlrSeNMpfd8=; b=eWQ0XXunS3Bdh4CWPvTL/S5SmhQrwh216lYTNfP8dQ3dm/1ykoQdy1Tq2l7tHy1W76e3WA AMpljI38HMRlHg7cTlmrCj27KbBq4IjyxeeIk/rhtZD0bfIiHN+3dqz9ubirr5TK7bJwOn w1XNUUVsPwY5DOFAOl7Se6ChXQWiX8s= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728395919; a=rsa-sha256; cv=none; b=b8J1+0jm79gw3luGNMjbCNrVKkqCMgFouFvR7MRZHKQHBFWZJmTdnN6JJVx2V4xua0Z6lO /AdFLErF0YMYA7v/tuIMPKotuBzzYSyhwZG3qe5ayI98W/W1EW0JYgT+aiSPOinxGKKqHf u8wBidu28+urbEiFVfqPnP8o/bHDhuY= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=nyUtUPPZ; spf=pass (imf25.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.176 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-7afcf0625a9so5276885a.0 for ; Tue, 08 Oct 2024 07:00:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728396053; x=1729000853; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BnKzHoFsIxSKvnCi298qgtwKBJFTSXeEqlrSeNMpfd8=; b=nyUtUPPZY1ntY8aw01/+UHE0YkNo1zj8/pBydPAGE8PBVay9pgZiiU9j+5IgYxrQQT U3nBIWWVnULvDjEfhH2mW2tgAnicnFF8Fq2ND+K7XzGUZccbqwkwMljd4IhWZGcnlGwv mO/FpYhuZYiod9rX1KbDzRx0ygu7knF4UWJvdPDxQsh2FcN+Q22DEA9ROXiYes8s7BI+ fxwgwjQ1dF9r5clbfIVP5oPrAqi2tKVvFAyc0sOBQZQP2oYqJ9EHXWnsWIjHZwHNKlVR mfd09JnumN6FpEoMELfEgA3AXebz4Cn0l1yOyUo0+yCswdcM9vNWzjHyuwIhgU7tCrHW 3JfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728396053; x=1729000853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BnKzHoFsIxSKvnCi298qgtwKBJFTSXeEqlrSeNMpfd8=; b=O57CjsYtcDIcA3jUqnBvVoi7JfqLIPTk6NZJhnEjKYbXK/MXWvVD82Dl2F3domeZ2A 1LlhodAcfWMls5auc6ixi7X4jXpVE2DuhC6Tke19vGVPaj2/EA+wbhudvQffRyRXweUR QTXG7rC9PsGVyDB8HpNwQBcAs76onEsMxQvaguiHdePk4alSbwNJ+xrYRUPwj4kUGhrC 9JrP4KIE7wCY6wmiyIR4eOeNhzrb4oLoNyqpx5hRuDdETLkOqqk2/C95RjNhm4WgzBBp Q3IJYR84WlnE3ozD8wLogxisPSgFO+A1jcqxYmFZ0y3S3daljKG1ZkxivcFkDsMSk3Nt orUw== X-Forwarded-Encrypted: i=1; AJvYcCXYZL4DknqiWLHzXZJCWPp35b4zQg7xiu0yoCCXAu/6Vm5rpt8s+biEHCA+WaQGiBCJQwHp1Cv8NQ==@kvack.org X-Gm-Message-State: AOJu0YzceLkdg7Sxr4j0jyIOD+CAHNn8VTsq0iNrEfsb0UshJ1uedMr/ 5hOPLssjjGoW6xTj5VpPQYQWz8wSLAq/iLTb3NI6Uw6vgN8o9xGDg56XTHBgaRQ= X-Google-Smtp-Source: AGHT+IEWQaL+bYEjt3RyVfxhNXMy9Cz/JlOtCh7gn34GRctxubICCfzpYSUr2ZJbwepGsgKWAr958Q== X-Received: by 2002:a0c:e5c4:0:b0:6cb:a705:c05e with SMTP id 6a1803df08f44-6cba705c0aamr170960556d6.6.1728396053465; Tue, 08 Oct 2024 07:00:53 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cbc78650efsm698396d6.39.2024.10.08.07.00.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Oct 2024 07:00:53 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org, linux-acpi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, david@redhat.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, akpm@linux-foundation.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, alison.schofield@intel.com, rrichter@amd.com, terry.bowman@amd.com, lenb@kernel.org, dave.jiang@intel.com, ira.weiny@intel.com Subject: [PATCH 3/3] acpi,srat: reduce memory block size if CFMWS has a smaller alignment Date: Tue, 8 Oct 2024 00:43:55 -0400 Message-ID: <20241008044355.4325-4-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241008044355.4325-1-gourry@gourry.net> References: <20241008044355.4325-1-gourry@gourry.net> MIME-Version: 1.0 X-Stat-Signature: itd5n6455qjn7kx46tj4xoz7e59zourh X-Rspamd-Queue-Id: 9F016A000E X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728396054-130892 X-HE-Meta: U2FsdGVkX18bBPbfDhNoqpX6dozX+EGA15T+AXjBmle/unHaIDikPDss4IM9Ci94tjDig6lmxVD9JHZ8ZgWPtbuDEANvxjToqCkATflRu54SML/OfImBPhkwX7qCWjv6ignz5E+GcjbatQ+GbtyXlfuJiLuLm12d/RPBmMO+eVjhTvqNjC8hkOoHLHmAiFLzJoFsFK+PqlYvQfr6gdXfgf7+U9ijAxyOkw3xPHbE9zc95ZaJQSglIR3dKRta3bQLC6WV5zVIy8zzJigHsjHo5W0XCxK7io3ZGttSP4VwSRSyPS9g/64ywETGqwxKAZAXZOZ0EsA50wyFQGs1+DKmbEVZnzvlQCfS4h7vHCtFLEz+JsRSCSf1nwN4tKHSEZyLKCPTROtsRTlyRhNXKn55vcmDX6bycpJmCm64h/3cYpZr/zFEvV66jbLC7CLqf6xOHeAC02ruSDTiVMZi+GQRnVb2RbuxcmNjezFsAOvHOV9TKxHR7T2MWyzGwmML9GZbtq+A+5itu7MXYwhd5scgOyQiz57Aq8Py4muXMpE4hIriMfAV2HjWfwWIeRCFW05LG+qKzG3dzAdyP1CfzfjqbyLEbeu3k7y0Wl3rDyQZCDUry+6MTHtirAfmrbULqREkxKS7QI43x63FC/wnjgFMaqcmkb7J6zPikgT6C/hCSxP/3qnCmELzbDqYvdNU68l5EdlBAWgaW1Gy5u/T/8ggcdx3D8w6LnLL7i2Kh7AKVsCq8qOcu6qymko4P5tKdhs7J1RRI60aE8LQtvEC0vQlEBMS2FMKrXPh8oqq+542E1HQj/yhJXIBUjgbMgac2Ghx2uVgYl6/x0WutU8olnWFYIvzbY8wjYyt86eGj4ryAC8rhk0kjR4Ag3ntse2wrRuVUIVV5C2h+bKyOZoRSl6Fawqp8CjesQXIgvKnPfkm5XmoIAIstpy8IO06wMbXaybJHXrdVzNWEy0BXIYH5RA o0ZJ6wDd BQ8KjxSGXB7TW2D22GWGNpIlL60o7oMbUNnNaWqawUALX/14wskhDeXz9d82qO6/nlvztct787fPUJONZsuE5ejZt+ELO1/GSLmV4fmx7rKAdgubg6W1XV9Lz95Ixj1R+Jc73kEHZG7BsxYQOy9f4pS5fzWyObM/M4DFR6WoX0oQmJV+tSZ+kIn9zS17F2bDy81SlCo/wN+pPI+caljMnkKWFm+sWs5YXiEJwPnMH4w9trAJ5nYOWDBObcxFKtx98cJOBhbrj5I9W7CRhJU7BAheExOc94NL31eFHPI4no12ZlifA/IWDSPzp8hPsMzGp09To+/GK04/kqWI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The CXL Fixed Memory Window allows for memory aligned down to the size of 256MB. However, by default on x86, memory blocks increase in size as total System RAM capacity increases. On x86, this caps out at 2G when 64GB of System RAM is reached. When the CFMWS regions are not aligned to memory block size, this results in lost capacity on either side of the alignment. Parse all CFMWS to detect the largest common denomenator among all regions, and reduce the block size accordingly. This can only be done when MEMORY_HOTPLUG and SPARSEMEM configs are enabled, but the surrounding code may not necessarily require these configs, so build accordingly. Suggested-by: Dan Williams Signed-off-by: Gregory Price --- drivers/acpi/numa/srat.c | 48 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c index 44f91f2c6c5d..9367d36eba9a 100644 --- a/drivers/acpi/numa/srat.c +++ b/drivers/acpi/numa/srat.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -333,6 +334,37 @@ acpi_parse_memory_affinity(union acpi_subtable_headers *header, return 0; } +#if defined(CONFIG_MEMORY_HOTPLUG) +/* + * CXL allows CFMW to be aligned along 256MB boundaries, but large memory + * systems default to larger alignments (2GB on x86). Misalignments can + * cause some capacity to become unreachable. Calculate the largest supported + * alignment for all CFMW to maximize the amount of mappable capacity. + */ +static int __init acpi_align_cfmws(union acpi_subtable_headers *header, + void *arg, const unsigned long table_end) +{ + struct acpi_cedt_cfmws *cfmws = (struct acpi_cedt_cfmws *)header; + u64 start = cfmws->base_hpa; + u64 size = cfmws->window_size; + unsigned long *fin_bz = arg; + unsigned long bz; + + for (bz = SZ_64T; bz >= SZ_256M; bz >>= 1) { + if (IS_ALIGNED(start, bz) && IS_ALIGNED(size, bz)) + break; + } + + /* Only adjust downward, we never want to increase block size */ + if (bz < *fin_bz && bz >= SZ_256M) + *fin_bz = bz; + else if (bz < SZ_256M) + pr_err("CFMWS: [BIOS BUG] base/size alignment violates spec\n"); + + return 0; +} +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ + static int __init acpi_parse_cfmws(union acpi_subtable_headers *header, void *arg, const unsigned long table_end) { @@ -501,6 +533,10 @@ acpi_table_parse_srat(enum acpi_srat_type id, int __init acpi_numa_init(void) { int i, fake_pxm, cnt = 0; +#if defined(CONFIG_MEMORY_HOTPLUG) + unsigned long block_sz = memory_block_size_bytes(); + unsigned long cfmw_align = block_sz; +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ if (acpi_disabled) return -EINVAL; @@ -552,6 +588,18 @@ int __init acpi_numa_init(void) } last_real_pxm = fake_pxm; fake_pxm++; + +#if defined(CONFIG_MEMORY_HOTPLUG) + /* Calculate and set largest supported memory block size alignment */ + acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_align_cfmws, + &cfmw_align); + if (cfmw_align < block_sz && cfmw_align >= SZ_256M) { + if (set_memory_block_size_order(ffs(cfmw_align)-1)) + pr_warn("CFMWS: Unable to adjust memory block size\n"); + } +#endif /* defined(CONFIG_MEMORY_HOTPLUG) */ + + /* Then parse and fill the numa nodes with the described memory */ acpi_table_parse_cedt(ACPI_CEDT_TYPE_CFMWS, acpi_parse_cfmws, &fake_pxm);