From patchwork Mon Feb 6 01:04:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13129266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80BEBC636CD for ; Mon, 6 Feb 2023 01:04:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C6B46B007B; Sun, 5 Feb 2023 20:04:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 176FE6B007E; Sun, 5 Feb 2023 20:04:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03E696B0087; Sun, 5 Feb 2023 20:04:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EA0E76B007B for ; Sun, 5 Feb 2023 20:04:08 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C6575AAB19 for ; Mon, 6 Feb 2023 01:04:08 +0000 (UTC) X-FDA: 80435070576.01.67D0A68 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf18.hostedemail.com (Postfix) with ESMTP id 86BE21C0015 for ; Mon, 6 Feb 2023 01:04:06 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GK1gs7gj; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675645446; a=rsa-sha256; cv=none; b=5UQUN47RVOnxkAbId0aSMsnwLo20sfksRHWRkGpfNEDjOonnY695vHRySq/XXc+4uYH9aR lmX+vT/dA2IebRLTlKvuAzfwywbfbNS9tX+Sak3jGX/fogKIegBJwvMChy8fn2tXtTBpQI jJhOFmqGcQiYYRX3N/0NssvxMcGcbN4= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=GK1gs7gj; spf=pass (imf18.hostedemail.com: domain of dan.j.williams@intel.com designates 192.55.52.151 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675645446; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2XqSYO6eKicwcLlCsbrsut86hBWoVutw8KDqDZ9u4tI=; b=0ZybmgsIizjVbdWPDFLwvB/d6DEm6UreyEK+Vq33R/9E+vq3fT+PhwKj3qBp0wG9pcPcD2 wElFbd2KWfdXzAxP2XccDbA9eKhYwNZOF52KMCBzv6TnPxOyR7vzU/DqfxzGFFN/z8Roib UKPWZeGW0y7JPowEgL0n2kJzyOyEuHM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675645446; x=1707181446; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hrvnto86ChTGsi4fHJ4Lqo9DG2au7l7fsMPus7NStq0=; b=GK1gs7gjNinPobEQFOc3k1Nm8HHd2JJM9E/4bB4HdriIxfEMKG5XNFt4 Jd5gEST7yqgEKCU9g485QmHjb1eas/X6CWWiF5J3NKnaaQCQ6Y0Usl0tH 5Jxi0PPg6cdsoLbfgzzLaWATrN8ECoQ8FpYmIqw26RJjxotTkde0r6sEu JDf59wAnKEN9RTg3MmJZaKr7n5oYDm+Khup3md3v11wwwjcp6n7sO4dTd h2AGgKld6nS6NzNNLBqKLsmE5jpj7Qjwg0tdkFQvfRrdmFsKiG9UjglUx prwWkl53zzNWPOhyljA7QUOC3JSDkk1lx1AhNnLbaYpAHvjzovuJo/qNG Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="309442703" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="309442703" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:04:06 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10612"; a="775006766" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="775006766" Received: from mkrysak-mobl.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.212.255.187]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Feb 2023 17:04:05 -0800 Subject: [PATCH 17/18] dax: Assign RAM regions to memory-hotplug by default From: Dan Williams To: linux-cxl@vger.kernel.org Cc: Michal Hocko , David Hildenbrand , Dave Hansen , linux-mm@kvack.org, linux-acpi@vger.kernel.org Date: Sun, 05 Feb 2023 17:04:05 -0800 Message-ID: <167564544513.847146.4645646177864365755.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> References: <167564534874.847146.5222419648551436750.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 86BE21C0015 X-Rspamd-Server: rspam01 X-Stat-Signature: 45u41n7otj1oad75bx3j841k615w71i8 X-HE-Tag: 1675645446-859271 X-HE-Meta: U2FsdGVkX1/crWvcmhmD6SFMdV6SlwDZOzOpJsRVzAaaveb59n/75yxUNpgscuvpMhcd/pgU9t9GdEi5KnvIllcmLs0DEsGxrfGuqL5uty7yrwrf7sm0zjkySnwsAmCkVtxp5GHK1FgCXjUZwl/8i1AdI2O+GwK+hbBc5qHSaSbt3Zb/usOXmcBMISBft4bNDXDj+zyupxZbnCJMuaNm4qnhFtg1W6N7KbRvEf4qhkwC1UePBXJCrWsCBdrWxxemx2cctrCgJNsdgO90z2WsaP1S2JLjXllizjn6bW2ZTOKl59RQ02NcyXwTS+yz1bGMftojJh3kBuFHEZjAsTE0x3EJ+bvNB5Xju3vVZAkIbNgBEAH7wob4IdaT8yYndXZxW29ITgKYjYOkisbpwBKVAVEjU1WA1aC6IuEPIZ7Lex2oBoUEXD19ojo+L6XtwIF7MjNoqZzufJEyOmXBVTs2AekDpLsm0fjRcYOzhF7jzWcLxeo912ebGA91BwpaU+VINCl/p3y4x8Z9X4SXCMzU00WjonDc98Xq3bS9RzGij98ieX9nPbQ8HIA+ZO8ambufOsv7ybbc0eeD62Kh8eSJfPOk7GhTAQ5tcv4lLuFcB9wqaXFkabKbRlzIwVxjc7ght3FVSHS3vphXg+tYMWfFLq0C3ak15L2Wo2Bk8fJgjWZJTnX3koisSfVYHchDx+HfUgGRacnyFxl9k5ILbzJ5eXnl4FhJRJAObRGPFfpX2v/Lr7/gs/kpW32f04Xgi+sgXPKe0cULTdICF09/8f7U/W/ceY4sVIYqNsESZ6h4Pz0IyWpGaitUTwacSetctAuFOZJm6NN6wUrWps0XpKd5loGe1dWkwo80QTOe3sCemMP6aGoOIiRWyvNro43IoY3ZJSjjAWRUyD0b4JYALXHOJbVoF8RAHd8B9xx/151NzIdymCadt7WeSRiXtKV29AEWG/BFMRrU6Eg2X2sCGOZ q4E+5N65 bx7RwVn04cY/1ytPZaEdcD/7IvpLNNd5PlAwBvgiMrckluw39no1fgRcpF2+Jr1+5cMuoc0uxM4T6c/rulXtacWfJgiAb+GLRZtsG+XdN7gv1f46mRyFC95Yt1YKbxCYkvnGRB7eALMtQb9QHRr/dh0bZHsLiGq9Kr6QVwe4XC1tpz0jBI/cYToJ9JN75UKDAmCoJoWjKrrp07ezSfXJa1IiKzgbHDfSErp8KAmUobHH5TpZct6CnINW3+2NesHO7WWb6jfFYbry6GRg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The default mode for device-dax instances is backwards for RAM-regions as evidenced by the fact that it tends to catch end users by surprise. "Where is my memory?". Recall that platforms are increasingly shipping with performance-differentiated memory pools beyond typical DRAM and NUMA effects. This includes HBM (high-bandwidth-memory) and CXL (dynamic interleave, varied media types, and future fabric attached possibilities). For this reason the EFI_MEMORY_SP (EFI Special Purpose Memory => Linux 'Soft Reserved') attribute is expected to be applied to all memory-pools that are not the general purpose pool. This designation gives an Operating System a chance to defer usage of a memory pool until later in the boot process where its performance properties can be interrogated and administrator policy can be applied. 'Soft Reserved' memory can be anything from too limited and precious to be part of the general purpose pool (HBM), too slow to host hot kernel data structures (some PMEM media), or anything in between. However, in the absence of an explicit policy, the memory should at least be made usable by default. The current device-dax default hides all non-general-purpose memory behind a device interface. The expectation is that the distribution of users that want the memory online by default vs device-dedicated-access by default follows the Pareto principle. A small number of enlightened users may want to do userspace memory management through a device, but general users just want the kernel to make the memory available with an option to get more advanced later. Arrange for all device-dax instances not backed by PMEM to default to attaching to the dax_kmem driver. From there the baseline memory hotplug policy (CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE / memhp_default_state=) gates whether the memory comes online or stays offline. Where, if it stays offline, it can be reliably converted back to device-mode where it can be partitioned, or fronted by a userspace allocator. So, if someone wants device-dax instances for their 'Soft Reserved' memory: 1/ Build a kernel with CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=n or boot with memhp_default_state=offline, or roll the dice and hope that the kernel has not pinned a page in that memory before step 2. 2/ Write a udev rule to convert the target dax device(s) from 'system-ram' mode to 'devdax' mode: daxctl reconfigure-device $dax -m devdax -f Cc: Michal Hocko Cc: David Hildenbrand Cc: Dave Hansen Signed-off-by: Dan Williams Reviewed-by: Gregory Price --- drivers/dax/Kconfig | 2 +- drivers/dax/bus.c | 53 ++++++++++++++++++++--------------------------- drivers/dax/bus.h | 12 +++++++++-- drivers/dax/device.c | 3 +-- drivers/dax/hmem/hmem.c | 12 ++++++++++- drivers/dax/kmem.c | 1 + 6 files changed, 46 insertions(+), 37 deletions(-) diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig index d13c889c2a64..1163eb62e5f6 100644 --- a/drivers/dax/Kconfig +++ b/drivers/dax/Kconfig @@ -50,7 +50,7 @@ config DEV_DAX_HMEM_DEVICES def_bool y config DEV_DAX_KMEM - tristate "KMEM DAX: volatile-use of persistent memory" + tristate "KMEM DAX: map dax-devices as System-RAM" default DEV_DAX depends on DEV_DAX depends on MEMORY_HOTPLUG # for add_memory() and friends diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 1dad813ee4a6..012d576004e9 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -56,6 +56,25 @@ static int dax_match_id(struct dax_device_driver *dax_drv, struct device *dev) return match; } +static int dax_match_type(struct dax_device_driver *dax_drv, struct device *dev) +{ + enum dax_driver_type type = DAXDRV_DEVICE_TYPE; + struct dev_dax *dev_dax = to_dev_dax(dev); + + if (dev_dax->region->res.flags & IORESOURCE_DAX_KMEM) + type = DAXDRV_KMEM_TYPE; + + if (dax_drv->type == type) + return 1; + + /* default to device mode if dax_kmem is disabled */ + if (dax_drv->type == DAXDRV_DEVICE_TYPE && + !IS_ENABLED(CONFIG_DEV_DAX_KMEM)) + return 1; + + return 0; +} + enum id_action { ID_REMOVE, ID_ADD, @@ -216,14 +235,9 @@ static int dax_bus_match(struct device *dev, struct device_driver *drv) { struct dax_device_driver *dax_drv = to_dax_drv(drv); - /* - * All but the 'device-dax' driver, which has 'match_always' - * set, requires an exact id match. - */ - if (dax_drv->match_always) + if (dax_match_id(dax_drv, dev)) return 1; - - return dax_match_id(dax_drv, dev); + return dax_match_type(dax_drv, dev); } /* @@ -1413,13 +1427,10 @@ struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data) } EXPORT_SYMBOL_GPL(devm_create_dev_dax); -static int match_always_count; - int __dax_driver_register(struct dax_device_driver *dax_drv, struct module *module, const char *mod_name) { struct device_driver *drv = &dax_drv->drv; - int rc = 0; /* * dax_bus_probe() calls dax_drv->probe() unconditionally. @@ -1434,26 +1445,7 @@ int __dax_driver_register(struct dax_device_driver *dax_drv, drv->mod_name = mod_name; drv->bus = &dax_bus_type; - /* there can only be one default driver */ - mutex_lock(&dax_bus_lock); - match_always_count += dax_drv->match_always; - if (match_always_count > 1) { - match_always_count--; - WARN_ON(1); - rc = -EINVAL; - } - mutex_unlock(&dax_bus_lock); - if (rc) - return rc; - - rc = driver_register(drv); - if (rc && dax_drv->match_always) { - mutex_lock(&dax_bus_lock); - match_always_count -= dax_drv->match_always; - mutex_unlock(&dax_bus_lock); - } - - return rc; + return driver_register(drv); } EXPORT_SYMBOL_GPL(__dax_driver_register); @@ -1463,7 +1455,6 @@ void dax_driver_unregister(struct dax_device_driver *dax_drv) struct dax_id *dax_id, *_id; mutex_lock(&dax_bus_lock); - match_always_count -= dax_drv->match_always; list_for_each_entry_safe(dax_id, _id, &dax_drv->ids, list) { list_del(&dax_id->list); kfree(dax_id); diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h index fbb940293d6d..8cd79ab34292 100644 --- a/drivers/dax/bus.h +++ b/drivers/dax/bus.h @@ -11,7 +11,10 @@ struct dax_device; struct dax_region; void dax_region_put(struct dax_region *dax_region); -#define IORESOURCE_DAX_STATIC (1UL << 0) +/* dax bus specific ioresource flags */ +#define IORESOURCE_DAX_STATIC BIT(0) +#define IORESOURCE_DAX_KMEM BIT(1) + struct dax_region *alloc_dax_region(struct device *parent, int region_id, struct range *range, int target_node, unsigned int align, unsigned long flags); @@ -25,10 +28,15 @@ struct dev_dax_data { struct dev_dax *devm_create_dev_dax(struct dev_dax_data *data); +enum dax_driver_type { + DAXDRV_KMEM_TYPE, + DAXDRV_DEVICE_TYPE, +}; + struct dax_device_driver { struct device_driver drv; struct list_head ids; - int match_always; + enum dax_driver_type type; int (*probe)(struct dev_dax *dev); void (*remove)(struct dev_dax *dev); }; diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 5494d745ced5..ecdff79e31f2 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -475,8 +475,7 @@ EXPORT_SYMBOL_GPL(dev_dax_probe); static struct dax_device_driver device_dax_driver = { .probe = dev_dax_probe, - /* all probe actions are unwound by devm, so .remove isn't necessary */ - .match_always = 1, + .type = DAXDRV_DEVICE_TYPE, }; static int __init dax_init(void) diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index e7bdff3132fa..5ec08f9f8a57 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -11,15 +11,25 @@ module_param_named(region_idle, region_idle, bool, 0644); static int dax_hmem_probe(struct platform_device *pdev) { + unsigned long flags = IORESOURCE_DAX_KMEM; struct device *dev = &pdev->dev; struct dax_region *dax_region; struct memregion_info *mri; struct dev_dax_data data; struct dev_dax *dev_dax; + /* + * @region_idle == true indicates that an administrative agent + * wants to manipulate the range partitioning before the devices + * are created, so do not send them to the dax_kmem driver by + * default. + */ + if (region_idle) + flags = 0; + mri = dev->platform_data; dax_region = alloc_dax_region(dev, pdev->id, &mri->range, - mri->target_node, PMD_SIZE, 0); + mri->target_node, PMD_SIZE, flags); if (!dax_region) return -ENOMEM; diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index 4852a2dbdb27..918d01d3fbaa 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -239,6 +239,7 @@ static void dev_dax_kmem_remove(struct dev_dax *dev_dax) static struct dax_device_driver device_dax_kmem_driver = { .probe = dev_dax_kmem_probe, .remove = dev_dax_kmem_remove, + .type = DAXDRV_KMEM_TYPE, }; static int __init dax_kmem_init(void)