From patchwork Fri Jun 26 17:31:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 11628119 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 80DA914E3 for ; Fri, 26 Jun 2020 17:34:20 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6A303206DD for ; Fri, 26 Jun 2020 17:34:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A303206DD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id DB3B811022297; Fri, 26 Jun 2020 10:34:19 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=79.96.170.134; helo=cloudserver094114.home.pl; envelope-from=rjw@rjwysocki.net; receiver= Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 36B4B11001AC1 for ; Fri, 26 Jun 2020 10:34:17 -0700 (PDT) Received: from 89-64-83-223.dynamic.chello.pl (89.64.83.223) (HELO kreacher.localnet) by serwer1319399.home.pl (79.96.170.134) with SMTP (IdeaSmtpServer 0.83.415) id 8fbdec14a0a21657; Fri, 26 Jun 2020 19:34:15 +0200 From: "Rafael J. Wysocki" To: Dan Williams , Erik Kaneda Subject: [RFT][PATCH v3 1/4] ACPICA: Take deferred unmapping of memory into account Date: Fri, 26 Jun 2020 19:31:16 +0200 Message-ID: <2545526.8MO9PuLXVt@kreacher> In-Reply-To: <2788992.3K7huLjdjL@kreacher> References: <158889473309.2292982.18007035454673387731.stgit@dwillia2-desk3.amr.corp.intel.com> <2713141.s8EVnczdoM@kreacher> <2788992.3K7huLjdjL@kreacher> MIME-Version: 1.0 Message-ID-Hash: 2N2DH5LOVBPFKAH2MDL2KKZHASYUFH5N X-Message-ID-Hash: 2N2DH5LOVBPFKAH2MDL2KKZHASYUFH5N X-MailFrom: rjw@rjwysocki.net X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: rafael.j.wysocki@intel.com, Len Brown , Borislav Petkov , James Morse , Myron Stowe , Andy Shevchenko , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org, Bob Moore X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: "Rafael J. Wysocki" The ACPI OS layer in Linux uses RCU to protect the walkers of the list of ACPI memory mappings from seeing an inconsistent state while it is being updated. Among other situations, that list can be walked in (NMI and non-NMI) interrupt context, so using a sleeping lock to protect it is not an option. However, performance issues related to the RCU usage in there appear, as described by Dan Williams: "Recently a performance problem was reported for a process invoking a non-trival ASL program. The method call in this case ends up repetitively triggering a call path like: acpi_ex_store acpi_ex_store_object_to_node acpi_ex_write_data_to_field acpi_ex_insert_into_field acpi_ex_write_with_update_rule acpi_ex_field_datum_io acpi_ex_access_region acpi_ev_address_space_dispatch acpi_ex_system_memory_space_handler acpi_os_map_cleanup.part.14 _synchronize_rcu_expedited.constprop.89 schedule The end result of frequent synchronize_rcu_expedited() invocation is tiny sub-millisecond spurts of execution where the scheduler freely migrates this apparently sleepy task. The overhead of frequent scheduler invocation multiplies the execution time by a factor of 2-3X." The source of this is that acpi_ex_system_memory_space_handler() unmaps the memory mapping currently cached by it at the access time if that mapping doesn't cover the memory area being accessed. Consequently, if there is a memory opregion with two fields separated from each other by an unused chunk of address space that is large enough for not being covered by a single mapping, and they happen to be used in an alternating pattern, the unmapping will occur on every acpi_ex_system_memory_space_handler() invocation for that memory opregion and that will lead to significant overhead. To address that, acpi_os_unmap_memory() provided by Linux can be modified so as to avoid unmapping the memory region matching the address range at hand right away and queue it up for later removal. However, that requires the deferred unmapping of unused memory regions to be carried out at least occasionally, so modify ACPICA to do that by invoking a new OS layer function, acpi_os_release_unused_mappings(), for this purpose every time the AML interpreter is exited. For completeness, also call that function from acpi_db_test_all_objects() after all of the fields have been tested. Reported-by: Dan Williams Signed-off-by: Rafael J. Wysocki --- drivers/acpi/acpica/dbtest.c | 4 ++++ drivers/acpi/acpica/exutils.c | 2 ++ include/acpi/acpiosxf.h | 4 ++++ 3 files changed, 10 insertions(+) diff --git a/drivers/acpi/acpica/dbtest.c b/drivers/acpi/acpica/dbtest.c index 6db44a5ac786..55931daa1779 100644 --- a/drivers/acpi/acpica/dbtest.c +++ b/drivers/acpi/acpica/dbtest.c @@ -220,6 +220,10 @@ static void acpi_db_test_all_objects(void) (void)acpi_walk_namespace(ACPI_TYPE_ANY, ACPI_ROOT_OBJECT, ACPI_UINT32_MAX, acpi_db_test_one_object, NULL, NULL, NULL); + + /* Release memory mappings that are not needed any more. */ + + acpi_os_release_unused_mappings(); } /******************************************************************************* diff --git a/drivers/acpi/acpica/exutils.c b/drivers/acpi/acpica/exutils.c index 8fefa6feac2f..ae2030095b63 100644 --- a/drivers/acpi/acpica/exutils.c +++ b/drivers/acpi/acpica/exutils.c @@ -106,6 +106,8 @@ void acpi_ex_exit_interpreter(void) "Could not release AML Interpreter mutex")); } + acpi_os_release_unused_mappings(); + return_VOID; } diff --git a/include/acpi/acpiosxf.h b/include/acpi/acpiosxf.h index 33bb8c9a089d..0efe2d1725e2 100644 --- a/include/acpi/acpiosxf.h +++ b/include/acpi/acpiosxf.h @@ -187,6 +187,10 @@ void *acpi_os_map_memory(acpi_physical_address where, acpi_size length); void acpi_os_unmap_memory(void *logical_address, acpi_size size); #endif +#ifndef ACPI_USE_ALTERNATE_PROTOTYPE_acpi_os_release_unused_mappings +#define acpi_os_release_unused_mappings() do { } while (FALSE) +#endif + #ifndef ACPI_USE_ALTERNATE_PROTOTYPE_acpi_os_get_physical_address acpi_status acpi_os_get_physical_address(void *logical_address, From patchwork Fri Jun 26 17:31:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 11628115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62A396C1 for ; Fri, 26 Jun 2020 17:34:18 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C3B120707 for ; Fri, 26 Jun 2020 17:34:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C3B120707 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B5609110122B6; Fri, 26 Jun 2020 10:34:17 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=79.96.170.134; helo=cloudserver094114.home.pl; envelope-from=rjw@rjwysocki.net; receiver= Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 61FA510FE3541 for ; Fri, 26 Jun 2020 10:34:15 -0700 (PDT) Received: from 89-64-83-223.dynamic.chello.pl (89.64.83.223) (HELO kreacher.localnet) by serwer1319399.home.pl (79.96.170.134) with SMTP (IdeaSmtpServer 0.83.415) id 85d3147370415763; Fri, 26 Jun 2020 19:34:13 +0200 From: "Rafael J. Wysocki" To: Dan Williams , Erik Kaneda Subject: [RFT][PATCH v3 2/4] ACPI: OSL: Implement deferred unmapping of ACPI memory Date: Fri, 26 Jun 2020 19:31:24 +0200 Message-ID: <4507864.RZ40Oy75i1@kreacher> In-Reply-To: <2788992.3K7huLjdjL@kreacher> References: <158889473309.2292982.18007035454673387731.stgit@dwillia2-desk3.amr.corp.intel.com> <2713141.s8EVnczdoM@kreacher> <2788992.3K7huLjdjL@kreacher> MIME-Version: 1.0 Message-ID-Hash: RKIONUBWARFHTU25TEUCMMYWWTYK5SJZ X-Message-ID-Hash: RKIONUBWARFHTU25TEUCMMYWWTYK5SJZ X-MailFrom: rjw@rjwysocki.net X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: rafael.j.wysocki@intel.com, Len Brown , Borislav Petkov , James Morse , Myron Stowe , Andy Shevchenko , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org, Bob Moore X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: "Rafael J. Wysocki" Rework acpi_os_unmap_memory() so that it does not release the memory mapping matching the given address range right away but queues it up for later removal, implement acpi_os_release_unused_mappings() that will remove the unused ACPI memory mappings and add invocations of it to acpi_release_memory() and to the table loading/unloading code, to get rid of memory mappings that may be left behind. Reported-by: Dan Williams Signed-off-by: Rafael J. Wysocki --- drivers/acpi/acpi_configfs.c | 3 + drivers/acpi/osl.c | 153 +++++++++++++++++++++++------- drivers/acpi/tables.c | 2 + include/acpi/platform/aclinux.h | 1 + include/acpi/platform/aclinuxex.h | 2 + 5 files changed, 125 insertions(+), 36 deletions(-) diff --git a/drivers/acpi/acpi_configfs.c b/drivers/acpi/acpi_configfs.c index ece8c1a921cc..dd167ff87dc4 100644 --- a/drivers/acpi/acpi_configfs.c +++ b/drivers/acpi/acpi_configfs.c @@ -59,6 +59,8 @@ static ssize_t acpi_table_aml_write(struct config_item *cfg, table->header = NULL; } + acpi_os_release_unused_mappings(); + return ret; } @@ -224,6 +226,7 @@ static void acpi_table_drop_item(struct config_group *group, ACPI_INFO(("Host-directed Dynamic ACPI Table Unload")); acpi_unload_table(table->index); + acpi_os_release_unused_mappings(); } static struct configfs_group_operations acpi_table_group_ops = { diff --git a/drivers/acpi/osl.c b/drivers/acpi/osl.c index 762c5d50b8fe..749ae3e32193 100644 --- a/drivers/acpi/osl.c +++ b/drivers/acpi/osl.c @@ -77,12 +77,16 @@ struct acpi_ioremap { void __iomem *virt; acpi_physical_address phys; acpi_size size; - unsigned long refcount; + union { + unsigned long refcount; + struct list_head gc; + } track; }; static LIST_HEAD(acpi_ioremaps); static DEFINE_MUTEX(acpi_ioremap_lock); #define acpi_ioremap_lock_held() lock_is_held(&acpi_ioremap_lock.dep_map) +static LIST_HEAD(unused_mappings); static void __init acpi_request_region (struct acpi_generic_address *gas, unsigned int length, char *desc) @@ -250,7 +254,7 @@ void __iomem *acpi_os_get_iomem(acpi_physical_address phys, unsigned int size) map = acpi_map_lookup(phys, size); if (map) { virt = map->virt + (phys - map->phys); - map->refcount++; + map->track.refcount++; } mutex_unlock(&acpi_ioremap_lock); return virt; @@ -335,7 +339,7 @@ void __iomem __ref /* Check if there's a suitable mapping already. */ map = acpi_map_lookup(phys, size); if (map) { - map->refcount++; + map->track.refcount++; goto out; } @@ -358,7 +362,7 @@ void __iomem __ref map->virt = virt; map->phys = pg_off; map->size = pg_sz; - map->refcount = 1; + map->track.refcount = 1; list_add_tail_rcu(&map->list, &acpi_ioremaps); @@ -375,40 +379,39 @@ void *__ref acpi_os_map_memory(acpi_physical_address phys, acpi_size size) EXPORT_SYMBOL_GPL(acpi_os_map_memory); /* Must be called with mutex_lock(&acpi_ioremap_lock) */ -static unsigned long acpi_os_drop_map_ref(struct acpi_ioremap *map) +static bool acpi_os_drop_map_ref(struct acpi_ioremap *map, bool defer) { - unsigned long refcount = --map->refcount; + if (--map->track.refcount) + return true; - if (!refcount) - list_del_rcu(&map->list); - return refcount; + list_del_rcu(&map->list); + + if (defer) { + INIT_LIST_HEAD(&map->track.gc); + list_add_tail(&map->track.gc, &unused_mappings); + } + return defer; } -static void acpi_os_map_cleanup(struct acpi_ioremap *map) +static void __acpi_os_map_cleanup(struct acpi_ioremap *map) { - synchronize_rcu_expedited(); acpi_unmap(map->phys, map->virt); kfree(map); } -/** - * acpi_os_unmap_iomem - Drop a memory mapping reference. - * @virt: Start of the address range to drop a reference to. - * @size: Size of the address range to drop a reference to. - * - * Look up the given virtual address range in the list of existing ACPI memory - * mappings, drop a reference to it and unmap it if there are no more active - * references to it. - * - * During early init (when acpi_permanent_mmap has not been set yet) this - * routine simply calls __acpi_unmap_table() to get the job done. Since - * __acpi_unmap_table() is an __init function, the __ref annotation is needed - * here. - */ -void __ref acpi_os_unmap_iomem(void __iomem *virt, acpi_size size) +static void acpi_os_map_cleanup(struct acpi_ioremap *map) +{ + if (!map) + return; + + synchronize_rcu_expedited(); + __acpi_os_map_cleanup(map); +} + +static void __ref __acpi_os_unmap_iomem(void __iomem *virt, acpi_size size, + bool defer) { struct acpi_ioremap *map; - unsigned long refcount; if (!acpi_permanent_mmap) { __acpi_unmap_table(virt, size); @@ -416,26 +419,97 @@ void __ref acpi_os_unmap_iomem(void __iomem *virt, acpi_size size) } mutex_lock(&acpi_ioremap_lock); + map = acpi_map_lookup_virt(virt, size); if (!map) { mutex_unlock(&acpi_ioremap_lock); WARN(true, PREFIX "%s: bad address %p\n", __func__, virt); return; } - refcount = acpi_os_drop_map_ref(map); + if (acpi_os_drop_map_ref(map, defer)) + map = NULL; + mutex_unlock(&acpi_ioremap_lock); - if (!refcount) - acpi_os_map_cleanup(map); + acpi_os_map_cleanup(map); +} + +/** + * acpi_os_unmap_iomem - Drop a memory mapping reference. + * @virt: Start of the address range to drop a reference to. + * @size: Size of the address range to drop a reference to. + * + * Look up the given virtual address range in the list of existing ACPI memory + * mappings, drop a reference to it and unmap it if there are no more active + * references to it. + * + * During early init (when acpi_permanent_mmap has not been set yet) this + * routine simply calls __acpi_unmap_table() to get the job done. Since + * __acpi_unmap_table() is an __init function, the __ref annotation is needed + * here. + */ +void __ref acpi_os_unmap_iomem(void __iomem *virt, acpi_size size) +{ + __acpi_os_unmap_iomem(virt, size, false); } EXPORT_SYMBOL_GPL(acpi_os_unmap_iomem); +/** + * acpi_os_unmap_memory - Drop a memory mapping reference. + * @virt: Start of the address range to drop a reference to. + * @size: Size of the address range to drop a reference to. + * + * Look up the given virtual address range in the list of existing ACPI memory + * mappings, drop a reference to it and if there are no more active references + * to it, put it in the list of unused memory mappings. + * + * During early init (when acpi_permanent_mmap has not been set yet) this + * routine behaves like acpi_os_unmap_iomem(). + */ void __ref acpi_os_unmap_memory(void *virt, acpi_size size) { - return acpi_os_unmap_iomem((void __iomem *)virt, size); + __acpi_os_unmap_iomem((void __iomem *)virt, size, true); } EXPORT_SYMBOL_GPL(acpi_os_unmap_memory); +/** + * acpi_os_release_unused_mappings - Release unused ACPI memory mappings. + */ +void acpi_os_release_unused_mappings(void) +{ + struct list_head list; + + INIT_LIST_HEAD(&list); + + /* + * First avoid looking at mappings that may be added to the "unused" + * list while the synchronize_rcu() below is running. + */ + mutex_lock(&acpi_ioremap_lock); + + list_splice_init(&unused_mappings, &list); + + mutex_unlock(&acpi_ioremap_lock); + + if (list_empty(&list)) + return; + + /* + * Wait for the possible users of the mappings in the "unused" list to + * stop using them. + */ + synchronize_rcu(); + + /* Release the unused mappings in the list. */ + do { + struct acpi_ioremap *map; + + map = list_entry(list.next, struct acpi_ioremap, track.gc); + list_del(&map->track.gc); + __acpi_os_map_cleanup(map); + } while (!list_empty(&list)); +} + int acpi_os_map_generic_address(struct acpi_generic_address *gas) { u64 addr; @@ -461,7 +535,6 @@ void acpi_os_unmap_generic_address(struct acpi_generic_address *gas) { u64 addr; struct acpi_ioremap *map; - unsigned long refcount; if (gas->space_id != ACPI_ADR_SPACE_SYSTEM_MEMORY) return; @@ -472,16 +545,18 @@ void acpi_os_unmap_generic_address(struct acpi_generic_address *gas) return; mutex_lock(&acpi_ioremap_lock); + map = acpi_map_lookup(addr, gas->bit_width / 8); if (!map) { mutex_unlock(&acpi_ioremap_lock); return; } - refcount = acpi_os_drop_map_ref(map); + if (acpi_os_drop_map_ref(map, false)) + map = NULL; + mutex_unlock(&acpi_ioremap_lock); - if (!refcount) - acpi_os_map_cleanup(map); + acpi_os_map_cleanup(map); } EXPORT_SYMBOL(acpi_os_unmap_generic_address); @@ -1566,11 +1641,17 @@ static acpi_status acpi_deactivate_mem_region(acpi_handle handle, u32 level, acpi_status acpi_release_memory(acpi_handle handle, struct resource *res, u32 level) { + acpi_status ret; + if (!(res->flags & IORESOURCE_MEM)) return AE_TYPE; - return acpi_walk_namespace(ACPI_TYPE_REGION, handle, level, + ret = acpi_walk_namespace(ACPI_TYPE_REGION, handle, level, acpi_deactivate_mem_region, NULL, res, NULL); + + acpi_os_release_unused_mappings(); + + return ret; } EXPORT_SYMBOL_GPL(acpi_release_memory); diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c index 0e905c3d1645..939484a860a1 100644 --- a/drivers/acpi/tables.c +++ b/drivers/acpi/tables.c @@ -816,6 +816,8 @@ int __init acpi_table_init(void) return -EINVAL; acpi_table_initrd_scan(); + acpi_os_release_unused_mappings(); + check_multiple_madt(); return 0; } diff --git a/include/acpi/platform/aclinux.h b/include/acpi/platform/aclinux.h index 987e2af7c335..784e294dc74c 100644 --- a/include/acpi/platform/aclinux.h +++ b/include/acpi/platform/aclinux.h @@ -133,6 +133,7 @@ #define ACPI_USE_ALTERNATE_PROTOTYPE_acpi_os_delete_raw_lock #define ACPI_USE_ALTERNATE_PROTOTYPE_acpi_os_acquire_raw_lock #define ACPI_USE_ALTERNATE_PROTOTYPE_acpi_os_release_raw_lock +#define ACPI_USE_ALTERNATE_PROTOTYPE_acpi_os_release_unused_mappings /* * OSL interfaces used by debugger/disassembler diff --git a/include/acpi/platform/aclinuxex.h b/include/acpi/platform/aclinuxex.h index 04f88f2de781..ad6b905358c5 100644 --- a/include/acpi/platform/aclinuxex.h +++ b/include/acpi/platform/aclinuxex.h @@ -120,6 +120,8 @@ static inline void acpi_os_delete_raw_lock(acpi_raw_spinlock handle) ACPI_FREE(handle); } +void acpi_os_release_unused_mappings(void); + static inline u8 acpi_os_readable(void *pointer, acpi_size length) { return TRUE; From patchwork Fri Jun 26 17:32:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Rafael J. Wysocki" X-Patchwork-Id: 11628113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E402614E3 for ; Fri, 26 Jun 2020 17:34:16 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CD69620836 for ; Fri, 26 Jun 2020 17:34:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD69620836 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=rjwysocki.net Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 9D3F8110122B4; Fri, 26 Jun 2020 10:34:16 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=79.96.170.134; helo=cloudserver094114.home.pl; envelope-from=rjw@rjwysocki.net; receiver= Received: from cloudserver094114.home.pl (cloudserver094114.home.pl [79.96.170.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id ECF8D11001ABB for ; Fri, 26 Jun 2020 10:34:13 -0700 (PDT) Received: from 89-64-83-223.dynamic.chello.pl (89.64.83.223) (HELO kreacher.localnet) by serwer1319399.home.pl (79.96.170.134) with SMTP (IdeaSmtpServer 0.83.415) id 33d6153b1b8ab52e; Fri, 26 Jun 2020 19:34:12 +0200 From: "Rafael J. Wysocki" To: Dan Williams , Erik Kaneda Subject: [RFT][PATCH v3 3/4] ACPICA: Preserve memory opregion mappings if supported by OS Date: Fri, 26 Jun 2020 19:32:27 +0200 Message-ID: <3156511.gHkMriIWEz@kreacher> In-Reply-To: <2788992.3K7huLjdjL@kreacher> References: <158889473309.2292982.18007035454673387731.stgit@dwillia2-desk3.amr.corp.intel.com> <2713141.s8EVnczdoM@kreacher> <2788992.3K7huLjdjL@kreacher> MIME-Version: 1.0 Message-ID-Hash: D3PYPHG4RRLJ25DBMDJJMX7YXEF4PI22 X-Message-ID-Hash: D3PYPHG4RRLJ25DBMDJJMX7YXEF4PI22 X-MailFrom: rjw@rjwysocki.net X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: rafael.j.wysocki@intel.com, Len Brown , Borislav Petkov , James Morse , Myron Stowe , Andy Shevchenko , linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org, Bob Moore X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: From: "Rafael J. Wysocki" The ACPICA's strategy with respect to the handling of memory mappings associated with memory operation regions is to avoid mapping the entire region at once which may be problematic at least in principle (for example, it may lead to conflicts with overlapping mappings having different attributes created by drivers). It may also be wasteful, because memory opregions on some systems take up vast chunks of address space while the fields in those regions actually accessed by AML are sparsely distributed. For this reason, a one-page "window" is mapped for a given opregion on the first memory access through it and if that "window" does not cover an address range accessed through that opregion subsequently, it is unmapped and a new "window" is mapped to replace it. Next, if the new "window" is not sufficient to acess memory through the opregion in question in the future, it will be replaced with yet another "window" and so on. That may lead to a suboptimal sequence of memory mapping and unmapping operations, for example if two fields in one opregion separated from each other by a sufficiently wide chunk of unused address space are accessed in an alternating pattern. The situation may still be suboptimal if the deferred unmapping introduced previously is supported by the OS layer. For instance, the alternating memory access pattern mentioned above may produce a relatively long list of mappings to release with substantial duplication among the entries in it, which could be avoided if acpi_ex_system_memory_space_handler() did not release the mapping used by it previously as soon as the current access was not covered by it. In order to improve that, modify acpi_ex_system_memory_space_handler() to take advantage of the memory mappings reference counting at the OS level if a suitable interface is provided. Namely, if ACPI_USE_FAST_PATH_MAPPING is set, the OS is expected to implement acpi_os_map_memory_fast_path() that will return NULL if there is no mapping covering the given address range known to it. If such a mapping is there, however, its reference counter will be incremented and a pointer representing the requested virtual address will be returned right away without any additional consequences. That allows acpi_ex_system_memory_space_handler() to acquire additional references to all new memory mappings with the help of acpi_os_map_memory_fast_path() so as to retain them until the memory opregions associated with them go away. The function will still use a new "window" mapping if the current one does not cover the address range at hand, but it will avoid unmapping the current one right away by adding it to a list of "known" mappings associated with the given memory opregion which will be deleted at the opregion deactivation time. The mappings in that list can be used every time a "new window" is needed so as to avoid overhead related to the mapping and unmapping of memory. Signed-off-by: Rafael J. Wysocki --- drivers/acpi/acpica/acinterp.h | 3 + drivers/acpi/acpica/evrgnini.c | 9 +- drivers/acpi/acpica/exregion.c | 154 ++++++++++++++++++++++++++++++++- 3 files changed, 156 insertions(+), 10 deletions(-) diff --git a/drivers/acpi/acpica/acinterp.h b/drivers/acpi/acpica/acinterp.h index a6d896cda2a5..95675a7a8a6b 100644 --- a/drivers/acpi/acpica/acinterp.h +++ b/drivers/acpi/acpica/acinterp.h @@ -479,6 +479,9 @@ void acpi_ex_pci_cls_to_string(char *dest, u8 class_code[3]); u8 acpi_is_valid_space_id(u8 space_id); +acpi_size acpi_ex_mem_space_context_size(void); +void acpi_ex_unmap_all_region_mappings(struct acpi_mem_space_context *mem_info); + /* * exregion - default op_region handlers */ diff --git a/drivers/acpi/acpica/evrgnini.c b/drivers/acpi/acpica/evrgnini.c index aefc0145e583..82f466a128d5 100644 --- a/drivers/acpi/acpica/evrgnini.c +++ b/drivers/acpi/acpica/evrgnini.c @@ -46,13 +46,10 @@ acpi_ev_system_memory_region_setup(acpi_handle handle, local_region_context = (struct acpi_mem_space_context *)*region_context; - /* Delete a cached mapping if present */ + /* Delete memory mappings if present */ if (local_region_context->mapped_length) { - acpi_os_unmap_memory(local_region_context-> - mapped_logical_address, - local_region_context-> - mapped_length); + acpi_ex_unmap_all_region_mappings(local_region_context); } ACPI_FREE(local_region_context); *region_context = NULL; @@ -63,7 +60,7 @@ acpi_ev_system_memory_region_setup(acpi_handle handle, /* Create a new context */ local_region_context = - ACPI_ALLOCATE_ZEROED(sizeof(struct acpi_mem_space_context)); + ACPI_ALLOCATE_ZEROED(acpi_ex_mem_space_context_size()); if (!(local_region_context)) { return_ACPI_STATUS(AE_NO_MEMORY); } diff --git a/drivers/acpi/acpica/exregion.c b/drivers/acpi/acpica/exregion.c index d15a66de26c0..4274582619d2 100644 --- a/drivers/acpi/acpica/exregion.c +++ b/drivers/acpi/acpica/exregion.c @@ -14,6 +14,73 @@ #define _COMPONENT ACPI_EXECUTER ACPI_MODULE_NAME("exregion") +struct acpi_mem_mapping { + acpi_physical_address physical_address; + u8 *logical_address; + acpi_size length; + struct acpi_mem_mapping *next_mm; +}; + +struct acpi_mm_context { + struct acpi_mem_space_context mem_info; + struct acpi_mem_mapping *first_mm; +}; + +/******************************************************************************* + * + * FUNCTION: acpi_ex_mem_space_context_size + * + * PARAMETERS: None + * + * RETURN: Size of internal memory operation region representation. + * + ******************************************************************************/ +acpi_size acpi_ex_mem_space_context_size(void) +{ + ACPI_FUNCTION_TRACE(acpi_ex_mem_space_context_size); + +#ifdef ACPI_USE_FAST_PATH_MAPPING + return sizeof(struct acpi_mm_context); +#else + return sizeof(struct acpi_mem_space_context); +#endif +} + +/******************************************************************************* + * + * FUNCTION: acpi_ex_unmap_all_region_mappings + * + * PARAMETERS: mem_info - Region specific context + * + * RETURN: None + * + * DESCRIPTION: Unmap all mappings associated with a memory operation region. + * + ******************************************************************************/ +void acpi_ex_unmap_all_region_mappings(struct acpi_mem_space_context *mem_info) +{ +#ifdef ACPI_USE_FAST_PATH_MAPPING + struct acpi_mm_context *mm_context = (struct acpi_mm_context *)mem_info; + struct acpi_mem_mapping *mm; +#endif + + ACPI_FUNCTION_TRACE(acpi_ex_unmap_all_region_mappings); + + acpi_os_unmap_memory(mem_info->mapped_logical_address, + mem_info->mapped_length); + +#ifdef ACPI_USE_FAST_PATH_MAPPING + while (mm_context->first_mm) { + mm = mm_context->first_mm; + mm_context->first_mm = mm->next_mm; + acpi_os_unmap_memory(mm->logical_address, mm->length); + ACPI_FREE(mm); + } +#endif + + return_VOID; +} + /******************************************************************************* * * FUNCTION: acpi_ex_system_memory_space_handler @@ -44,6 +111,10 @@ acpi_ex_system_memory_space_handler(u32 function, u32 length; acpi_size map_length; acpi_size page_boundary_map_length; +#ifdef ACPI_USE_FAST_PATH_MAPPING + struct acpi_mm_context *mm_context = (struct acpi_mm_context *)mem_info; + struct acpi_mem_mapping *mm; +#endif #ifdef ACPI_MISALIGNMENT_NOT_SUPPORTED u32 remainder; #endif @@ -102,7 +173,7 @@ acpi_ex_system_memory_space_handler(u32 function, mem_info->mapped_length))) { /* * The request cannot be resolved by the current memory mapping; - * Delete the existing mapping and create a new one. + * Delete the current cached mapping and get a new one. */ if (mem_info->mapped_length) { @@ -112,6 +183,36 @@ acpi_ex_system_memory_space_handler(u32 function, mem_info->mapped_length); } +#ifdef ACPI_USE_FAST_PATH_MAPPING + /* + * Look for an existing saved mapping matching the address range + * at hand. If found, make the OS layer bump up the reference + * counter of that mapping, cache it and carry out the access. + */ + for (mm = mm_context->first_mm; mm; mm = mm->next_mm) { + if (address < mm->physical_address) + continue; + + if ((u64)address + length > + (u64)mm->physical_address + mm->length) + continue; + + /* + * When called on a known-existing memory mapping, + * acpi_os_map_memory_fast_path() must return the same + * logical address as before or NULL. + */ + if (!acpi_os_map_memory_fast_path(mm->physical_address, + mm->length)) + continue; + + mem_info->mapped_logical_address = mm->logical_address; + mem_info->mapped_physical_address = mm->physical_address; + mem_info->mapped_length = mm->length; + goto access; + } +#endif /* ACPI_USE_FAST_PATH_MAPPING */ + /* * October 2009: Attempt to map from the requested address to the * end of the region. However, we will never map more than one @@ -143,9 +244,8 @@ acpi_ex_system_memory_space_handler(u32 function, /* Create a new mapping starting at the address given */ - mem_info->mapped_logical_address = - acpi_os_map_memory(address, map_length); - if (!mem_info->mapped_logical_address) { + logical_addr_ptr = acpi_os_map_memory(address, map_length); + if (!logical_addr_ptr) { ACPI_ERROR((AE_INFO, "Could not map memory at 0x%8.8X%8.8X, size %u", ACPI_FORMAT_UINT64(address), @@ -156,10 +256,56 @@ acpi_ex_system_memory_space_handler(u32 function, /* Save the physical address and mapping size */ + mem_info->mapped_logical_address = logical_addr_ptr; mem_info->mapped_physical_address = address; mem_info->mapped_length = map_length; + +#ifdef ACPI_USE_FAST_PATH_MAPPING + /* + * Create a new mm list entry to save the new mapping for + * removal at the operation region deactivation time. + */ + mm = ACPI_ALLOCATE_ZEROED(sizeof(*mm)); + if (!mm) { + /* + * No room to save the new mapping, but this is not + * critical. Just log the error and carry out the + * access as requested. + */ + ACPI_ERROR((AE_INFO, + "Not enough memory to save memory mapping at 0x%8.8X%8.8X, size %u", + ACPI_FORMAT_UINT64(address), + (u32)map_length)); + goto access; + } + /* + * Bump up the new mapping's reference counter in the OS layer + * to prevent it from getting dropped prematurely. + */ + if (!acpi_os_map_memory_fast_path(address, map_length)) { + /* + * Something has gone wrong, but this is not critical. + * Log the error, free the mm list entry that won't be + * used and carry out the access as requested. + */ + ACPI_ERROR((AE_INFO, + "Unable to save memory mapping at 0x%8.8X%8.8X, size %u", + ACPI_FORMAT_UINT64(address), + (u32)map_length)); + ACPI_FREE(mm); + goto access; + } + mm->physical_address = address; + mm->logical_address = logical_addr_ptr; + mm->length = map_length; + mm->next_mm = mm_context->first_mm; + mm_context->first_mm = mm; } +access: +#else /* !ACPI_USE_FAST_PATH_MAPPING */ + } +#endif /* !ACPI_USE_FAST_PATH_MAPPING */ /* * Generate a logical pointer corresponding to the address we want to * access