From patchwork Wed May 23 05:10:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10420177 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 64A9A6016C for ; Wed, 23 May 2018 05:21:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 535EE2909D for ; Wed, 23 May 2018 05:21:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 480182909F; Wed, 23 May 2018 05:21:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7DB292909D for ; Wed, 23 May 2018 05:20:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F0CE6B000C; Wed, 23 May 2018 01:20:48 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0A1D36B000D; Wed, 23 May 2018 01:20:48 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED1386B000E; Wed, 23 May 2018 01:20:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg0-f69.google.com (mail-pg0-f69.google.com [74.125.83.69]) by kanga.kvack.org (Postfix) with ESMTP id ACE756B000C for ; Wed, 23 May 2018 01:20:47 -0400 (EDT) Received: by mail-pg0-f69.google.com with SMTP id t2-v6so46044pgo.0 for ; Tue, 22 May 2018 22:20:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=qwMEqyYL+vPq8AmdqAq8GH/YJydj0HqRIvb4JVukrOg=; b=T4ySp/rMKkyozB77vnkhSh1R8ELvSijYubUwlEF/BeK3Smv9eSQ2QrLCr+ZRhu8dh6 yQLibtLtAbUN5IwTyFJoTnm6SI0kXyNan2f7A28nz+Rg9QCmvjGATmXVDBHdksSGcXeK urGtcfDfQ1p4V3AuaEc9kvbB9KJUPM/xDBHyShp9JW+vRjIz1/D9iEH28KhOIJ25Xwmw QZ4CGM0yird7T+d59+54TLM/0mMyWPo0J/eBQNqqfLlP7eK10z53u+hJB3RKuLGL5bVR GCDtYy6TnGjKsqctDVviA22wHh6QZqEOdpYly/eBvOQwuKsd6JJ+J0LO3ujCFo2ZC9+P M2Ww== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ALKqPwc1n1Nj8Hc27kL4Omhp5jONaKe/sr54w1DmeiL6KYQMssGGxgud h/ln1VA+Si4TNpm+BQcu2KV38fytoNnHmu5DA8xPnq7ELsafaTiAt/1S3MaG4RnrEi0blPT+7Z3 u2nNYLJT0qyz7Lme7MK77TxBy/iRA79gad3A2FKUwSbw8yuHtlG9cqmdTDQoJRH2DNw== X-Received: by 2002:a17:902:8d8d:: with SMTP id v13-v6mr1432695plo.362.1527052847379; Tue, 22 May 2018 22:20:47 -0700 (PDT) X-Google-Smtp-Source: AB8JxZolC9cUgnFcU9nmdDd1lq1oLGPegQQvlmO1T/Uq+n00Q6kQ42hRGr51iHU5dp3Cg9TuJ6T7 X-Received: by 2002:a17:902:8d8d:: with SMTP id v13-v6mr1432652plo.362.1527052846509; Tue, 22 May 2018 22:20:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527052846; cv=none; d=google.com; s=arc-20160816; b=W4yqWp/t76RAYCMNW4XzTcZc6sooEBanTo8BkCw9s8JhIxS2TN6fMW4i2Y5HQWEPfG yqiPtbSD0JkAv+bKdSrAGNmsI6wiwFVqgho3DMSMbsZhjOHEI4ML6RNqHVoAtR32r2pS QqkJRlYBFh+MmK9MXW6dcgWD8as9bM6TuVDJCMP8X00lAvJZ/IFbWPiYutZIQBCDZigv h2pXSzLklYA4IYQcutuuIAmh4dzvHbgxNRKZr51VxYHQS/SZnBE2EfBQ7kd8ngbSZW77 CDsVHuAHRWB9SDWvnwx5N0DtvWqqXNPdaLFO8t/RKdjcBAwdavgxWum840VZDcA9TJan 7Qbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject :arc-authentication-results; bh=qwMEqyYL+vPq8AmdqAq8GH/YJydj0HqRIvb4JVukrOg=; b=rpfNQP93KR0OUAY/pmBTcW/AR9CDZyFIZ6nA/WH0SHMZKEYaFzeGRDs8MtnBWxwon5 hANCtSo52E8fKAOtBPtAFdNy7XljDzxulkxnYjD0dqKpkZmsFXc/EbUh705CrkS8Y5AZ BmgtfyrziMmZhfpbPljXh6bhRRLbYsjGpqMzP5cDRN62ZomLIyGDb/NY2Og1h8LARS4G jRHOazA4mYb0l19XHlk2dyVbgQwy7CkJ6uCoJhNjQXooSkkpoEBATM6IUy62AfyEHR1S nsaQC0N7qCZ6HjKWp7Ls0bpa0orBOhDH20QhWT2SxoW+4DLBy+cCrmJ+T+SN2mOlJNO7 k6tQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga09.intel.com (mga09.intel.com. [134.134.136.24]) by mx.google.com with ESMTPS id 63-v6si1349059pla.140.2018.05.22.22.20.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 May 2018 22:20:46 -0700 (PDT) Received-SPF: pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) client-ip=134.134.136.24; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dan.j.williams@intel.com designates 134.134.136.24 as permitted sender) smtp.mailfrom=dan.j.williams@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 May 2018 22:20:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,431,1520924400"; d="scan'208";a="44025643" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.16]) by orsmga006.jf.intel.com with ESMTP; 22 May 2018 22:20:46 -0700 Subject: [PATCH v2 6/7] mm, hmm: Replace hmm_devmem_pages_create() with devm_memremap_pages() From: Dan Williams To: akpm@linux-foundation.org Cc: Christoph Hellwig , =?utf-8?b?SsOpcsO0bWU=?= Glisse , Logan Gunthorpe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Date: Tue, 22 May 2018 22:10:49 -0700 Message-ID: <152705224935.21414.2023354160412207966.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <152705221686.21414.771870778478134768.stgit@dwillia2-desk3.amr.corp.intel.com> References: <152705221686.21414.771870778478134768.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.18-2-gc94f MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Commit e8d513483300 "memremap: change devm_memremap_pages interface to use struct dev_pagemap" refactored devm_memremap_pages() to allow a dev_pagemap instance to be supplied. Passing in a dev_pagemap interface simplifies the design of pgmap type drivers in that they can rely on container_of() to lookup any private data associated with the given dev_pagemap instance. In addition to the cleanups this also gives hmm users multi-order-radix improvements that arrived with commit ab1b597ee0e4 "mm, devm_memremap_pages: use multi-order radix for ZONE_DEVICE lookups" As part of the conversion to the devm_memremap_pages() method of handling the percpu_ref relative to when pages are put, the percpu_ref completion needs to move to hmm_devmem_ref_exit(). See commit 71389703839e ("mm, zone_device: Replace {get, put}_zone_device_page...") for details. Reviewed-by: Christoph Hellwig Cc: "Jérôme Glisse" Cc: Logan Gunthorpe Signed-off-by: Dan Williams --- mm/hmm.c | 198 ++++++++------------------------------------------------------ 1 file changed, 26 insertions(+), 172 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index d081857f29b5..5723331f6910 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -927,7 +927,6 @@ struct page *hmm_vma_alloc_locked_page(struct vm_area_struct *vma, } EXPORT_SYMBOL(hmm_vma_alloc_locked_page); - static void hmm_devmem_ref_release(struct percpu_ref *ref) { struct hmm_devmem *devmem; @@ -942,17 +941,16 @@ static void hmm_devmem_ref_exit(void *data) struct hmm_devmem *devmem; devmem = container_of(ref, struct hmm_devmem, ref); + wait_for_completion(&devmem->completion); percpu_ref_exit(ref); } -static void hmm_devmem_ref_kill(void *data) +static void hmm_devmem_ref_kill(struct percpu_ref *ref) { - struct percpu_ref *ref = data; struct hmm_devmem *devmem; devmem = container_of(ref, struct hmm_devmem, ref); percpu_ref_kill(ref); - wait_for_completion(&devmem->completion); } static int hmm_devmem_fault(struct vm_area_struct *vma, @@ -973,155 +971,6 @@ static void hmm_devmem_free(struct page *page, void *data) devmem->ops->free(devmem, page); } -static DEFINE_MUTEX(hmm_devmem_lock); -static RADIX_TREE(hmm_devmem_radix, GFP_KERNEL); - -static void hmm_devmem_radix_release(struct resource *resource) -{ - resource_size_t key, align_start, align_size; - - align_start = resource->start & ~(PA_SECTION_SIZE - 1); - align_size = ALIGN(resource_size(resource), PA_SECTION_SIZE); - - mutex_lock(&hmm_devmem_lock); - for (key = resource->start; - key <= resource->end; - key += PA_SECTION_SIZE) - radix_tree_delete(&hmm_devmem_radix, key >> PA_SECTION_SHIFT); - mutex_unlock(&hmm_devmem_lock); -} - -static void hmm_devmem_release(void *data) -{ - struct hmm_devmem *devmem = data; - struct resource *resource = devmem->resource; - unsigned long start_pfn, npages; - struct zone *zone; - struct page *page; - - /* pages are dead and unused, undo the arch mapping */ - start_pfn = (resource->start & ~(PA_SECTION_SIZE - 1)) >> PAGE_SHIFT; - npages = ALIGN(resource_size(resource), PA_SECTION_SIZE) >> PAGE_SHIFT; - - page = pfn_to_page(start_pfn); - zone = page_zone(page); - - mem_hotplug_begin(); - if (resource->desc == IORES_DESC_DEVICE_PRIVATE_MEMORY) - __remove_pages(zone, start_pfn, npages, NULL); - else - arch_remove_memory(start_pfn << PAGE_SHIFT, - npages << PAGE_SHIFT, NULL); - mem_hotplug_done(); - - hmm_devmem_radix_release(resource); -} - -static int hmm_devmem_pages_create(struct hmm_devmem *devmem) -{ - resource_size_t key, align_start, align_size, align_end; - struct device *device = devmem->device; - int ret, nid, is_ram; - unsigned long pfn; - - align_start = devmem->resource->start & ~(PA_SECTION_SIZE - 1); - align_size = ALIGN(devmem->resource->start + - resource_size(devmem->resource), - PA_SECTION_SIZE) - align_start; - - is_ram = region_intersects(align_start, align_size, - IORESOURCE_SYSTEM_RAM, - IORES_DESC_NONE); - if (is_ram == REGION_MIXED) { - WARN_ONCE(1, "%s attempted on mixed region %pr\n", - __func__, devmem->resource); - return -ENXIO; - } - if (is_ram == REGION_INTERSECTS) - return -ENXIO; - - if (devmem->resource->desc == IORES_DESC_DEVICE_PUBLIC_MEMORY) - devmem->pagemap.type = MEMORY_DEVICE_PUBLIC; - else - devmem->pagemap.type = MEMORY_DEVICE_PRIVATE; - - devmem->pagemap.res = *devmem->resource; - devmem->pagemap.page_fault = hmm_devmem_fault; - devmem->pagemap.page_free = hmm_devmem_free; - devmem->pagemap.dev = devmem->device; - devmem->pagemap.ref = &devmem->ref; - devmem->pagemap.data = devmem; - - mutex_lock(&hmm_devmem_lock); - align_end = align_start + align_size - 1; - for (key = align_start; key <= align_end; key += PA_SECTION_SIZE) { - struct hmm_devmem *dup; - - dup = radix_tree_lookup(&hmm_devmem_radix, - key >> PA_SECTION_SHIFT); - if (dup) { - dev_err(device, "%s: collides with mapping for %s\n", - __func__, dev_name(dup->device)); - mutex_unlock(&hmm_devmem_lock); - ret = -EBUSY; - goto error; - } - ret = radix_tree_insert(&hmm_devmem_radix, - key >> PA_SECTION_SHIFT, - devmem); - if (ret) { - dev_err(device, "%s: failed: %d\n", __func__, ret); - mutex_unlock(&hmm_devmem_lock); - goto error_radix; - } - } - mutex_unlock(&hmm_devmem_lock); - - nid = dev_to_node(device); - if (nid < 0) - nid = numa_mem_id(); - - mem_hotplug_begin(); - /* - * For device private memory we call add_pages() as we only need to - * allocate and initialize struct page for the device memory. More- - * over the device memory is un-accessible thus we do not want to - * create a linear mapping for the memory like arch_add_memory() - * would do. - * - * For device public memory, which is accesible by the CPU, we do - * want the linear mapping and thus use arch_add_memory(). - */ - if (devmem->pagemap.type == MEMORY_DEVICE_PUBLIC) - ret = arch_add_memory(nid, align_start, align_size, NULL, - false); - else - ret = add_pages(nid, align_start >> PAGE_SHIFT, - align_size >> PAGE_SHIFT, NULL, false); - if (ret) { - mem_hotplug_done(); - goto error_add_memory; - } - move_pfn_range_to_zone(&NODE_DATA(nid)->node_zones[ZONE_DEVICE], - align_start >> PAGE_SHIFT, - align_size >> PAGE_SHIFT, NULL); - mem_hotplug_done(); - - for (pfn = devmem->pfn_first; pfn < devmem->pfn_last; pfn++) { - struct page *page = pfn_to_page(pfn); - - page->pgmap = &devmem->pagemap; - } - return 0; - -error_add_memory: - untrack_pfn(NULL, PHYS_PFN(align_start), align_size); -error_radix: - hmm_devmem_radix_release(devmem->resource); -error: - return ret; -} - /* * hmm_devmem_add() - hotplug ZONE_DEVICE memory for device memory * @@ -1145,6 +994,7 @@ struct hmm_devmem *hmm_devmem_add(const struct hmm_devmem_ops *ops, { struct hmm_devmem *devmem; resource_size_t addr; + void *result; int ret; static_branch_enable(&device_private_key); @@ -1199,14 +1049,18 @@ struct hmm_devmem *hmm_devmem_add(const struct hmm_devmem_ops *ops, devmem->pfn_last = devmem->pfn_first + (resource_size(devmem->resource) >> PAGE_SHIFT); - ret = hmm_devmem_pages_create(devmem); - if (ret) - return ERR_PTR(ret); - - ret = devm_add_action_or_reset(device, hmm_devmem_release, devmem); - if (ret) - return ERR_PTR(ret); + devmem->pagemap.type = MEMORY_DEVICE_PRIVATE; + devmem->pagemap.res = *devmem->resource; + devmem->pagemap.page_fault = hmm_devmem_fault; + devmem->pagemap.page_free = hmm_devmem_free; + devmem->pagemap.altmap_valid = false; + devmem->pagemap.ref = &devmem->ref; + devmem->pagemap.data = devmem; + result = devm_memremap_pages(devmem->device, &devmem->pagemap, + hmm_devmem_ref_kill); + if (IS_ERR(result)) + return result; return devmem; } EXPORT_SYMBOL(hmm_devmem_add); @@ -1216,6 +1070,7 @@ struct hmm_devmem *hmm_devmem_add_resource(const struct hmm_devmem_ops *ops, struct resource *res) { struct hmm_devmem *devmem; + void *result; int ret; if (res->desc != IORES_DESC_DEVICE_PUBLIC_MEMORY) @@ -1248,19 +1103,18 @@ struct hmm_devmem *hmm_devmem_add_resource(const struct hmm_devmem_ops *ops, devmem->pfn_last = devmem->pfn_first + (resource_size(devmem->resource) >> PAGE_SHIFT); - ret = hmm_devmem_pages_create(devmem); - if (ret) - return ERR_PTR(ret); - - ret = devm_add_action_or_reset(device, hmm_devmem_release, devmem); - if (ret) - return ERR_PTR(ret); - - ret = devm_add_action_or_reset(device, hmm_devmem_ref_kill, - &devmem->ref); - if (ret) - return ERR_PTR(ret); + devmem->pagemap.type = MEMORY_DEVICE_PUBLIC; + devmem->pagemap.res = *devmem->resource; + devmem->pagemap.page_fault = hmm_devmem_fault; + devmem->pagemap.page_free = hmm_devmem_free; + devmem->pagemap.altmap_valid = false; + devmem->pagemap.ref = &devmem->ref; + devmem->pagemap.data = devmem; + result = devm_memremap_pages(devmem->device, &devmem->pagemap, + hmm_devmem_ref_kill); + if (IS_ERR(result)) + return result; return devmem; } EXPORT_SYMBOL(hmm_devmem_add_resource);