From patchwork Fri Nov 9 20:39:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barret Rhoden X-Patchwork-Id: 10676563 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41B4414E2 for ; Fri, 9 Nov 2018 20:40:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EB4C2F214 for ; Fri, 9 Nov 2018 20:40:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2241C2F235; Fri, 9 Nov 2018 20:40:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BC8F82F214 for ; Fri, 9 Nov 2018 20:40:29 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 5DB8021CAD9BA; Fri, 9 Nov 2018 12:40:28 -0800 (PST) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=2607:f8b0:4864:20::a49; helo=mail-vk1-xa49.google.com; envelope-from=3gvdlwwqkdn8csiphpphmf.dpnmjovy-owejnnmjtut.bc.psh@flex--brho.bounces.google.com; receiver=linux-nvdimm@lists.01.org Received: from mail-vk1-xa49.google.com (mail-vk1-xa49.google.com [IPv6:2607:f8b0:4864:20::a49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 30FE921A00AE6 for ; Fri, 9 Nov 2018 12:39:32 -0800 (PST) Received: by mail-vk1-xa49.google.com with SMTP id c80so653899vke.22 for ; Fri, 09 Nov 2018 12:39:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=BV7rhakurU1AWkPC5l+AZsWY8HTPaGn55zHW5QZMdW4=; b=uKVD1wcNQGFPfemLDd88boZY+8v5esaYoAvtG6qmN78/9qKOE0AZLh+FPIYQH4/Y8m sBEbFD8xxfbryrXV1/rQGS8/9tq9NZag2fpfHFxwlsm9s1SUOoFVJ9LFQXotFH2/fsZw Ah2nIn89RlGU+SqpzsuvUE0bho2WaCcj1NXwuvXTUHQ0+7U6pEdxTZVaHY33BHVDYYGj Bn00w7q9DBhtj6bbvLjMOfJbyJEWWF98hDAXpThE9ZsEbkk3mHbOoRDBJuHzab1qVdqb Ltm7xWNmgTI7lzYmJ4hWNRbJTzXaoU+IiuGBgzFN/UFbKVChgKeEbb90pGeIfsyLNBE7 AddQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BV7rhakurU1AWkPC5l+AZsWY8HTPaGn55zHW5QZMdW4=; b=Sc2dt8y5WYAFmb3IJ8zroV3dYwZkKegtPnBqbYjHVRhAWP/JABzb6IkINZceC7hLOn 9Vl6uLawCWtGF7aHJMha/JMQfT1aPf6V5tn9F8tYr1eGrChs/EbHjmc+dWUnx0fefufq 58BR7fm+EitgDgfXLAS7tSpt+4+0pEEbMVc6fuLY0R/3+M6S0kkLoOpkG+khO/qHfa9h 8o/9Q9iyy/wwpZ1J3YtM6Q/rcG39MAPNecsDxzXMIr3iDH4qhkdr/2JkU2cqCAKbB+Pa +65PfzEiZHLE3hL0XNsfuYMkEBDPb4xiJrlUdhoqz7ed3AzQi3++ctKA9o+siYOJxRsh 0D7Q== X-Gm-Message-State: AGRZ1gJe22YZ4ZELeBbfIrp6Vs/EibBl11A7J6eu5XGwhoBmTIwzqYLc Jr+5/ZnVmpK3SSinnIsPSEW5pZLI X-Google-Smtp-Source: AJdET5dbQ+P6qIQlP79fD8VtNEBYDxa83xp81N1BJmzclE5Naz+tEJU+aIT3XYxO03rUY8yu4nln49At X-Received: by 2002:a1f:cc2:: with SMTP id 185mr8595140vkm.5.1541795970837; Fri, 09 Nov 2018 12:39:30 -0800 (PST) Date: Fri, 9 Nov 2018 15:39:20 -0500 In-Reply-To: <20181109203921.178363-1-brho@google.com> Message-Id: <20181109203921.178363-2-brho@google.com> Mime-Version: 1.0 References: <20181109203921.178363-1-brho@google.com> X-Mailer: git-send-email 2.19.1.930.g4563a0d9d0-goog Subject: [PATCH 1/2] mm: make dev_pagemap_mapping_shift() externally visible From: Barret Rhoden To: Dan Williams , Dave Jiang , Ross Zwisler , Vishal Verma , Paolo Bonzini , " =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= " , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Naoya Horiguchi X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, yu.c.zhang@intel.com, linux-nvdimm@lists.01.org, x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, "H. Peter Anvin" , yi.z.zhang@intel.com Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP KVM has a use case for determining the size of a dax mapping. The KVM code has easy access to the address and the mm; hence the change in parameters. Signed-off-by: Barret Rhoden Reviewed-by: David Hildenbrand --- include/linux/mm.h | 3 +++ mm/memory-failure.c | 38 +++----------------------------------- mm/util.c | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 40 insertions(+), 35 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 5411de93a363..51215d695753 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -935,6 +935,9 @@ static inline bool is_pci_p2pdma_page(const struct page *page) } #endif /* CONFIG_DEV_PAGEMAP_OPS */ +unsigned long dev_pagemap_mapping_shift(unsigned long address, + struct mm_struct *mm); + static inline void get_page(struct page *page) { page = compound_head(page); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 0cd3de3550f0..c3f2c6a8607e 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -265,40 +265,6 @@ void shake_page(struct page *p, int access) } EXPORT_SYMBOL_GPL(shake_page); -static unsigned long dev_pagemap_mapping_shift(struct page *page, - struct vm_area_struct *vma) -{ - unsigned long address = vma_address(page, vma); - pgd_t *pgd; - p4d_t *p4d; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - - pgd = pgd_offset(vma->vm_mm, address); - if (!pgd_present(*pgd)) - return 0; - p4d = p4d_offset(pgd, address); - if (!p4d_present(*p4d)) - return 0; - pud = pud_offset(p4d, address); - if (!pud_present(*pud)) - return 0; - if (pud_devmap(*pud)) - return PUD_SHIFT; - pmd = pmd_offset(pud, address); - if (!pmd_present(*pmd)) - return 0; - if (pmd_devmap(*pmd)) - return PMD_SHIFT; - pte = pte_offset_map(pmd, address); - if (!pte_present(*pte)) - return 0; - if (pte_devmap(*pte)) - return PAGE_SHIFT; - return 0; -} - /* * Failure handling: if we can't find or can't kill a process there's * not much we can do. We just print a message and ignore otherwise. @@ -329,7 +295,9 @@ static void add_to_kill(struct task_struct *tsk, struct page *p, tk->addr = page_address_in_vma(p, vma); tk->addr_valid = 1; if (is_zone_device_page(p)) - tk->size_shift = dev_pagemap_mapping_shift(p, vma); + tk->size_shift = + dev_pagemap_mapping_shift(vma_address(page, vma), + vma->vm_mm); else tk->size_shift = compound_order(compound_head(p)) + PAGE_SHIFT; diff --git a/mm/util.c b/mm/util.c index 8bf08b5b5760..61bc9bab931d 100644 --- a/mm/util.c +++ b/mm/util.c @@ -780,3 +780,37 @@ int get_cmdline(struct task_struct *task, char *buffer, int buflen) out: return res; } + +unsigned long dev_pagemap_mapping_shift(unsigned long address, + struct mm_struct *mm) +{ + pgd_t *pgd; + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + pte_t *pte; + + pgd = pgd_offset(mm, address); + if (!pgd_present(*pgd)) + return 0; + p4d = p4d_offset(pgd, address); + if (!p4d_present(*p4d)) + return 0; + pud = pud_offset(p4d, address); + if (!pud_present(*pud)) + return 0; + if (pud_devmap(*pud)) + return PUD_SHIFT; + pmd = pmd_offset(pud, address); + if (!pmd_present(*pmd)) + return 0; + if (pmd_devmap(*pmd)) + return PMD_SHIFT; + pte = pte_offset_map(pmd, address); + if (!pte_present(*pte)) + return 0; + if (pte_devmap(*pte)) + return PAGE_SHIFT; + return 0; +} +EXPORT_SYMBOL_GPL(dev_pagemap_mapping_shift); From patchwork Fri Nov 9 20:39:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barret Rhoden X-Patchwork-Id: 10676565 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8CC8175A for ; Fri, 9 Nov 2018 20:40:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B96372F214 for ; Fri, 9 Nov 2018 20:40:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC7502F235; Fri, 9 Nov 2018 20:40:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5C9682F214 for ; Fri, 9 Nov 2018 20:40:31 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 76E9E21CB74B7; Fri, 9 Nov 2018 12:40:28 -0800 (PST) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=2607:f8b0:4864:20::74a; helo=mail-qk1-x74a.google.com; envelope-from=3hpdlwwqkdoeeukrjrrjoh.frpolqxa-qyglppolvwv.de.ruj@flex--brho.bounces.google.com; receiver=linux-nvdimm@lists.01.org Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id F40D321A00AE6 for ; Fri, 9 Nov 2018 12:39:32 -0800 (PST) Received: by mail-qk1-x74a.google.com with SMTP id 67so5817296qkj.18 for ; Fri, 09 Nov 2018 12:39:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fRNmz9gb6VrAwP0ad3suXpCd45/0Nzbx55I8Ob0Nxw8=; b=lHERq8Q4HYqaAxGtbFCNPXNjW/o1OBADouxj8g1FuxzZEueE/zn25nbHwssVZKcRXG 6in38OCNmSPHkwrlLMmZ7KPYCQzv5PF5b/5fVyQyPWOm1AmcfC9vyl9B84T8qrWcRyGx rWcomYRaHpJbbDn5GcKH54UwEMPYm2jTOIOoY5MkflAJv+Y8A3WuHbpPqigTK53wK+9X qWApRbCyNX+1F9am8vCrTSM22C4LklypAScWEKX0NkqbT8kxmO/gFh0Z91D1HO83XQgd gKdWYYRDdTugkpaPxi9VubaNIfdDbdiq2uleU3bzzB39RwmZ69nr8VsSMos+QHwPHTYp F6Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fRNmz9gb6VrAwP0ad3suXpCd45/0Nzbx55I8Ob0Nxw8=; b=LVnjZ+XFupS3fhdM1vleCdijGnKiZiNP4MatovbrCTUO9npAIYge9mQTuJGei12Q3i p65qLgMZu4TULf1H32oxpUsc5OGc9OAPrnGVi0qy5+A5j6gaTXDzgWpO1KpMOBcKbdQk Fyvbc7/nW9gpqppLLsMwA9ePvGHzEUu9HW15z7IXNgj+J6nAGapKcSoDHomCfv+8kWTN Sc9N9ZLS0KXv+Y4H7nOFL3lLbIqFPYdxhgaQGqUIekQxSBm6gwTggyZmjde/i5Zeq46o kSRUwLATkDI8oU3I3o2VUU+uUgp1+/vAIZDTCEBiQPPTBj3btAOM7RRTN4jR9qLvDcYC iOFQ== X-Gm-Message-State: AGRZ1gKVj34zG+Q5KDWZdVDJYOPTlp0Cm8l/1BgpP/eoTAHmud0cI/Zi eJt7n/XlavicdkTDCXufE9IGg3S+ X-Google-Smtp-Source: AJdET5c72sXqRFS23DYfcp7KtuzEUK5qHEqYsH1dZDbfT6cQJniU25zASApUpTizlH/1a4frDMn1gl7p X-Received: by 2002:ac8:fe6:: with SMTP id f35mr4922088qtk.5.1541795972066; Fri, 09 Nov 2018 12:39:32 -0800 (PST) Date: Fri, 9 Nov 2018 15:39:21 -0500 In-Reply-To: <20181109203921.178363-1-brho@google.com> Message-Id: <20181109203921.178363-3-brho@google.com> Mime-Version: 1.0 References: <20181109203921.178363-1-brho@google.com> X-Mailer: git-send-email 2.19.1.930.g4563a0d9d0-goog Subject: [PATCH 2/2] kvm: Use huge pages for DAX-backed files From: Barret Rhoden To: Dan Williams , Dave Jiang , Ross Zwisler , Vishal Verma , Paolo Bonzini , " =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= " , Thomas Gleixner , Ingo Molnar , Borislav Petkov X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, yu.c.zhang@intel.com, linux-nvdimm@lists.01.org, x86@kernel.org, linux-kernel@vger.kernel.org, "H. Peter Anvin" , yi.z.zhang@intel.com Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP This change allows KVM to map DAX-backed files made of huge pages with huge mappings in the EPT/TDP. DAX pages are not PageTransCompound. The existing check is trying to determine if the mapping for the pfn is a huge mapping or not. For non-DAX maps, e.g. hugetlbfs, that means checking PageTransCompound. For DAX, we can check the page table itself. Note that KVM already faulted in the page (or huge page) in the host's page table, and we hold the KVM mmu spinlock (grabbed before checking the mmu seq). Signed-off-by: Barret Rhoden --- arch/x86/kvm/mmu.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index cf5f572f2305..2df8c459dc6a 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -3152,6 +3152,36 @@ static int kvm_handle_bad_page(struct kvm_vcpu *vcpu, gfn_t gfn, kvm_pfn_t pfn) return -EFAULT; } +static bool pfn_is_huge_mapped(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) +{ + struct page *page = pfn_to_page(pfn); + unsigned long hva, map_shift; + + if (!is_zone_device_page(page)) + return PageTransCompoundMap(page); + + /* + * DAX pages do not use compound pages. The page should have already + * been mapped into the host-side page table during try_async_pf(), so + * we can check the page tables directly. + */ + hva = gfn_to_hva(kvm, gfn); + if (kvm_is_error_hva(hva)) + return false; + + /* + * Our caller grabbed the KVM mmu_lock with a successful + * mmu_notifier_retry, so we're safe to walk the page table. + */ + map_shift = dev_pagemap_mapping_shift(hva, current->mm); + switch (map_shift) { + case PMD_SHIFT: + case PUD_SIZE: + return true; + } + return false; +} + static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu, gfn_t *gfnp, kvm_pfn_t *pfnp, int *levelp) @@ -3168,7 +3198,7 @@ static void transparent_hugepage_adjust(struct kvm_vcpu *vcpu, */ if (!is_error_noslot_pfn(pfn) && !kvm_is_reserved_pfn(pfn) && level == PT_PAGE_TABLE_LEVEL && - PageTransCompoundMap(pfn_to_page(pfn)) && + pfn_is_huge_mapped(vcpu->kvm, gfn, pfn) && !mmu_gfn_lpage_is_disallowed(vcpu, gfn, PT_DIRECTORY_LEVEL)) { unsigned long mask; /* @@ -5678,7 +5708,7 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm, */ if (sp->role.direct && !kvm_is_reserved_pfn(pfn) && - PageTransCompoundMap(pfn_to_page(pfn))) { + pfn_is_huge_mapped(kvm, sp->gfn, pfn)) { pte_list_remove(rmap_head, sptep); need_tlb_flush = 1; goto restart;