From patchwork Fri Sep 7 18:03:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 10591919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33F33139B for ; Fri, 7 Sep 2018 09:24:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F52728537 for ; Fri, 7 Sep 2018 09:24:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 23A842AD99; Fri, 7 Sep 2018 09:24:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.9 required=2.0 tests=BAYES_00,DATE_IN_FUTURE_03_06, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A0FA28537 for ; Fri, 7 Sep 2018 09:24:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728324AbeIGOE2 (ORCPT ); Fri, 7 Sep 2018 10:04:28 -0400 Received: from mga09.intel.com ([134.134.136.24]:44890 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728297AbeIGOE2 (ORCPT ); Fri, 7 Sep 2018 10:04:28 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Sep 2018 02:24:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,341,1531810800"; d="scan'208";a="71327138" Received: from linux.intel.com ([10.54.29.200]) by orsmga008.jf.intel.com with ESMTP; 07 Sep 2018 02:24:25 -0700 Received: from dazhang1-ssd.sh.intel.com (unknown [10.239.48.163]) by linux.intel.com (Postfix) with ESMTP id 708385802B3; Fri, 7 Sep 2018 02:24:22 -0700 (PDT) From: Zhang Yi To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, pbonzini@redhat.com, dan.j.williams@intel.com, dave.jiang@intel.com, yu.c.zhang@intel.com, pagupta@redhat.com, david@redhat.com, jack@suse.cz, hch@lst.de Cc: linux-mm@kvack.org, rkrcmar@redhat.com, jglisse@redhat.com, yi.z.zhang@intel.com, Zhang Yi Subject: [PATCH V5 1/4] kvm: remove redundant reserved page check Date: Sat, 8 Sep 2018 02:03:28 +0800 Message-Id: <26f79872e78cc643937059003763b5cfc1333167.1536342881.git.yi.z.zhang@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP PageReserved() is already checked inside kvm_is_reserved_pfn(), remove it from kvm_set_pfn_dirty(). Signed-off-by: Zhang Yi Signed-off-by: Zhang Yu Reviewed-by: David Hildenbrand Acked-by: Pankaj Gupta --- virt/kvm/kvm_main.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 8b47507f..c44c406 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1690,12 +1690,8 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty); void kvm_set_pfn_dirty(kvm_pfn_t pfn) { - if (!kvm_is_reserved_pfn(pfn)) { - struct page *page = pfn_to_page(pfn); - - if (!PageReserved(page)) - SetPageDirty(page); - } + if (!kvm_is_reserved_pfn(pfn)) + SetPageDirty(pfn_to_page(pfn)); } EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty); From patchwork Fri Sep 7 18:03:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 10591925 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0721613BB for ; Fri, 7 Sep 2018 09:24:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0175F28537 for ; Fri, 7 Sep 2018 09:24:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E8C722AD99; Fri, 7 Sep 2018 09:24:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.9 required=2.0 tests=BAYES_00,DATE_IN_FUTURE_03_06, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A019228537 for ; Fri, 7 Sep 2018 09:24:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728345AbeIGOEj (ORCPT ); Fri, 7 Sep 2018 10:04:39 -0400 Received: from mga06.intel.com ([134.134.136.31]:23986 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728297AbeIGOEj (ORCPT ); Fri, 7 Sep 2018 10:04:39 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Sep 2018 02:24:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,341,1531810800"; d="scan'208";a="72355804" Received: from linux.intel.com ([10.54.29.200]) by orsmga006.jf.intel.com with ESMTP; 07 Sep 2018 02:24:35 -0700 Received: from dazhang1-ssd.sh.intel.com (unknown [10.239.48.163]) by linux.intel.com (Postfix) with ESMTP id 06B1D5803DA; Fri, 7 Sep 2018 02:24:32 -0700 (PDT) From: Zhang Yi To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, pbonzini@redhat.com, dan.j.williams@intel.com, dave.jiang@intel.com, yu.c.zhang@intel.com, pagupta@redhat.com, david@redhat.com, jack@suse.cz, hch@lst.de Cc: linux-mm@kvack.org, rkrcmar@redhat.com, jglisse@redhat.com, yi.z.zhang@intel.com, Zhang Yi Subject: [PATCH V5 2/4] mm: introduce memory type MEMORY_DEVICE_DEV_DAX Date: Sat, 8 Sep 2018 02:03:39 +0800 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, NVDIMM pages will be marked 'PageReserved'. However, unlike other reserved PFNs, pages on NVDIMM shall still behave like normal ones in many cases, i.e. when used as backend memory of KVM guest. This patch introduces a new memory type, MEMORY_DEVICE_DEV_DAX. And set this flag while dax driver hotplug the device memory. Signed-off-by: Zhang Yi Signed-off-by: Zhang Yu Reviewed-by: Jan Kara --- drivers/dax/pmem.c | 1 + include/linux/memremap.h | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/drivers/dax/pmem.c b/drivers/dax/pmem.c index fd49b24..fb3f363 100644 --- a/drivers/dax/pmem.c +++ b/drivers/dax/pmem.c @@ -111,6 +111,7 @@ static int dax_pmem_probe(struct device *dev) return rc; dax_pmem->pgmap.ref = &dax_pmem->ref; + dax_pmem->pgmap.type = MEMORY_DEVICE_DEV_DAX; addr = devm_memremap_pages(dev, &dax_pmem->pgmap); if (IS_ERR(addr)) return PTR_ERR(addr); diff --git a/include/linux/memremap.h b/include/linux/memremap.h index f91f9e7..cd07ca8 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -53,11 +53,19 @@ struct vmem_altmap { * wakeup event whenever a page is unpinned and becomes idle. This * wakeup is used to coordinate physical address space management (ex: * fs truncate/hole punch) vs pinned pages (ex: device dma). + * + * MEMORY_DEVICE_DEV_DAX: + * Device memory that support raw access to persistent memory. Without need + * of an intervening filesystem, it could be directed mapped via an mmap + * capable character device. Together with the type MEMORY_DEVICE_FS_DAX, + * we could distinguish the persistent memory pages from normal ZONE_DEVICE + * pages. */ enum memory_type { MEMORY_DEVICE_PRIVATE = 1, MEMORY_DEVICE_PUBLIC, MEMORY_DEVICE_FS_DAX, + MEMORY_DEVICE_DEV_DAX, }; /* From patchwork Fri Sep 7 18:03:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 10591931 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB583139B for ; Fri, 7 Sep 2018 09:24:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E57C228537 for ; Fri, 7 Sep 2018 09:24:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D8B9F2AD99; Fri, 7 Sep 2018 09:24:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.9 required=2.0 tests=BAYES_00,DATE_IN_FUTURE_03_06, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C84828537 for ; Fri, 7 Sep 2018 09:24:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728367AbeIGOEz (ORCPT ); Fri, 7 Sep 2018 10:04:55 -0400 Received: from mga03.intel.com ([134.134.136.65]:33040 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728316AbeIGOEz (ORCPT ); Fri, 7 Sep 2018 10:04:55 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Sep 2018 02:24:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,341,1531810800"; d="scan'208";a="71100215" Received: from linux.intel.com ([10.54.29.200]) by orsmga007.jf.intel.com with ESMTP; 07 Sep 2018 02:24:52 -0700 Received: from dazhang1-ssd.sh.intel.com (unknown [10.239.48.163]) by linux.intel.com (Postfix) with ESMTP id 780375802B3; Fri, 7 Sep 2018 02:24:49 -0700 (PDT) From: Zhang Yi To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, pbonzini@redhat.com, dan.j.williams@intel.com, dave.jiang@intel.com, yu.c.zhang@intel.com, pagupta@redhat.com, david@redhat.com, jack@suse.cz, hch@lst.de Cc: linux-mm@kvack.org, rkrcmar@redhat.com, jglisse@redhat.com, yi.z.zhang@intel.com, Zhang Yi Subject: [PATCH V5 3/4] mm: add a function to differentiate the pages is from DAX device memory Date: Sat, 8 Sep 2018 02:03:55 +0800 Message-Id: <044309496afbb4121447dff6a453bd6b96d6068d.1536342881.git.yi.z.zhang@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP DAX driver hotplug the device memory and move it to memory zone, these pages will be marked reserved flag, however, some other kernel componet will misconceive these pages are reserved mmio (ex: we map these dev_dax or fs_dax pages to kvm for DIMM/NVDIMM backend). Together with the type MEMORY_DEVICE_FS_DAX, we can use is_dax_page() to differentiate the pages is DAX device memory or not. Signed-off-by: Zhang Yi Signed-off-by: Zhang Yu Acked-by: Jan Kara --- include/linux/mm.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 68a5121..de5cbc3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -889,6 +889,13 @@ static inline bool is_device_public_page(const struct page *page) page->pgmap->type == MEMORY_DEVICE_PUBLIC; } +static inline bool is_dax_page(const struct page *page) +{ + return is_zone_device_page(page) && + (page->pgmap->type == MEMORY_DEVICE_FS_DAX || + page->pgmap->type == MEMORY_DEVICE_DEV_DAX); +} + #else /* CONFIG_DEV_PAGEMAP_OPS */ static inline void dev_pagemap_get_ops(void) { @@ -912,6 +919,11 @@ static inline bool is_device_public_page(const struct page *page) { return false; } + +static inline bool is_dax_page(const struct page *page) +{ + return false; +} #endif /* CONFIG_DEV_PAGEMAP_OPS */ static inline void get_page(struct page *page) From patchwork Fri Sep 7 18:04:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Yi" X-Patchwork-Id: 10591937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 804E2139B for ; Fri, 7 Sep 2018 09:25:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7ACD02AD9C for ; Fri, 7 Sep 2018 09:25:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6EC362ADA3; Fri, 7 Sep 2018 09:25:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.9 required=2.0 tests=BAYES_00,DATE_IN_FUTURE_03_06, MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 23B0E2AD9C for ; Fri, 7 Sep 2018 09:25:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728404AbeIGOFI (ORCPT ); Fri, 7 Sep 2018 10:05:08 -0400 Received: from mga14.intel.com ([192.55.52.115]:32754 "EHLO mga14.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728317AbeIGOFI (ORCPT ); Fri, 7 Sep 2018 10:05:08 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Sep 2018 02:25:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,341,1531810800"; d="scan'208";a="81637128" Received: from linux.intel.com ([10.54.29.200]) by orsmga003.jf.intel.com with ESMTP; 07 Sep 2018 02:25:04 -0700 Received: from dazhang1-ssd.sh.intel.com (unknown [10.239.48.163]) by linux.intel.com (Postfix) with ESMTP id 5C75C5803DA; Fri, 7 Sep 2018 02:25:02 -0700 (PDT) From: Zhang Yi To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvdimm@lists.01.org, pbonzini@redhat.com, dan.j.williams@intel.com, dave.jiang@intel.com, yu.c.zhang@intel.com, pagupta@redhat.com, david@redhat.com, jack@suse.cz, hch@lst.de Cc: linux-mm@kvack.org, rkrcmar@redhat.com, jglisse@redhat.com, yi.z.zhang@intel.com, Zhang Yi Subject: [PATCH V5 4/4] kvm: add a check if pfn is from NVDIMM pmem. Date: Sat, 8 Sep 2018 02:04:08 +0800 Message-Id: <4e8c2e0facd46cfaf4ab79e19c9115958ab6f218.1536342881.git.yi.z.zhang@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For device specific memory space, when we move these area of pfn to memory zone, we will set the page reserved flag at that time, some of these reserved for device mmio, and some of these are not, such as NVDIMM pmem. Now, we map these dev_dax or fs_dax pages to kvm for DIMM/NVDIMM backend, since these pages are reserved, the check of kvm_is_reserved_pfn() misconceives those pages as MMIO. Therefor, we introduce 2 page map types, MEMORY_DEVICE_FS_DAX/MEMORY_DEVICE_DEV_DAX, to identify these pages are from NVDIMM pmem and let kvm treat these as normal pages. Without this patch, many operations will be missed due to this mistreatment to pmem pages, for example, a page may not have chance to be unpinned for KVM guest(in kvm_release_pfn_clean), not able to be marked as dirty/accessed(in kvm_set_pfn_dirty/accessed) etc. Signed-off-by: Zhang Yi Acked-by: Pankaj Gupta Signed-off-by: Alexander Duyck --- virt/kvm/kvm_main.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index c44c406..9c49634 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -147,8 +147,20 @@ __weak void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm, bool kvm_is_reserved_pfn(kvm_pfn_t pfn) { - if (pfn_valid(pfn)) - return PageReserved(pfn_to_page(pfn)); + struct page *page; + + if (pfn_valid(pfn)) { + page = pfn_to_page(pfn); + + /* + * For device specific memory space, there is a case + * which we need pass MEMORY_DEVICE_FS[DEV]_DAX pages + * to kvm, these pages marked reserved flag as it is a + * zone device memory, we need to identify these pages + * and let kvm treat these as normal pages + */ + return PageReserved(page) && !is_dax_page(page); + } return true; }