From patchwork Wed Aug 21 04:07:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Hubbard X-Patchwork-Id: 11105405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3673F1813 for ; Wed, 21 Aug 2019 04:07:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E9E30206BA for ; Wed, 21 Aug 2019 04:07:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="qUOBGoJI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9E30206BA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E1C836B0292; Wed, 21 Aug 2019 00:07:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C00776B0293; Wed, 21 Aug 2019 00:07:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC7776B0294; Wed, 21 Aug 2019 00:07:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0193.hostedemail.com [216.40.44.193]) by kanga.kvack.org (Postfix) with ESMTP id 84B0B6B0290 for ; Wed, 21 Aug 2019 00:07:36 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 1CED68248AA3 for ; Wed, 21 Aug 2019 04:07:36 +0000 (UTC) X-FDA: 75845100912.08.jar75_4248b535ccc26 X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,jhubbard@nvidia.com,:akpm@linux-foundation.org:hch@infradead.org:dan.j.williams@intel.com:david@fromorbit.com:ira.weiny@intel.com:jack@suse.cz:jgg@ziepe.ca:jglisse@redhat.com:vbabka@suse.cz:linux-kernel@vger.kernel.org::linux-fsdevel@vger.kernel.org:linux-rdma@vger.kernel.org:jhubbard@nvidia.com,RULES_HIT:30003:30034:30051:30054:30064:30070:30079:30090,0,RBL:216.228.121.143:@nvidia.com:.lbl8.mailshell.net-62.18.0.100 64.10.201.10,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: jar75_4248b535ccc26 X-Filterd-Recvd-Size: 8197 Received: from hqemgate14.nvidia.com (hqemgate14.nvidia.com [216.228.121.143]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Wed, 21 Aug 2019 04:07:35 +0000 (UTC) Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqemgate14.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Tue, 20 Aug 2019 21:07:34 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate102.nvidia.com (PGP Universal service); Tue, 20 Aug 2019 21:07:34 -0700 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Tue, 20 Aug 2019 21:07:34 -0700 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 21 Aug 2019 04:07:34 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 21 Aug 2019 04:07:33 +0000 Received: from hqnvemgw02.nvidia.com (172.16.227.111) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Wed, 21 Aug 2019 04:07:34 +0000 Received: from blueforge.nvidia.com (Not Verified[10.110.48.28]) by hqnvemgw02.nvidia.com with Trustwave SEG (v7,5,8,10121) id ; Tue, 20 Aug 2019 21:07:33 -0700 From: John Hubbard To: Andrew Morton CC: Christoph Hellwig , Dan Williams , Dave Chinner , Ira Weiny , Jan Kara , Jason Gunthorpe , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Vlastimil Babka , LKML , , , , John Hubbard Subject: [PATCH v2 3/3] mm/gup: introduce vaddr_pin_pages_remote(), and invoke it Date: Tue, 20 Aug 2019 21:07:27 -0700 Message-ID: <20190821040727.19650-4-jhubbard@nvidia.com> X-Mailer: git-send-email 2.22.1 In-Reply-To: <20190821040727.19650-1-jhubbard@nvidia.com> References: <20190821040727.19650-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1566360454; bh=yaS48N2o2CPvqMWzUWhpSpOm4JNtAfActS1Th7wctzE=; h=X-PGP-Universal:From:To:CC:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:X-NVConfidentiality: Content-Transfer-Encoding:Content-Type; b=qUOBGoJIEf/YL08oCTcD4iSkBMJKo16mo4WSO2uW9Tcw8WKfd/nCAvDlVcUPKWCNP Z8ZgbCOK0B2mKWctwfX3z0PsDzknh2dJF+WCT+ZUaDmv055HjoS1XzCjVmsC6tOxq+ cm/0B4U9iyKnl+GKGAAYdhgqzxvQju0MNYSprL2O16ZJrKl9ELPyKIyrjzVtIy73eB /SCUMKsMJVqoEMLZGyud5/rFqD2njRVbT+8DSl9Za/uWEWrDvphikno0VX7t4VZx4U PNsDCO0gp7taqzGYS1hizU+vuZMc2c+h7C+EDcp9KG8QhNUiIZEt/1ppoCRsXfGzJH 1EdKGW5PtlAbw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: vaddr_pin_user_pages_remote() is the "vaddr_pin_pages" corresponding variant to get_user_pages_remote(), except that: a) it sets FOLL_PIN, and b) it can handle FOLL_LONGTERM (and the associated vaddr_pin arg). Change process_vm_rw_single_vec() to invoke the new function. Signed-off-by: John Hubbard Cc: Ira Weiny --- include/linux/mm.h | 5 +++++ mm/gup.c | 34 ++++++++++++++++++++++++++++++++++ mm/process_vm_access.c | 23 +++++++++++++---------- 3 files changed, 52 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 6e7de424bf5e..849b509e9f89 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1606,6 +1606,11 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages, bool inc, long vaddr_pin_pages(unsigned long addr, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vaddr_pin *vaddr_pin); +long vaddr_pin_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, + unsigned long start, unsigned long nr_pages, + unsigned int gup_flags, struct page **pages, + struct vm_area_struct **vmas, int *locked, + struct vaddr_pin *vaddr_pin); void vaddr_unpin_pages(struct page **pages, unsigned long nr_pages, struct vaddr_pin *vaddr_pin, bool make_dirty); bool mapping_inode_has_layout(struct vaddr_pin *vaddr_pin, struct page *page); diff --git a/mm/gup.c b/mm/gup.c index ba316d960d7a..d713ed9d4b9a 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2522,3 +2522,37 @@ void vaddr_unpin_pages(struct page **pages, unsigned long nr_pages, __put_user_pages_dirty_lock(pages, nr_pages, make_dirty, vaddr_pin); } EXPORT_SYMBOL(vaddr_unpin_pages); + +/** + * vaddr_pin_user_pages_remote() - pin pages by virtual address and return the + * pages to the user. + * + * @tsk: the task_struct to use for page fault accounting, or + * NULL if faults are not to be recorded. + * @mm: mm_struct of target mm + * @addr: start address + * @nr_pages: number of pages to pin + * @gup_flags: flags to use for the pin. Please see FOLL_* documentation in + * mm.h. + * @pages: array of pages returned + * @vaddr_pin: If FOLL_LONGTERM is set, then vaddr_pin should point to an + * initialized struct that contains the owning mm and file. Otherwise, vaddr_pin + * should be set to NULL. + * + * This is the "vaddr_pin_pages" corresponding variant to + * get_user_pages_remote(), except that: + * a) it sets FOLL_PIN, and + * b) it can handle FOLL_LONGTERM (and the associated vaddr_pin arg). + */ +long vaddr_pin_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, + unsigned long start, unsigned long nr_pages, + unsigned int gup_flags, struct page **pages, + struct vm_area_struct **vmas, int *locked, + struct vaddr_pin *vaddr_pin) +{ + gup_flags |= FOLL_TOUCH | FOLL_REMOTE | FOLL_PIN; + + return __get_user_pages_locked(tsk, mm, start, nr_pages, pages, vmas, + locked, gup_flags, vaddr_pin); +} +EXPORT_SYMBOL(vaddr_pin_user_pages_remote); diff --git a/mm/process_vm_access.c b/mm/process_vm_access.c index 357aa7bef6c0..28e0a17b6080 100644 --- a/mm/process_vm_access.c +++ b/mm/process_vm_access.c @@ -44,7 +44,6 @@ static int process_vm_rw_pages(struct page **pages, if (vm_write) { copied = copy_page_from_iter(page, offset, copy, iter); - set_page_dirty_lock(page); } else { copied = copy_page_to_iter(page, offset, copy, iter); } @@ -96,7 +95,7 @@ static int process_vm_rw_single_vec(unsigned long addr, flags |= FOLL_WRITE; while (!rc && nr_pages && iov_iter_count(iter)) { - int pages = min(nr_pages, max_pages_per_loop); + int pinned_pages = min(nr_pages, max_pages_per_loop); int locked = 1; size_t bytes; @@ -106,14 +105,17 @@ static int process_vm_rw_single_vec(unsigned long addr, * current/current->mm */ down_read(&mm->mmap_sem); - pages = get_user_pages_remote(task, mm, pa, pages, flags, - process_pages, NULL, &locked); + + pinned_pages = vaddr_pin_user_pages_remote(task, mm, pa, + pinned_pages, flags, + process_pages, NULL, + &locked, NULL); if (locked) up_read(&mm->mmap_sem); - if (pages <= 0) + if (pinned_pages <= 0) return -EFAULT; - bytes = pages * PAGE_SIZE - start_offset; + bytes = pinned_pages * PAGE_SIZE - start_offset; if (bytes > len) bytes = len; @@ -122,10 +124,11 @@ static int process_vm_rw_single_vec(unsigned long addr, vm_write); len -= bytes; start_offset = 0; - nr_pages -= pages; - pa += pages * PAGE_SIZE; - while (pages) - put_page(process_pages[--pages]); + nr_pages -= pinned_pages; + pa += pinned_pages * PAGE_SIZE; + + /* If vm_write is set, the pages need to be made dirty: */ + vaddr_unpin_pages(process_pages, pinned_pages, NULL, vm_write); } return rc;