From patchwork Thu Jan 23 01:46:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Geffon X-Patchwork-Id: 11346689 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D29E109A for ; Thu, 23 Jan 2020 01:47:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5007E21734 for ; Thu, 23 Jan 2020 01:47:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="svRs4+Nn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5007E21734 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8ADBC6B0006; Wed, 22 Jan 2020 20:47:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 85F626B0007; Wed, 22 Jan 2020 20:47:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74CF96B0008; Wed, 22 Jan 2020 20:47:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 5C3956B0006 for ; Wed, 22 Jan 2020 20:47:00 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id DCF43181AEF15 for ; Thu, 23 Jan 2020 01:46:59 +0000 (UTC) X-FDA: 76407210558.17.sheet65_2d312815e13a X-Spam-Summary: 2,0,0,4d7a78d00ee12d19,d41d8cd98f00b204,3efsoxgckckidighhqpiqqing.eqonkpwz-oomxcem.qti@flex--bgeffon.bounces.google.com,::akpm@linux-foundation.org:mst@redhat.com:arnd@arndb.de:bgeffon@google.com:sonnyrao@google.com:minchan@kernel.org:joel@joelfernandes.org:lokeshgidra@google.com:linux-kernel@vger.kernel.org:linux-api@vger.kernel.org:yuzhao@google.com:jsbarnes@google.com,RULES_HIT:2:41:152:355:379:541:800:960:968:973:988:989:1260:1277:1313:1314:1345:1431:1437:1516:1518:1535:1593:1594:1605:1606:1730:1747:1777:1792:1981:2194:2199:2393:2553:2559:2562:2693:2890:2901:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3871:3872:3874:4119:4250:4605:5007:6261:6653:6742:7903:8660:9036:9969:10004:11026:11473:11658:11914:12043:12291:12296:12297:12555:12679:12683:12895:13148:13230:13972:14096:14097:14394:14659:21080:21324:21444:21450:21451:21627:21990:30003:30025:30054:30070:30090:30091,0,RBL:209.85.215.202:@flex--bgeffon.bounces.google.com:.lbl8.mailshell.net-62.18.0 .100 66. X-HE-Tag: sheet65_2d312815e13a X-Filterd-Recvd-Size: 8556 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Thu, 23 Jan 2020 01:46:59 +0000 (UTC) Received: by mail-pg1-f202.google.com with SMTP id u14so790168pgq.16 for ; Wed, 22 Jan 2020 17:46:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=QCHXxNK8/BbuJuJBSwzdO5h2VdmSh01YZxJCYkkB4Qw=; b=svRs4+NnIb8miZBRuWlIo1vn0BvkuOjAkFbeLBRV0SSyde+oZbYjiDCB4uNM5bN0mO DdDMAC88hxUA//KKlSgC9kaUgBcRkpQbKbAKpsyJUNVS8Whp9pUs36g2putQngJj8TQz RGR3CFpXTJSA8JPk3nzeZF26g8idlPT8YXAYLSzLkY0bhiezkAKuDrNC+uRVo3E3b0cf GVfow4dQF9ULUNIr1amkN6ddAaB5PftcRqKchcJQRR81QtyzcWb5oqvgX1UauvBhVvtK nCcBRmh3baRGraWhSGPSUDnVu3WquIUlG7edbRVHYSiAcrVe8sAZ0Fmz9u7bPr4JrZRt ME0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=QCHXxNK8/BbuJuJBSwzdO5h2VdmSh01YZxJCYkkB4Qw=; b=GgKdJr9TLxxaS5poD5UusGLzEo45NnAd+SyZRiSwO9LeI3iJikfrJB7fBdtwfEC9U7 d47+1HZxZGONz0AIF2iMbHkZ/HMpDSCdEK9x6zjVilsaxmBCiTmZwBtmoiOWxU/tgDJ/ MJsBmZqDEFTXzuGYYE+xXtg3x11BlNspM9G8rwOTMURQcU/jSxoHqjSA10rpUME+V4rH kf7l2br3oLrskCHoq8uya15qpBVOUWGX/3hueRdNGejEz+yFIylVaXbtl3+eF5B4WgT1 Iq4nLcoqaLjVUbRXCqcjyyQZdglz1c/7oZI/aPPH6ZjchW2zrY1gDALJcgZdRkxhW3tq slGw== X-Gm-Message-State: APjAAAV+DrPKdpVhrH6StWXU7hmFCwenteeAQu13NpfL1mrj281ZctuQ 89jguRw2GUJDiCP6XAxBhzCr9azYM0rwFInSrjvLY3CcTDoRREOOU8Zqbn5JskofHpo9gxjyV+b t3TR+zsebiAcukQIOIzwICMeEQo442X8yaaos17b/Si7zuQk8TYPXVBFaA5w= X-Google-Smtp-Source: APXvYqwEYpTzBJK3gG4R2RwTyQ62AuSI118fUjTqIftveRta/RrFTaFwaouAg3YZavLW7ZKqqATAdPCrm6PT X-Received: by 2002:a63:e649:: with SMTP id p9mr1318848pgj.15.1579744017943; Wed, 22 Jan 2020 17:46:57 -0800 (PST) Date: Wed, 22 Jan 2020 17:46:27 -0800 Message-Id: <20200123014627.71720-1-bgeffon@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.25.0.341.g760bfbb309-goog Subject: [PATCH] mm: Add MREMAP_DONTUNMAP to mremap(). From: Brian Geffon To: linux-mm@kvack.org Cc: Andrew Morton , "Michael S . Tsirkin" , Arnd Bergmann , Brian Geffon , Sonny Rao , Minchan Kim , Joel Fernandes , Lokesh Gidra , linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, Yu Zhao , Jesse Barnes X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: MREMAP_DONTUNMAP is an additional flag that can be used with MREMAP_FIXED to move a mapping to a new address. Normally, mremap(2) would then tear down the old vma so subsequent accesses to the vma cause a segfault. However, with this new flag it will keep the old vma with zapping PTEs so any access to the old VMA after that point will result in a pagefault. This feature will find a use in ChromeOS along with userfaultfd. Specifically we will want to register a VMA with userfaultfd and then pull it out from under a running process. By using MREMAP_DONTUNMAP we don't have to worry about mprotecting and then potentially racing with VMA permission changes from a running process. This feature also has a use case in Android, Lokesh Gidra has said that "As part of using userfaultfd for GC, We'll have to move the physical pages of the java heap to a separate location. For this purpose mremap will be used. Without the MREMAP_DONTUNMAP flag, when I mremap the java heap, its virtual mapping will be removed as well. Therefore, we'll require performing mmap immediately after. This is not only time consuming but also opens a time window where a native thread may call mmap and reserve the java heap's address range for its own usage. This flag solves the problem." Signed-off-by: Brian Geffon Reported-by: kbuild test robot Reported-by: Dan Carpenter --- include/uapi/linux/mman.h | 5 +++-- mm/mremap.c | 37 ++++++++++++++++++++++++++++++------- 2 files changed, 33 insertions(+), 9 deletions(-) diff --git a/include/uapi/linux/mman.h b/include/uapi/linux/mman.h index fc1a64c3447b..923cc162609c 100644 --- a/include/uapi/linux/mman.h +++ b/include/uapi/linux/mman.h @@ -5,8 +5,9 @@ #include #include -#define MREMAP_MAYMOVE 1 -#define MREMAP_FIXED 2 +#define MREMAP_MAYMOVE 1 +#define MREMAP_FIXED 2 +#define MREMAP_DONTUNMAP 4 #define OVERCOMMIT_GUESS 0 #define OVERCOMMIT_ALWAYS 1 diff --git a/mm/mremap.c b/mm/mremap.c index 122938dcec15..bf97c3eb538b 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -318,8 +318,8 @@ unsigned long move_page_tables(struct vm_area_struct *vma, static unsigned long move_vma(struct vm_area_struct *vma, unsigned long old_addr, unsigned long old_len, unsigned long new_len, unsigned long new_addr, - bool *locked, struct vm_userfaultfd_ctx *uf, - struct list_head *uf_unmap) + bool *locked, unsigned long flags, + struct vm_userfaultfd_ctx *uf, struct list_head *uf_unmap) { struct mm_struct *mm = vma->vm_mm; struct vm_area_struct *new_vma; @@ -408,6 +408,13 @@ static unsigned long move_vma(struct vm_area_struct *vma, if (unlikely(vma->vm_flags & VM_PFNMAP)) untrack_pfn_moved(vma); + if (unlikely(!err && (flags & MREMAP_DONTUNMAP))) { + if (vm_flags & VM_ACCOUNT) + vma->vm_flags |= VM_ACCOUNT; + + goto out; + } + if (do_munmap(mm, old_addr, old_len, uf_unmap) < 0) { /* OOM: unable to split vma, just get accounts right */ vm_unacct_memory(excess >> PAGE_SHIFT); @@ -422,6 +429,7 @@ static unsigned long move_vma(struct vm_area_struct *vma, vma->vm_next->vm_flags |= VM_ACCOUNT; } +out: if (vm_flags & VM_LOCKED) { mm->locked_vm += new_len >> PAGE_SHIFT; *locked = true; @@ -497,7 +505,7 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, static unsigned long mremap_to(unsigned long addr, unsigned long old_len, unsigned long new_addr, unsigned long new_len, bool *locked, - struct vm_userfaultfd_ctx *uf, + unsigned long flags, struct vm_userfaultfd_ctx *uf, struct list_head *uf_unmap_early, struct list_head *uf_unmap) { @@ -545,6 +553,17 @@ static unsigned long mremap_to(unsigned long addr, unsigned long old_len, old_len = new_len; } + /* + * MREMAP_DONTUNMAP expands by old_len + (new_len - old_len), we will + * check that we can expand by old_len and vma_to_resize will handle + * the vma growing. + */ + if (unlikely(flags & MREMAP_DONTUNMAP && !may_expand_vm(mm, + vma->vm_flags, old_len >> PAGE_SHIFT))) { + ret = -ENOMEM; + goto out; + } + vma = vma_to_resize(addr, old_len, new_len, &charged); if (IS_ERR(vma)) { ret = PTR_ERR(vma); @@ -561,7 +580,7 @@ static unsigned long mremap_to(unsigned long addr, unsigned long old_len, if (IS_ERR_VALUE(ret)) goto out1; - ret = move_vma(vma, addr, old_len, new_len, new_addr, locked, uf, + ret = move_vma(vma, addr, old_len, new_len, new_addr, locked, flags, uf, uf_unmap); if (!(offset_in_page(ret))) goto out; @@ -609,12 +628,15 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, addr = untagged_addr(addr); new_addr = untagged_addr(new_addr); - if (flags & ~(MREMAP_FIXED | MREMAP_MAYMOVE)) + if (flags & ~(MREMAP_FIXED | MREMAP_MAYMOVE | MREMAP_DONTUNMAP)) return ret; if (flags & MREMAP_FIXED && !(flags & MREMAP_MAYMOVE)) return ret; + if (flags & MREMAP_DONTUNMAP && !(flags & MREMAP_FIXED)) + return ret; + if (offset_in_page(addr)) return ret; @@ -634,7 +656,8 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, if (flags & MREMAP_FIXED) { ret = mremap_to(addr, old_len, new_addr, new_len, - &locked, &uf, &uf_unmap_early, &uf_unmap); + &locked, flags, &uf, &uf_unmap_early, + &uf_unmap); goto out; } @@ -712,7 +735,7 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, } ret = move_vma(vma, addr, old_len, new_len, new_addr, - &locked, &uf, &uf_unmap); + &locked, flags, &uf, &uf_unmap); } out: if (offset_in_page(ret)) {