From patchwork Wed Aug 15 18:49:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10566745 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D679E921 for ; Wed, 15 Aug 2018 18:50:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ACDC62AF18 for ; Wed, 15 Aug 2018 18:50:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB1862AF35; Wed, 15 Aug 2018 18:50:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E2AC2AF36 for ; Wed, 15 Aug 2018 18:50:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 909EC6B0007; Wed, 15 Aug 2018 14:50:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 890EB6B0008; Wed, 15 Aug 2018 14:50:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 732E66B000A; Wed, 15 Aug 2018 14:50:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 30AC26B0007 for ; Wed, 15 Aug 2018 14:50:22 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id z18-v6so912949pfe.19 for ; Wed, 15 Aug 2018 11:50:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=HK+nkvH3q+28k2vljA7/OO0Kl2USle1UcBkBLdrxdd4=; b=mEsWNQZ1j0r1i+Ur94tVJeR1xVBwqz6DTd5uJ5Z2PMz+US6g8/5Kbhhm8LbIW7Af/B WMPe+xDGy3jjLR9nmAHxIG32+rN/l368Hsws9QqppmhRJTW1sRYoHE1NX+l4Omnn3xY3 1QEINq9gHYwTeLgFegG9YQUEo/4NryZ5OwHRVsT/dhR0tMw0MqwNmQGzNuiDlta9OQ2V JHwxDZfm/E07dn0Fb/ZIBAy6dcF1bJwk078Fy3V8kL1CjKMF7u8EQI+QAI9r6IchR2nE atEzUU7+hR7kKtU114o1WACHdwyW+R6adk3zrI/cYlc6dihlFnLnAAmhNpyzzjqshq31 S0jQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: AOUpUlEgEAreLIG61NJ1FON40IcsvnP7YLGgydg3KHehXSlYEbcv3V8O D6bUpJQPsyUvLoNBcMI9q3GnYIOGAjszUWeZje7aYCzLWWTFDNOPqnshLtR/0s9ppzdes25UUsS AEhwih1vaDwiqHDzifsy1pjD8t+hTIMAH0KZ0tylAQUPCUTs/47LowXSOnYPJVO2+cA== X-Received: by 2002:a63:6345:: with SMTP id x66-v6mr26417386pgb.43.1534359021867; Wed, 15 Aug 2018 11:50:21 -0700 (PDT) X-Google-Smtp-Source: AA+uWPw8w+/3BgiIsWL2X0wzkf/HPmpwyQEcNPHqcbMA/CETeaeuuJzIcmt42w7WYLFmCzqq8JsF X-Received: by 2002:a63:6345:: with SMTP id x66-v6mr26417355pgb.43.1534359020951; Wed, 15 Aug 2018 11:50:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534359020; cv=none; d=google.com; s=arc-20160816; b=uEZ8jPT1aDZeQ3nuYOBYWpfIR+M/b0XiDaVUZ9pfJCQ/8DY1vtij2XtsyJ8xAu7DE2 Y4osj6LfprSKUSI5Z+h8KRLnHv7310JVt7xRQZUcJtTu5xh7sORH6vsXu6ZC8MEFHnnU 9KsFaKKLGoNcXkLawVeuu35PUrlhP24uEOtGkCZ0i+Yhkz6k8iWH32o5ZLc/pDUFP8/p 6bmELqQqmcSwjp/Dcxdp+sxdHZCFitzEfSOBNifhd/yzaZQRRUOjUnYhm3w9JJOWG39q xseQNmCpyGdIahkjJ0+3EM9fCrtXtS8wo2c/UHWiPu0Fy6223/uoXW1Uu6a7j7S4fYz2 QS5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=HK+nkvH3q+28k2vljA7/OO0Kl2USle1UcBkBLdrxdd4=; b=qZnsLBBfek14Lyh3uDeJTCpD21l1bqHr3UIaOCAN0zdS+pRdcoAzTG0KRZJucQeeb2 XBi8KE42i1hU7kN9ocwGCofqA9Us037rM7oCmpSOAmNJiKxqrwUOKy5rBo780kzOHVJh S0L+r0tK1KEBVA2oDXPD1WRPibqAnOD8rBVR7iM4r6ja4DN9CU3cKU7BfiYZCC+t5VdR mYDj6Bp/6NNWO3xFwhILjJcynmBD86XauBvFFK6eVnJ5/db6JJROEbin6mjvEcBpuXMi YbS8K+jpzZ0+0ffirPVyWLV9OB7w8kvq17Q7WPIS1Tl4RBSV/jRZ0ghkdYU48aa7cke9 LszA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com. [115.124.30.133]) by mx.google.com with ESMTPS id b1-v6si19328959pls.367.2018.08.15.11.50.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Aug 2018 11:50:20 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) client-ip=115.124.30.133; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e07402;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0T6gJtNa_1534358999; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T6gJtNa_1534358999) by smtp.aliyun-inc.com(127.0.0.1); Thu, 16 Aug 2018 02:50:07 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, kirill@shutemov.name, vbabka@suse.cz, akpm@linux-foundation.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, alexander.shishkin@linux.intel.com, jolsa@redhat.com, namhyung@kernel.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v8 PATCH 1/5] mm: refactor do_munmap() to extract the common part Date: Thu, 16 Aug 2018 02:49:46 +0800 Message-Id: <1534358990-85530-2-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> References: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Introduces three new helper functions: * addr_ok() * munmap_lookup_vma() * munlock_vmas() They will be used by do_munmap() and the new do_munmap with zapping large mapping early in the later patch. There is no functional change, just code refactor. Reviewed-by: Laurent Dufour Acked-by: Vlastimil Babka Signed-off-by: Yang Shi --- mm/mmap.c | 106 +++++++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 74 insertions(+), 32 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 17bbf4d..f05f49b 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2681,35 +2681,42 @@ int split_vma(struct mm_struct *mm, struct vm_area_struct *vma, return __split_vma(mm, vma, addr, new_below); } -/* Munmap is split into 2 main parts -- this part which finds - * what needs doing, and the areas themselves, which do the - * work. This now handles partial unmappings. - * Jeremy Fitzhardinge - */ -int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, - struct list_head *uf) +static inline bool addr_ok(unsigned long start, size_t len) { - unsigned long end; - struct vm_area_struct *vma, *prev, *last; - if ((offset_in_page(start)) || start > TASK_SIZE || len > TASK_SIZE-start) - return -EINVAL; + return false; - len = PAGE_ALIGN(len); - if (len == 0) - return -EINVAL; + if (PAGE_ALIGN(len) == 0) + return false; + + return true; +} + +/* + * munmap_lookup_vma: find the first overlap vma and split overlap vmas. + * @mm: mm_struct + * @start: start address + * @end: end address + * + * Return: %NULL if no VMA overlaps this range. An ERR_PTR if an + * overlapping VMA could not be split. Otherwise a pointer to the first + * VMA which overlaps the range. + */ +static struct vm_area_struct *munmap_lookup_vma(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + struct vm_area_struct *vma, *prev, *last; /* Find the first overlapping VMA */ vma = find_vma(mm, start); if (!vma) - return 0; - prev = vma->vm_prev; - /* we have start < vma->vm_end */ + return NULL; + /* we have start < vma->vm_end */ /* if it doesn't overlap, we have nothing.. */ - end = start + len; if (vma->vm_start >= end) - return 0; + return NULL; + prev = vma->vm_prev; /* * If we need to split any vma, do it now to save pain later. @@ -2727,11 +2734,11 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, * its limit temporarily, to help free resources as expected. */ if (end < vma->vm_end && mm->map_count >= sysctl_max_map_count) - return -ENOMEM; + return ERR_PTR(-ENOMEM); error = __split_vma(mm, vma, start, 0); if (error) - return error; + return ERR_PTR(error); prev = vma; } @@ -2740,10 +2747,53 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, if (last && end > last->vm_start) { int error = __split_vma(mm, last, end, 1); if (error) - return error; + return ERR_PTR(error); } vma = prev ? prev->vm_next : mm->mmap; + return vma; +} + +static inline void munlock_vmas(struct vm_area_struct *vma, + unsigned long end) +{ + struct mm_struct *mm = vma->vm_mm; + + while (vma && vma->vm_start < end) { + if (vma->vm_flags & VM_LOCKED) { + mm->locked_vm -= vma_pages(vma); + munlock_vma_pages_all(vma); + } + vma = vma->vm_next; + } +} + +/* Munmap is split into 2 main parts -- this part which finds + * what needs doing, and the areas themselves, which do the + * work. This now handles partial unmappings. + * Jeremy Fitzhardinge + */ +int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, + struct list_head *uf) +{ + unsigned long end; + struct vm_area_struct *vma, *prev; + + if (!addr_ok(start, len)) + return -EINVAL; + + len = PAGE_ALIGN(len); + + end = start + len; + + vma = munmap_lookup_vma(mm, start, end); + if (!vma) + return 0; + if (IS_ERR(vma)) + return PTR_ERR(vma); + + prev = vma->vm_prev; + if (unlikely(uf)) { /* * If userfaultfd_unmap_prep returns an error the vmas @@ -2762,16 +2812,8 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, /* * unlock any mlock()ed ranges before detaching vmas */ - if (mm->locked_vm) { - struct vm_area_struct *tmp = vma; - while (tmp && tmp->vm_start < end) { - if (tmp->vm_flags & VM_LOCKED) { - mm->locked_vm -= vma_pages(tmp); - munlock_vma_pages_all(tmp); - } - tmp = tmp->vm_next; - } - } + if (mm->locked_vm) + munlock_vmas(vma, end); /* * Remove the vma's, and unmap the actual pages From patchwork Wed Aug 15 18:49:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10566753 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7EE62921 for ; Wed, 15 Aug 2018 18:50:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 557E42AF5F for ; Wed, 15 Aug 2018 18:50:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 52A232AF34; Wed, 15 Aug 2018 18:50:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A1E922AF36 for ; Wed, 15 Aug 2018 18:50:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 673276B0269; Wed, 15 Aug 2018 14:50:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6203F6B026A; Wed, 15 Aug 2018 14:50:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C2AB6B026B; Wed, 15 Aug 2018 14:50:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 01A946B0269 for ; Wed, 15 Aug 2018 14:50:36 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id d22-v6so924634pfn.3 for ; Wed, 15 Aug 2018 11:50:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=LJTorXFucv5v4F07S1owqMgoSIlym/JkIF2Oybwf2xo=; b=PJs8D1wEjjmsPY8oOcioJA/2xjRgOpKhQWxVekDr7ql9g+gBE8EM4hLBUIxokMcQHh FXeZUmHYrnxaUcFIfN3PO+UlLjgfN18PSHNsaVaOTdgqTfMybc+lhoDKAcnRaHBqPQdZ DbItAtUXshpBmTlUOLSU50BEh4zZvic2pd9/RcGA5tYWJY3eyps3JESnJ1t4NLu6tfUO /TQZAOmsLWIRNNPzkKtymyzkrQKwtu3ZJ2Q9N2lPJY0RMKdCxhTRvhHYLSUnL62zPUUe HQI8r5MOXp/QYck73PlqjevCON285Y7A1jty0fdQdRL2r++CUIzspL0VqVD231fARCqS FbAQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.36 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: AOUpUlFLhbKQVBUAIg1OxSJaGrNQgMVfAtQQkBI1hNk06HKh4SFW9XJ9 21eUoKn+ccXbKMZHtSQJrX6bLrKCWSuvA1mBeF3LNb8TMr3CbOdOFHpZMPN5zMyuQWZMNRBa5+1 E9358aJy+Flb7C4Q0jZ2f5bNaK4tUQPLGe0ijV5lO4aRf6CubG508UneXMfIsj0Oo/A== X-Received: by 2002:a17:902:6f16:: with SMTP id w22-v6mr6454312plk.127.1534359035677; Wed, 15 Aug 2018 11:50:35 -0700 (PDT) X-Google-Smtp-Source: AA+uWPxtKOp7WGNSaUCDdwC1XSif+Rzqt/J9RCIsNrBuh632FxYlSvWuUeEK3bx6zYygI5EorwkZ X-Received: by 2002:a17:902:6f16:: with SMTP id w22-v6mr6454269plk.127.1534359034627; Wed, 15 Aug 2018 11:50:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534359034; cv=none; d=google.com; s=arc-20160816; b=HJl/+DtxchqJx1aqCcaipQiIu3ic3PknjOWdPJLxBsbfW7yK1KFTR2h8EHyHQW5HLW 7kAjxOJiEu/ZszZ0SXdxNYrf+sfN59cNU4kxx7wFWsUKVAZr/ab5oRSMn+nXpYx/ZFWq ZsWMgA94w7oFfOZQDccmGqxLjFSdyROJf1owKC8Z2UF3HD6kR3uRzJAF1z5T2ALe8/xo jbFxopwliqUj4L/fmFiMTMU8SIo9kqFveTVHZMidkOa6BNGuFzjpJPdFLa0gvId5bNNT hcqsQGfOxeo0+ndm9wVRbXSrpsswmvRZnWHN7JJ3cL64SVFDs0FI0OfqGG/q6LhE2s/9 AFgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=LJTorXFucv5v4F07S1owqMgoSIlym/JkIF2Oybwf2xo=; b=zXUBrb/Wl0Zb0Qd7k3mLUO3GN/QWl0jzQ48i0KMg0B9S7gQxq7mLg4CjzBLfKK/sws EuvOyW2KxffFxTvLiSNhROF9B/16eYIQyfQhw3NELnsXhnH0+zkT0mvoMIodKD1EVOZY PagQLjHwJVTJYZtirqENa2NxLzPK//mRhC4vQDtq9vEqr5NAjnD929bpSGzA6m13HqVF GGitfwcN1iZiKXdWgcVryU2Q0zGqladUtoLcG6wFXmTaX0jGmkkNmyOIlDvf2AyFrlL0 12ZxRYRQyxBa9Le1zUaTg8IYqSUeOAaQxqk3T4/mQn696PaGBmXGcUDFoyp/Ezo8j8pt zbAA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.36 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out4436.biz.mail.alibaba.com (out4436.biz.mail.alibaba.com. [47.88.44.36]) by mx.google.com with ESMTPS id d9-v6si20206719pgo.470.2018.08.15.11.50.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Aug 2018 11:50:34 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.36 as permitted sender) client-ip=47.88.44.36; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 47.88.44.36 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01422;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0T6gJtNa_1534358999; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T6gJtNa_1534358999) by smtp.aliyun-inc.com(127.0.0.1); Thu, 16 Aug 2018 02:50:07 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, kirill@shutemov.name, vbabka@suse.cz, akpm@linux-foundation.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, alexander.shishkin@linux.intel.com, jolsa@redhat.com, namhyung@kernel.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v8 PATCH 2/5] uprobes: introduce has_uprobes helper Date: Thu, 16 Aug 2018 02:49:47 +0800 Message-Id: <1534358990-85530-3-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> References: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We need check if mm or vma has uprobes in the following patch to check if a vma could be unmapped with holding read mmap_sem. The checks and pre-conditions used by uprobe_munmap() look just suitable for this purpose. Extracting those checks into a helper function, has_uprobes(). Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Arnaldo Carvalho de Melo Cc: Alexander Shishkin Cc: Jiri Olsa Cc: Namhyung Kim Cc: Vlastimil Babka Signed-off-by: Yang Shi --- include/linux/uprobes.h | 7 +++++++ kernel/events/uprobes.c | 23 ++++++++++++++++------- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index 0a294e9..418764e 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -149,6 +149,8 @@ struct uprobes_state { extern bool arch_uprobe_ignore(struct arch_uprobe *aup, struct pt_regs *regs); extern void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, void *src, unsigned long len); +extern bool has_uprobes(struct vm_area_struct *vma, unsigned long start, + unsigned long end); #else /* !CONFIG_UPROBES */ struct uprobes_state { }; @@ -203,5 +205,10 @@ static inline void uprobe_copy_process(struct task_struct *t, unsigned long flag static inline void uprobe_clear_state(struct mm_struct *mm) { } +static inline bool has_uprobes(struct vm_area_struct *vma, unsigned long start, + unsgined long end) +{ + return false; +} #endif /* !CONFIG_UPROBES */ #endif /* _LINUX_UPROBES_H */ diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index aed1ba5..568481c 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -1114,22 +1114,31 @@ int uprobe_mmap(struct vm_area_struct *vma) return !!n; } -/* - * Called in context of a munmap of a vma. - */ -void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end) +bool +has_uprobes(struct vm_area_struct *vma, unsigned long start, unsigned long end) { if (no_uprobe_events() || !valid_vma(vma, false)) - return; + return false; if (!atomic_read(&vma->vm_mm->mm_users)) /* called by mmput() ? */ - return; + return false; if (!test_bit(MMF_HAS_UPROBES, &vma->vm_mm->flags) || test_bit(MMF_RECALC_UPROBES, &vma->vm_mm->flags)) - return; + return false; if (vma_has_uprobes(vma, start, end)) + return true; + + return false; +} + +/* + * Called in context of a munmap of a vma. + */ +void uprobe_munmap(struct vm_area_struct *vma, unsigned long start, unsigned long end) +{ + if (has_uprobes(vma, start, end)) set_bit(MMF_RECALC_UPROBES, &vma->vm_mm->flags); } From patchwork Wed Aug 15 18:49:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10566743 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEAAB921 for ; Wed, 15 Aug 2018 18:50:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A1BFF2AF34 for ; Wed, 15 Aug 2018 18:50:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9F6912AF1B; Wed, 15 Aug 2018 18:50:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EE972AF1B for ; Wed, 15 Aug 2018 18:50:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A6A7E6B0005; Wed, 15 Aug 2018 14:50:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9CA756B0006; Wed, 15 Aug 2018 14:50:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 844736B0007; Wed, 15 Aug 2018 14:50:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f69.google.com (mail-pl0-f69.google.com [209.85.160.69]) by kanga.kvack.org (Postfix) with ESMTP id 418A16B0005 for ; Wed, 15 Aug 2018 14:50:14 -0400 (EDT) Received: by mail-pl0-f69.google.com with SMTP id w11-v6so1123489plq.8 for ; Wed, 15 Aug 2018 11:50:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Fkb9Ii+TjHwtHEZlQiWM4pDISi6LoiAmvbWphuMvQjs=; b=JQfuWQf9nkRaHHnTgtx0ft7sE/1guFzMtnip6MAOfKHDrYe7GOSPuUl7YzlVhRkclg tCo4S3RGVQvbgCSIsq7LlMirlX2VnbMj7QfTETaoT5bF2pLEWk+QPvpog3X/BYhBgm19 wbIGzI3Bo8VYNQjGdH2W2lOKOF23xLLsG4dzY4DG+7WzWsgauf77Nyi3QXj1gI3ZrgPm ybjQwssWbnDId7OJpaHP/ns5HU+rKFnDxZQsVY9SFzJ3OwbAbefgBbybrqnCBJ1Yjaw9 Y9NQFD6bj4l5yvpkpFEhSayn4jQzt8BowaPXtOKzPDCbVfx6q3JGJwdiMmL7Qjc9uIz/ elNQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: AOUpUlEYJyDdb/0DihOkP7tsLpdUWu0hmxWJZBJTqTx52ZfjnYt+aht0 1WftUkBITepO/985p+uu9nKeHt2t7Loytxqein1qzD45zNY6qGV6P8J/jdVgcXjXlmb9TyHKOvV jUORSm6Hqilhyy6vjF91b4Ys3RqvXOjGvktcJAHp0TtVaDQUXHmU00dsiVhyCcw3Khg== X-Received: by 2002:a17:902:6b05:: with SMTP id o5-v6mr25228147plk.338.1534359013927; Wed, 15 Aug 2018 11:50:13 -0700 (PDT) X-Google-Smtp-Source: AA+uWPxy1ZykhC5Y9QuqohnCbCVAlBhPEvFo0JzyPP3LcgndbqXwy5QzpWCff4eRtQ4qcJmsC/Nm X-Received: by 2002:a17:902:6b05:: with SMTP id o5-v6mr25228075plk.338.1534359012373; Wed, 15 Aug 2018 11:50:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534359012; cv=none; d=google.com; s=arc-20160816; b=lcpLC0AMrLPI1rHasa61U4Wu1B/jm5xPqnkMQD6y7rxyABO650Og83EmSGQetr234s jyva1CsbfQK6FJ1Q9TFW8cuFc6CDCmlMat0YuQaywi9YE6MbdGPCY6rYvosDGKIr9oLQ GCp6tMxYxx8rvOIlfp9biSjBL5xvQbQJb1CVrcQtwpMI715mInCCyoesDIBX2Umv3rva YGsMQ+XyFAmSXPJNoltDqqY3Ppw1ru8jttER2xXl8dBgSMMbEb+WUdu2GsaW1ndfP/FZ j/8nbLNzmTeaOCY7UvTmihdjzht2euTX84BlFLBezD2CpMIl499q7BJB2R5gaRd5Ci8a h2KQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=Fkb9Ii+TjHwtHEZlQiWM4pDISi6LoiAmvbWphuMvQjs=; b=fbQ7phBj0gHZA2hTJDgwKGpYplvol0adlQoFdqfSx+aZaA+nfS6S7Go8D1VGZDkZTj Hw0tGndy2KVT9TULUbbiRECKo+eiGwGuYXyZNSJmfzagEaetnnDJPkVCPh8Ckj1JAU8m CbhFDIhmreKeMted6uNhazFaNatSuaixu0gSryfQo351cUb4Hf546pKH2PR6iMvLPrEX bEq4Em8ec4CKPu0tGbRsV/rEB9QfU73AARldGnxscXIPrs7wbazgRC2qpT01o6WAPGL1 4cIQffanQaO0ArxlucjjUf89vGVFgr4mEM3CqRndbw88YFTtd9lPZ8qs6DpxteNPKkZJ bNPg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com. [115.124.30.133]) by mx.google.com with ESMTPS id a35-v6si20538241pla.27.2018.08.15.11.50.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Aug 2018 11:50:12 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) client-ip=115.124.30.133; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R571e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01f04455;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0T6gJtNa_1534358999; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T6gJtNa_1534358999) by smtp.aliyun-inc.com(127.0.0.1); Thu, 16 Aug 2018 02:50:08 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, kirill@shutemov.name, vbabka@suse.cz, akpm@linux-foundation.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, alexander.shishkin@linux.intel.com, jolsa@redhat.com, namhyung@kernel.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v8 PATCH 3/5] mm: mmap: zap pages with read mmap_sem in munmap Date: Thu, 16 Aug 2018 02:49:48 +0800 Message-Id: <1534358990-85530-4-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> References: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When running some mmap/munmap scalability tests with large memory (i.e. > 300GB), the below hung task issue may happen occasionally. INFO: task ps:14018 blocked for more than 120 seconds. Tainted: G E 4.9.79-009.ali3000.alios7.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. ps D 0 14018 1 0x00000004 ffff885582f84000 ffff885e8682f000 ffff880972943000 ffff885ebf499bc0 ffff8828ee120000 ffffc900349bfca8 ffffffff817154d0 0000000000000040 00ffffff812f872a ffff885ebf499bc0 024000d000948300 ffff880972943000 Call Trace: [] ? __schedule+0x250/0x730 [] schedule+0x36/0x80 [] rwsem_down_read_failed+0xf0/0x150 [] call_rwsem_down_read_failed+0x18/0x30 [] down_read+0x20/0x40 [] proc_pid_cmdline_read+0xd9/0x4e0 [] ? do_filp_open+0xa5/0x100 [] __vfs_read+0x37/0x150 [] ? security_file_permission+0x9b/0xc0 [] vfs_read+0x96/0x130 [] SyS_read+0x55/0xc0 [] entry_SYSCALL_64_fastpath+0x1a/0xc5 It is because munmap holds mmap_sem exclusively from very beginning to all the way down to the end, and doesn't release it in the middle. When unmapping large mapping, it may take long time (take ~18 seconds to unmap 320GB mapping with every single page mapped on an idle machine). Zapping pages is the most time consuming part, according to the suggestion from Michal Hocko [1], zapping pages can be done with holding read mmap_sem, like what MADV_DONTNEED does. Then re-acquire write mmap_sem to cleanup vmas. But, some part may need write mmap_sem, for example, vma splitting. So, the design is as follows: acquire write mmap_sem lookup vmas (find and split vmas) deal with special mappings detach vmas downgrade_write zap pages free page tables release mmap_sem The vm events with read mmap_sem may come in during page zapping, but since vmas have been detached before, they, i.e. page fault, gup, etc, will not be able to find valid vma, then just return SIGSEGV or -EFAULT as expected. If the vma has VM_HUGETLB | VM_PFNMAP or uprobe, they are considered as special mappings. They will be handled by falling back to regular do_munmap() with exclusive mmap_sem held in this patch since they may update vm flags. But, with the "detach vmas first" approach, the vmas have been detached when vm flags are updated, so it sounds safe to update vm flags with read mmap_sem for this specific case. So, VM_HUGETLB and VM_PFNMAP will be handled by using the optimized path in the following separate patches for bisectable sake. However, uprobes mappings will keep using regular do_munmap() since unmapping uprobe areas may need update mm flags. It might be not safe with just holding read mmap_sem even though affected vmas have been detached. With the "detach vmas first" approach we don't have to re-acquire mmap_sem again to clean up vmas to avoid race window which might get the address space changed since downgrade_write() doesn't release the lock to lead regression, which simply downgrades to read lock. And, since the lock acquire/release cost is managed to the minimum and almost as same as before, the optimization could be extended to any size of mapping without incurring significant penalty to small mappings. For the time being, just do this in munmap syscall path. Other vm_munmap() or do_munmap() call sites (i.e mmap, mremap, etc) remain intact due to some implementation difficulties since they acquire write mmap_sem from very beginning and hold it until the end, do_munmap() might be called in the middle. But, the optimized do_munmap would like to be called without mmap_sem held so that we can do the optimization. So, if we want to do the similar optimization for mmap/mremap path, I'm afraid we would have to redesign them. mremap might be called on very large area depending on the usecases, the optimization to it will be considered in the future. With the patches, exclusive mmap_sem hold time when munmap a 80GB address space on a machine with 32 cores of E5-2680 @ 2.70GHz dropped to us level from second. munmap_test-15002 [008] 594.380138: funcgraph_entry: | vm_munmap_zap_rlock() { munmap_test-15002 [008] 594.380146: funcgraph_entry: !2485684 us | unmap_region(); munmap_test-15002 [008] 596.865836: funcgraph_exit: !2485692 us | } Here the excution time of unmap_region() is used to evaluate the time of holding read mmap_sem, then the remaining time is used with holding exclusive lock. [1] https://lwn.net/Articles/753269/ Suggested-by: Michal Hocko Suggested-by: Kirill A. Shutemov Cc: Matthew Wilcox Cc: Laurent Dufour Cc: Vlastimil Babka Cc: Andrew Morton Signed-off-by: Yang Shi --- mm/mmap.c | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index f05f49b..e92f680 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2768,6 +2768,89 @@ static inline void munlock_vmas(struct vm_area_struct *vma, } } +/* + * Zap pages with read mmap_sem held + * + * uf is the list for userfaultfd + */ +static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, + size_t len, struct list_head *uf) +{ + unsigned long end; + struct vm_area_struct *start_vma, *prev, *vma; + int ret = 0; + + if (!addr_ok(start, len)) + return -EINVAL; + + len = PAGE_ALIGN(len); + + end = start + len; + + /* + * Need write mmap_sem to split vmas and detach vmas + * splitting vma up-front to save PITA to clean if it is failed + */ + if (down_write_killable(&mm->mmap_sem)) + return -EINTR; + + start_vma = munmap_lookup_vma(mm, start, end); + if (!start_vma) + goto out; + if (IS_ERR(start_vma)) { + ret = PTR_ERR(start_vma); + goto out; + } + + prev = start_vma->vm_prev; + + if (unlikely(uf)) { + ret = userfaultfd_unmap_prep(start_vma, start, end, uf); + if (ret) + goto out; + } + + /* + * Unmapping vmas, which have: + * VM_HUGETLB or + * VM_PFNMAP or + * uprobes + * need get done with write mmap_sem held since they may update + * vm_flags. Deal with such mappings with regular do_munmap() call. + */ + for (vma = start_vma; vma && vma->vm_start < end; vma = vma->vm_next) { + if ((vma->vm_file && + has_uprobes(vma, vma->vm_start, vma->vm_end)) || + (vma->vm_flags & (VM_HUGETLB | VM_PFNMAP))) + goto regular_path; + } + + /* Handle mlocked vmas */ + if (mm->locked_vm) + munlock_vmas(start_vma, end); + + /* Detach vmas from rbtree */ + detach_vmas_to_be_unmapped(mm, start_vma, prev, end); + + downgrade_write(&mm->mmap_sem); + + /* Zap mappings with read mmap_sem */ + unmap_region(mm, start_vma, prev, start, end); + + arch_unmap(mm, start_vma, start, end); + remove_vma_list(mm, start_vma); + up_read(&mm->mmap_sem); + + return 0; + +regular_path: + ret = do_munmap(mm, start, len, uf); + +out: + up_write(&mm->mmap_sem); + return ret; +} + /* Munmap is split into 2 main parts -- this part which finds * what needs doing, and the areas themselves, which do the * work. This now handles partial unmappings. @@ -2829,6 +2912,17 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, return 0; } +static int vm_munmap_zap_rlock(unsigned long start, size_t len) +{ + int ret; + struct mm_struct *mm = current->mm; + LIST_HEAD(uf); + + ret = do_munmap_zap_rlock(mm, start, len, &uf); + userfaultfd_unmap_complete(mm, &uf); + return ret; +} + int vm_munmap(unsigned long start, size_t len) { int ret; @@ -2848,10 +2942,9 @@ int vm_munmap(unsigned long start, size_t len) SYSCALL_DEFINE2(munmap, unsigned long, addr, size_t, len) { profile_munmap(addr); - return vm_munmap(addr, len); + return vm_munmap_zap_rlock(addr, len); } - /* * Emulation of deprecated remap_file_pages() syscall. */ From patchwork Wed Aug 15 18:49:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10566749 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85C6A109C for ; Wed, 15 Aug 2018 18:50:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59E1D2AF00 for ; Wed, 15 Aug 2018 18:50:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 578FF2AF4F; Wed, 15 Aug 2018 18:50:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E31F02AF37 for ; Wed, 15 Aug 2018 18:50:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CDAB26B000D; Wed, 15 Aug 2018 14:50:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C62636B0010; Wed, 15 Aug 2018 14:50:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AB6166B000D; Wed, 15 Aug 2018 14:50:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f72.google.com (mail-pl0-f72.google.com [209.85.160.72]) by kanga.kvack.org (Postfix) with ESMTP id 4FA066B000A for ; Wed, 15 Aug 2018 14:50:23 -0400 (EDT) Received: by mail-pl0-f72.google.com with SMTP id 33-v6so1124630plf.19 for ; Wed, 15 Aug 2018 11:50:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=r4uNaHCCdvFcps3AE+6NmK5z1ocDktKQXStql8Qqpwg=; b=EZa7vdGVpAUBGRSSpiSbsI49jyzmoSvyZiDamu48a1mkMGJH4uyO+vshjKVmQ6AgBE lej7sad4PQTQNXgRQA7GPlGy5SegdO0EdbFfgY7F6HhczWdDKX5YF12GP3nOA/MGNAYH k/DpK3DO6J+aXSc9eMXRCAsf6uxCGDopFIU52j4QM1j0aQsmqUKc7IC6gRt7Sqa3QBFA FF7b1hst/qE3ckkVfeBcUX4O8KAZXAqDtYkp/P5BgpFEOkUqmYO0Qv5fMoPJDhQWaxVQ +vTwvjo3RbMvJr3Mp/BXqHNuUYoD87FTh6mVbpPUwPkRm+Q7rrq5qAuQyhnhgI4m9cNx x/9w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: AOUpUlF7yFR5pBuvHFnrOq3aHSdwKFYmFoSqQT1JmJakBguE1LnG/+Co 7sPfXa3my3AHV4RMbXO1WkgYwqNk4a4CKm3WVeACfKjhSfCd5LciHQ1yumUS2Ff0ve7wCC89kAS 5EozGSpc5StiV0GwjbNhHB/4sKUwElnEXR79Rz4v3YQhFHALwJb6ruX+THSbmc17tpw== X-Received: by 2002:a63:66c7:: with SMTP id a190-v6mr26034185pgc.411.1534359023014; Wed, 15 Aug 2018 11:50:23 -0700 (PDT) X-Google-Smtp-Source: AA+uWPxb1v3r2nQsiDVLSXfheSGyPDoDSt1+xy6oCZGZytayhFzxMnuqG7rCrvvSU9GIxdp8yxPz X-Received: by 2002:a63:66c7:: with SMTP id a190-v6mr26034154pgc.411.1534359022340; Wed, 15 Aug 2018 11:50:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534359022; cv=none; d=google.com; s=arc-20160816; b=qjDYRddOgyilekAJtdYV1/SbHO4i0tmaORa7ixWpqoVBSVpzvgWjR4D+IuDbBnlVRC 0oYTzuH7SpYCmGt+9efVvTTq6/CKYCtpRHKZCpIV0Sr5k5GkBLNK2LW90veWXt8T6Itw SQcxw/BIWsu9YUKc2qTemOkmtCdJyj/awhA8PVfTUkGnd0NCHitVT5zH98XWxxCNZ14i JDsSxaYJrv6Xt3yunertPYiXdyYy9mBZHPvQcWmhBWoQtKxK3yPZdU23LSuJJJ82jlpj 2aYJyEzgDAsIYppR5xKnQAzJJlAKR45PSbvpNbultQFh8fDTwYBydqgfa6f1A3fbwDkj guzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=r4uNaHCCdvFcps3AE+6NmK5z1ocDktKQXStql8Qqpwg=; b=eNUVhZWBh2JqOrBbMwJLGojtCk/OAPVbKwXtRNL7iuFjNISrY9F1RcR3VU1VuwL1O7 zsh3MwxIlewTsdsfCbmtcty0slz3yjZIxAjMiAqCxW5veaWx/g2ZqWO2IIzf/xGznb9M SP7ioI2M7oJb05h3CL8lXTPyTVJI93i29i0+VOIcNrgJRCqN+at7f6CiiGuzgTSRlIvU 8qKqvzGJFMlMmpp7u+LXu8NWnOa2zEq6tkAoat82d32mIOaO079iFHpKX0hq3lp+adJS 0q0NZAeIegDyDaXXkX4t0s20KAR3eTAqc4N1asdOOvvB1wjsvTbR5DNF0VgIlM9s5n4j vKLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com. [115.124.30.132]) by mx.google.com with ESMTPS id y16-v6si19909316plr.469.2018.08.15.11.50.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Aug 2018 11:50:22 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) client-ip=115.124.30.132; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01f04452;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0T6gJtNa_1534358999; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T6gJtNa_1534358999) by smtp.aliyun-inc.com(127.0.0.1); Thu, 16 Aug 2018 02:50:08 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, kirill@shutemov.name, vbabka@suse.cz, akpm@linux-foundation.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, alexander.shishkin@linux.intel.com, jolsa@redhat.com, namhyung@kernel.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v8 PATCH 4/5] mm: unmap VM_HUGETLB mappings with optimized path Date: Thu, 16 Aug 2018 02:49:49 +0800 Message-Id: <1534358990-85530-5-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> References: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When unmapping VM_HUGETLB mappings, vm flags need to be updated. Since the vmas have been detached, so it sounds safe to update vm flags with read mmap_sem. Cc: Michal Hocko Cc: Vlastimil Babka Signed-off-by: Yang Shi --- mm/mmap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index e92f680..3b9f734 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2812,7 +2812,6 @@ static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, /* * Unmapping vmas, which have: - * VM_HUGETLB or * VM_PFNMAP or * uprobes * need get done with write mmap_sem held since they may update @@ -2821,7 +2820,7 @@ static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, for (vma = start_vma; vma && vma->vm_start < end; vma = vma->vm_next) { if ((vma->vm_file && has_uprobes(vma, vma->vm_start, vma->vm_end)) || - (vma->vm_flags & (VM_HUGETLB | VM_PFNMAP))) + (vma->vm_flags & VM_PFNMAP)) goto regular_path; } From patchwork Wed Aug 15 18:49:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10566751 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD3611669 for ; Wed, 15 Aug 2018 18:50:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B45542AF4A for ; Wed, 15 Aug 2018 18:50:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1AC22AF30; Wed, 15 Aug 2018 18:50:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B6CE2AF4F for ; Wed, 15 Aug 2018 18:50:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 403CA6B000A; Wed, 15 Aug 2018 14:50:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 38ACC6B000E; Wed, 15 Aug 2018 14:50:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 209E46B0010; Wed, 15 Aug 2018 14:50:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id B04936B000A for ; Wed, 15 Aug 2018 14:50:23 -0400 (EDT) Received: by mail-pg1-f199.google.com with SMTP id d12-v6so906695pgv.12 for ; Wed, 15 Aug 2018 11:50:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=g6+xXxH2+yhu7fnIgV5Oumf4ikagVDQR1nHSrpctNDk=; b=LwrHEb7zLSFOvAwG6lyOQUX0BCNaIv3VguAUhioOx7h4kWaibQybiSULMZgE5XyjDj e/rTl7G9DEA3qjDkEjlcRvtJF+ZKtbAxTq3xrQ53SokgY2GOjQBQ8Or0NX8fjUdeWMCI bhkPZ2rTrsK3RtiDEM9SZE5+Yq/VJyDcOmbZ6qSOMkVM7pdUB59iVWP1blWSzBPrcEV2 f8Nk91JN3jzM7hmTPDVfSFLoxqHrvYMOtC9Lt4g4kGYL9FJam6YO/z2Gje0AS0yujlXS F7v1CDb2iRqsVlGOmqsiVUW91KKbo/DMRUiuZWzaqJzVte2iSKXSJxv2tnZTrfy5Rqa+ FMAA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: AOUpUlFind4c50IR+VaIgaj/47jysej1VKN9AdRWryR/NIzK8zi/VHBN j+eA1XaHgnFW2a3uNja6wL0TTui5fMHdYDIDbWA9AkQRs/5aorj9FCltwFZw9nr6IVzy7Z6CS1N TFJ5cHVZWjOWclPO1V3TpMTokFsbVaY0pWYUeedclUbmmuM/kfhMkjMnASzTE7fNVEQ== X-Received: by 2002:a62:5613:: with SMTP id k19-v6mr28966877pfb.212.1534359023394; Wed, 15 Aug 2018 11:50:23 -0700 (PDT) X-Google-Smtp-Source: AA+uWPzQ61HBqsbL2qFCZj2ymlOkxixklVF1jIZnNiG/hSl5DTFwtVT79BvcaoOw8gP7PRGshYV4 X-Received: by 2002:a62:5613:: with SMTP id k19-v6mr28966835pfb.212.1534359022508; Wed, 15 Aug 2018 11:50:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534359022; cv=none; d=google.com; s=arc-20160816; b=jD5p+G6N5d8VWBAaox65OwHYBnmjCWB7FROooM/5A+fyTvGzYJvB8RYWsf0otaGrY4 zj2G4YruYJMtECzFSnez/e9C475z6d9HtrJzgMKQ8n//G+f/t5+XYW59STopuRBgxlRA Zuh0UI5Y6wGbqczJBFFIZlnQP++bd3RVP4z+ztgO1Ds+nMVJaFTp7A/n7CMOCXvIX/Dm kCfTM9TW+s6oZIPSZYZNWDn42ejqRDsZ2+9o8laHbk+ZQgrOXuhRozb8t6gc5ACrCAhf LW2Til/i8SsmHUJhZMytceQXJWcTXDcgZs6DaKgdD7SVKel6SrYvGDrlpvAnAM5bbNCn TWLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=g6+xXxH2+yhu7fnIgV5Oumf4ikagVDQR1nHSrpctNDk=; b=Fl+5dmS6JX5gf5UKEaeoS+xm1me8kfUosS5gSwBUdAW2eXZdi+OkYebKbYeqLB44mX mHHlMRashhINazOBmNNG/eI0xQzRg8kbKma4JXM4T6yW9J8gqe8KluRpMaP994vHhEUh YfB3ehjL75ZlydWXyJotgLu7qx7X4YKZEGOk6pzL5xCLf08mxPjGeSW3wJKlTEk2vUUr XBE9IFBDwJo6x+2pPTKOigld8KTIRGHBeusBiT1+KgN/mLD/p51jGlHsMeaF2ZLjQzwY 1hmxPBalHmQjUk68l4waLCsoDozwDAlKZtazcHf3qkoNlpL7PGV6OSx+xQt3PZhJTIl6 D/pg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com. [115.124.30.130]) by mx.google.com with ESMTPS id a18-v6si20228338plm.122.2018.08.15.11.50.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Aug 2018 11:50:22 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.130 as permitted sender) client-ip=115.124.30.130; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.130 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R391e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01353;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=15;SR=0;TI=SMTPD_---0T6gJtNa_1534358999; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T6gJtNa_1534358999) by smtp.aliyun-inc.com(127.0.0.1); Thu, 16 Aug 2018 02:50:08 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, kirill@shutemov.name, vbabka@suse.cz, akpm@linux-foundation.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org, alexander.shishkin@linux.intel.com, jolsa@redhat.com, namhyung@kernel.org Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v8 PATCH 5/5] mm: unmap VM_PFNMAP mappings with optimized path Date: Thu, 16 Aug 2018 02:49:50 +0800 Message-Id: <1534358990-85530-6-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> References: <1534358990-85530-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When unmapping VM_PFNMAP mappings, vm flags need to be updated. Since the vmas have been detached, so it sounds safe to update vm flags with read mmap_sem. Cc: Michal Hocko Cc: Vlastimil Babka Signed-off-by: Yang Shi --- mm/mmap.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 3b9f734..0a9960d 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2811,16 +2811,13 @@ static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, } /* - * Unmapping vmas, which have: - * VM_PFNMAP or - * uprobes - * need get done with write mmap_sem held since they may update - * vm_flags. Deal with such mappings with regular do_munmap() call. + * Unmapping vmas, which have uprobes need get done with write + * mmap_sem held since they may update vm_flags. Deal with such + * mappings with regular do_munmap() call. */ for (vma = start_vma; vma && vma->vm_start < end; vma = vma->vm_next) { - if ((vma->vm_file && - has_uprobes(vma, vma->vm_start, vma->vm_end)) || - (vma->vm_flags & VM_PFNMAP)) + if (vma->vm_file && + has_uprobes(vma, vma->vm_start, vma->vm_end)) goto regular_path; }