From patchwork Tue Sep 11 20:58:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10596315 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D802112B for ; Tue, 11 Sep 2018 20:58:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5E4D129DC1 for ; Tue, 11 Sep 2018 20:58:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5282D29DCF; Tue, 11 Sep 2018 20:58:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 20F2929DC1 for ; Tue, 11 Sep 2018 20:58:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CDEAC8E0006; Tue, 11 Sep 2018 16:58:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C671D8E0001; Tue, 11 Sep 2018 16:58:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B03B58E0006; Tue, 11 Sep 2018 16:58:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 6FA6B8E0001 for ; Tue, 11 Sep 2018 16:58:36 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id d22-v6so13445272pfn.3 for ; Tue, 11 Sep 2018 13:58:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=mdrLuO88nuRuyNRN/IyrXfz40XhoiZdKXx4T6zLnk3k=; b=Rzn7QlK7NtfTl9vcKOaKGbfpCrd3tAQmK6NZDHIhr9BOlMKolSnbkmPyGhAWaUSw3A MAx9uIopjw2JFUES8pBqCzJPab4sddNGZ8Zv43eP0fgVNJDb/ksjtAFy26C4stR4UX8X gnKDzcdZIChnCXanpU2GVvtg585GpwZnpP5MZ0Y81zqgcmrTx467W6JL46MbTAN72LWq rfNY5I4jP0LnUSZyW5ATpP24WblLkhlBHko61w0m9fAmSZCQXfQKlwy5PufceZ715SFQ 2shlPwt9XXtY796R2RETe1lwAZjg0nWDnpu+sxbV9Z0kfiaRPGlP0X5HFbc/lpe2NmEo ukTA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: APzg51Bpz1+yxct7UM+dQGb1O3b2UuSThpTHs3J2Bq96J0zSWSm+hXK7 EO1BXcJ76AN20WuAQYzKjm64aJ1sU4rGY1O4AbenmkkVMwlCD+Nzax1G4Wx3GaaieZAjS/VLr4o ivgFCzo2NM9fQ4DhJgoGDQIgEFI/InzO1RckfnRKhOsMuBmE8wSML+Rd0k01T9qFQYQ== X-Received: by 2002:a17:902:7447:: with SMTP id e7-v6mr29170061plt.186.1536699516128; Tue, 11 Sep 2018 13:58:36 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZf7IrY2NEX2jqNNGZKJn2GFJJs9Tw8AMrHEBOcauq2uEHrK+dEHxdrjV09Z9feq0OPNdJg X-Received: by 2002:a17:902:7447:: with SMTP id e7-v6mr29169998plt.186.1536699514817; Tue, 11 Sep 2018 13:58:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536699514; cv=none; d=google.com; s=arc-20160816; b=V9Um+5NAZTcJiWgF7HNHH8XAj4fJBXSzKvAVrjUOvBzkTRhG4QrdG1zTFcdOdeYprg GRjU3Es0RmaNLIUqYx2uot67HhgZT0O5X55ovaxhSCxL80Wmvq9pwNOIDFESnTm5kE/L ksQi4ghgEjCImMs4jMT/MgrnRbQtfqwF/vHWqn5T1QSnHBG24F17/vk7Qv4BOk2hmJjN KsGAq9EiKLHVyh4LKULXz5SR2s70h8skzi8hN1IfHCjq1yo/f7pvLr8R2/qaXa8ceo3B vN4vJvK3RMVFU0p+3Y8bnoh8hYbIumFqZ1esCXzs/IHutOzt5V5aWbuSj8SqRRRIXc85 VOqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=mdrLuO88nuRuyNRN/IyrXfz40XhoiZdKXx4T6zLnk3k=; b=tntT37WLaBuJApXl3VgXype05Cxkvlom4r9E5SjyEsgKNsQRrBYxw2VRD8AeRW0NQl qRRF6xHEdYppzDkWdZ2H+LVfq42qvleoH+6NrtsvhjEykEixYzU5sCJn0jVLtQSyukPw EbW6r7N1WWR2MVCVUXD67thSswbHOA7ASjuP7qM/6M/83UiqhK/+C2v+EzJlnHdv0wC4 2Br1Uzc+y96nbZgAplq+ZjIEOaaQ+iStmbRxdVorzhHi8uPyJwmUuc7sg+wMBdzWXAQ1 mlqGg0rl07fCbl2NAjv3fTc1tOcv15ocWKSmVtKlfdi8XrOJDXhTuq4uGELgS7/Ievi0 Oclw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com. [115.124.30.132]) by mx.google.com with ESMTPS id 1-v6si20025473plv.28.2018.09.11.13.58.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 13:58:34 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) client-ip=115.124.30.132; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04400;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0T8Sg6Yy_1536699501; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T8Sg6Yy_1536699501) by smtp.aliyun-inc.com(127.0.0.1); Wed, 12 Sep 2018 04:58:29 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, vbabka@suse.cz, akpm@linux-foundation.org, dave.hansen@intel.com, oleg@redhat.com, srikar@linux.vnet.ibm.com Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v9 PATCH 1/4] mm: refactor do_munmap() to extract the common part Date: Wed, 12 Sep 2018 04:58:10 +0800 Message-Id: <1536699493-69195-2-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> References: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Introduces three new helper functions: * addr_ok() * munmap_lookup_vma() * munlock_vmas() They will be used by do_munmap() and the new do_munmap with zapping large mapping early in the later patch. There is no functional change, just code refactor. Reviewed-by: Laurent Dufour Acked-by: Vlastimil Babka Signed-off-by: Yang Shi --- mm/mmap.c | 106 +++++++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 74 insertions(+), 32 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 5f2b2b1..b7092b4 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2682,35 +2682,42 @@ int split_vma(struct mm_struct *mm, struct vm_area_struct *vma, return __split_vma(mm, vma, addr, new_below); } -/* Munmap is split into 2 main parts -- this part which finds - * what needs doing, and the areas themselves, which do the - * work. This now handles partial unmappings. - * Jeremy Fitzhardinge - */ -int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, - struct list_head *uf) +static inline bool addr_ok(unsigned long start, size_t len) { - unsigned long end; - struct vm_area_struct *vma, *prev, *last; - if ((offset_in_page(start)) || start > TASK_SIZE || len > TASK_SIZE-start) - return -EINVAL; + return false; - len = PAGE_ALIGN(len); - if (len == 0) - return -EINVAL; + if (PAGE_ALIGN(len) == 0) + return false; + + return true; +} + +/* + * munmap_lookup_vma: find the first overlap vma and split overlap vmas. + * @mm: mm_struct + * @start: start address + * @end: end address + * + * Return: %NULL if no VMA overlaps this range. An ERR_PTR if an + * overlapping VMA could not be split. Otherwise a pointer to the first + * VMA which overlaps the range. + */ +static struct vm_area_struct *munmap_lookup_vma(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + struct vm_area_struct *vma, *prev, *last; /* Find the first overlapping VMA */ vma = find_vma(mm, start); if (!vma) - return 0; - prev = vma->vm_prev; - /* we have start < vma->vm_end */ + return NULL; + /* we have start < vma->vm_end */ /* if it doesn't overlap, we have nothing.. */ - end = start + len; if (vma->vm_start >= end) - return 0; + return NULL; + prev = vma->vm_prev; /* * If we need to split any vma, do it now to save pain later. @@ -2728,11 +2735,11 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, * its limit temporarily, to help free resources as expected. */ if (end < vma->vm_end && mm->map_count >= sysctl_max_map_count) - return -ENOMEM; + return ERR_PTR(-ENOMEM); error = __split_vma(mm, vma, start, 0); if (error) - return error; + return ERR_PTR(error); prev = vma; } @@ -2741,10 +2748,53 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, if (last && end > last->vm_start) { int error = __split_vma(mm, last, end, 1); if (error) - return error; + return ERR_PTR(error); } vma = prev ? prev->vm_next : mm->mmap; + return vma; +} + +static inline void munlock_vmas(struct vm_area_struct *vma, + unsigned long end) +{ + struct mm_struct *mm = vma->vm_mm; + + while (vma && vma->vm_start < end) { + if (vma->vm_flags & VM_LOCKED) { + mm->locked_vm -= vma_pages(vma); + munlock_vma_pages_all(vma); + } + vma = vma->vm_next; + } +} + +/* Munmap is split into 2 main parts -- this part which finds + * what needs doing, and the areas themselves, which do the + * work. This now handles partial unmappings. + * Jeremy Fitzhardinge + */ +int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, + struct list_head *uf) +{ + unsigned long end; + struct vm_area_struct *vma, *prev; + + if (!addr_ok(start, len)) + return -EINVAL; + + len = PAGE_ALIGN(len); + + end = start + len; + + vma = munmap_lookup_vma(mm, start, end); + if (!vma) + return 0; + if (IS_ERR(vma)) + return PTR_ERR(vma); + + prev = vma->vm_prev; + if (unlikely(uf)) { /* * If userfaultfd_unmap_prep returns an error the vmas @@ -2763,16 +2813,8 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, /* * unlock any mlock()ed ranges before detaching vmas */ - if (mm->locked_vm) { - struct vm_area_struct *tmp = vma; - while (tmp && tmp->vm_start < end) { - if (tmp->vm_flags & VM_LOCKED) { - mm->locked_vm -= vma_pages(tmp); - munlock_vma_pages_all(tmp); - } - tmp = tmp->vm_next; - } - } + if (mm->locked_vm) + munlock_vmas(vma, end); /* * Remove the vma's, and unmap the actual pages From patchwork Tue Sep 11 20:58:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10596319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01C19109C for ; Tue, 11 Sep 2018 20:58:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E67BA29DCA for ; Tue, 11 Sep 2018 20:58:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA88629DD2; Tue, 11 Sep 2018 20:58:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1760429DCA for ; Tue, 11 Sep 2018 20:58:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E9338E000A; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6712B8E0001; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 539898E0009; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 053AD8E0001 for ; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) Received: by mail-pl1-f200.google.com with SMTP id h4-v6so11943226pls.17 for ; Tue, 11 Sep 2018 13:58:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=3srAs/M2r1mxKZgukfo+gGB9/UnQPUYb7w4NjgAmhoY=; b=gHqRY1co4MDNFrYY3i7SuT7J2r1Vv/d0TMSzZ3mXwi1al4pBczAjfcNCQtd2IZ49Js 9Jr/k2pCkjfSJlIVDDbYYzYvZ8HzmcHhOsba9nN/KPut6ip8OtUfn9AIvN2yqCv7nNVS QgGhPkQqIzM0oJMWqdjO/T2rK62UirK8eqQqWYZDh1caTqIWng7eoUhs1TXRWfZ1Wm1J u0RhRY4oPW/fWwDWaVBuV26/Pijw2tniQJCbarc2KxkZDma8DIHqXFG2nyl9o1f5GLpq YZXNe9nmI+2OiKBrsQMNYJDu0KDGLctlhcBqJ/PFHYwWQekQEbVJcVyVBvwhHEEpsMWd zkQw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: APzg51Dp4o5vRT0H/1MIAW63wkBkhMV4kyBhcw8/g6MUmqBQmVoUyS47 3odV+EPV8nWUdI+AXIQZpPcUSGw3y2eUMfNczjds4csXwjcH3Tu4uvs/isJbPGtdGJfmh3CMHjA HfxhPBAO2jXgQVFbdd7oKcoD1g0BZph0HTessCJPYGNOYoA5QnOhPxPElvkcKtvVzkQ== X-Received: by 2002:a17:902:8506:: with SMTP id bj6-v6mr28594830plb.210.1536699525587; Tue, 11 Sep 2018 13:58:45 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZ4teH6LEmrZveNrp83iPtGFY+HcTH86MmPTZEFPMu7rHA3vOfd4a1FPm3t4UsZAOcMDKQg X-Received: by 2002:a17:902:8506:: with SMTP id bj6-v6mr28594787plb.210.1536699524546; Tue, 11 Sep 2018 13:58:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536699524; cv=none; d=google.com; s=arc-20160816; b=u1yYuoyaCFHeyhC/mDgigGu2gPUsRtAO4yRBVton1G7Tyqijj77SI3Xn7aRfMkP1oi JZyzZsMbDQw1y3l43oVVhpEVhV2POgOcUnOK53I7Vb5eVvzlJtQc0nwGiDha7d0XmuMj tIv91uK9mdL3jKzosv7/Nrwokof+RAcwANpdREKKGvDfCD+KnwNbXdU5XKxDF6Z7nu96 1wGrIi3tOcGl6KnuEtHphJh+94llKwasA5mI8cdl0RX9pxjtHNtBasr+8GZ2VNRg7uRo 9vt8skMy/TU5TOhA13EsNUwNTa5dV+3EVF3dt8xp/FD1NVo6xtK8ZMYG70d+Ev/ti6zB +Iyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=3srAs/M2r1mxKZgukfo+gGB9/UnQPUYb7w4NjgAmhoY=; b=AXCjRvT04iMKcZ9/Jxbk/Y6aaElLseLJ60FC+4gWqpxa330WERg09WAZYdKuJSwLsA 4CimrS54ASghOuIZzqMSCoNcpF3xNOWOgYwSnJ6hke0E7wnrK8dYVEmI6xi5yBp3maD3 2bYrxcln/l7UVA/QicCBAHSHRReNdWSYowdbItuNTtyPw3puJCPrISLbC0BQhUe97r0p h8GGDKbg/xtnWxldsFISyQwgF+rFoygJLxZ5QrD2MaGkuD3hxrrJ8bMeHOw+catGCh1l dcdbIljXqeSJPkYc10To9gV5KbdVYKI0QIqF4qbVbqYtv8qrpQe/85xSf6OraEd3Zg3U sG9w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com. [115.124.30.131]) by mx.google.com with ESMTPS id d1-v6si21724470pld.322.2018.09.11.13.58.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 13:58:44 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) client-ip=115.124.30.131; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R191e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01353;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0T8Sg6Yy_1536699501; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T8Sg6Yy_1536699501) by smtp.aliyun-inc.com(127.0.0.1); Wed, 12 Sep 2018 04:58:29 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, vbabka@suse.cz, akpm@linux-foundation.org, dave.hansen@intel.com, oleg@redhat.com, srikar@linux.vnet.ibm.com Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v9 PATCH 2/4] mm: mmap: zap pages with read mmap_sem in munmap Date: Wed, 12 Sep 2018 04:58:11 +0800 Message-Id: <1536699493-69195-3-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> References: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When running some mmap/munmap scalability tests with large memory (i.e. > 300GB), the below hung task issue may happen occasionally. INFO: task ps:14018 blocked for more than 120 seconds. Tainted: G E 4.9.79-009.ali3000.alios7.x86_64 #1 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. ps D 0 14018 1 0x00000004 ffff885582f84000 ffff885e8682f000 ffff880972943000 ffff885ebf499bc0 ffff8828ee120000 ffffc900349bfca8 ffffffff817154d0 0000000000000040 00ffffff812f872a ffff885ebf499bc0 024000d000948300 ffff880972943000 Call Trace: [] ? __schedule+0x250/0x730 [] schedule+0x36/0x80 [] rwsem_down_read_failed+0xf0/0x150 [] call_rwsem_down_read_failed+0x18/0x30 [] down_read+0x20/0x40 [] proc_pid_cmdline_read+0xd9/0x4e0 [] ? do_filp_open+0xa5/0x100 [] __vfs_read+0x37/0x150 [] ? security_file_permission+0x9b/0xc0 [] vfs_read+0x96/0x130 [] SyS_read+0x55/0xc0 [] entry_SYSCALL_64_fastpath+0x1a/0xc5 It is because munmap holds mmap_sem exclusively from very beginning to all the way down to the end, and doesn't release it in the middle. When unmapping large mapping, it may take long time (take ~18 seconds to unmap 320GB mapping with every single page mapped on an idle machine). Zapping pages is the most time consuming part, according to the suggestion from Michal Hocko [1], zapping pages can be done with holding read mmap_sem, like what MADV_DONTNEED does. Then re-acquire write mmap_sem to cleanup vmas. But, some part may need write mmap_sem, for example, vma splitting. So, the design is as follows: acquire write mmap_sem lookup vmas (find and split vmas) deal with special mappings detach vmas downgrade_write zap pages free page tables release mmap_sem The vm events with read mmap_sem may come in during page zapping, but since vmas have been detached before, they, i.e. page fault, gup, etc, will not be able to find valid vma, then just return SIGSEGV or -EFAULT as expected. If the vma has VM_HUGETLB | VM_PFNMAP, they are considered as special mappings. They will be handled by falling back to regular do_munmap() with exclusive mmap_sem held in this patch since they may update vm flags. But, with the "detach vmas first" approach, the vmas have been detached when vm flags are updated, so it sounds safe to update vm flags with read mmap_sem for this specific case. So, VM_HUGETLB and VM_PFNMAP will be handled by using the optimized path in the following separate patches for bisectable sake. Unmapping uprobe areas may need update mm flags (MMF_RECALC_UPROBES). However it is fine to have false-positive MMF_RECALC_UPROBES according to uprobes developer. So, uprobe unmap will not be handled by the regular path. With the "detach vmas first" approach we don't have to re-acquire mmap_sem again to clean up vmas to avoid race window which might get the address space changed since downgrade_write() doesn't release the lock to lead regression, which simply downgrades to read lock. And, since the lock acquire/release cost is managed to the minimum and almost as same as before, the optimization could be extended to any size of mapping without incurring significant penalty to small mappings. For the time being, just do this in munmap syscall path. Other vm_munmap() or do_munmap() call sites (i.e mmap, mremap, etc) remain intact due to some implementation difficulties since they acquire write mmap_sem from very beginning and hold it until the end, do_munmap() might be called in the middle. But, the optimized do_munmap would like to be called without mmap_sem held so that we can do the optimization. So, if we want to do the similar optimization for mmap/mremap path, I'm afraid we would have to redesign them. mremap might be called on very large area depending on the usecases, the optimization to it will be considered in the future. With the patches, exclusive mmap_sem hold time when munmap a 80GB address space on a machine with 32 cores of E5-2680 @ 2.70GHz dropped to us level from second. munmap_test-15002 [008] 594.380138: funcgraph_entry: | vm_munmap_zap_rlock() { munmap_test-15002 [008] 594.380146: funcgraph_entry: !2485684 us | unmap_region(); munmap_test-15002 [008] 596.865836: funcgraph_exit: !2485692 us | } Here the excution time of unmap_region() is used to evaluate the time of holding read mmap_sem, then the remaining time is used with holding exclusive lock. [1] https://lwn.net/Articles/753269/ Suggested-by: Michal Hocko Suggested-by: Kirill A. Shutemov Cc: Matthew Wilcox Cc: Laurent Dufour Cc: Vlastimil Babka Cc: Andrew Morton Signed-off-by: Yang Shi --- mm/mmap.c | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index b7092b4..937d2f2 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2769,6 +2769,89 @@ static inline void munlock_vmas(struct vm_area_struct *vma, } } +/* + * Zap pages with read mmap_sem held + * + * uf is the list for userfaultfd + */ +static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, + size_t len, struct list_head *uf) +{ + unsigned long end; + struct vm_area_struct *start_vma, *prev, *vma; + int ret = 0; + + if (!addr_ok(start, len)) + return -EINVAL; + + len = PAGE_ALIGN(len); + + end = start + len; + + /* + * Need write mmap_sem to split vmas and detach vmas + * splitting vma up-front to save PITA to clean if it is failed + */ + if (down_write_killable(&mm->mmap_sem)) + return -EINTR; + + start_vma = munmap_lookup_vma(mm, start, end); + if (!start_vma) + goto out; + if (IS_ERR(start_vma)) { + ret = PTR_ERR(start_vma); + goto out; + } + + prev = start_vma->vm_prev; + + if (unlikely(uf)) { + ret = userfaultfd_unmap_prep(start_vma, start, end, uf); + if (ret) + goto out; + } + + /* + * Unmapping vmas, which have VM_HUGETLB or VM_PFNMAP + * need get done with write mmap_sem held since they may update + * vm_flags. Deal with such mappings with regular do_munmap() call. + */ + for (vma = start_vma; vma && vma->vm_start < end; vma = vma->vm_next) { + if (vma->vm_flags & (VM_HUGETLB | VM_PFNMAP)) + goto regular_path; + } + + /* Handle mlocked vmas */ + if (mm->locked_vm) + munlock_vmas(start_vma, end); + + /* Detach vmas from rbtree */ + detach_vmas_to_be_unmapped(mm, start_vma, prev, end); + + /* + * mpx unmap need to be handled with write mmap_sem. It is safe to + * deal with it before unmap_region(). + */ + arch_unmap(mm, start_vma, start, end); + + downgrade_write(&mm->mmap_sem); + + /* Zap mappings with read mmap_sem */ + unmap_region(mm, start_vma, prev, start, end); + + remove_vma_list(mm, start_vma); + up_read(&mm->mmap_sem); + + return 0; + +regular_path: + ret = do_munmap(mm, start, len, uf); + +out: + up_write(&mm->mmap_sem); + return ret; +} + /* Munmap is split into 2 main parts -- this part which finds * what needs doing, and the areas themselves, which do the * work. This now handles partial unmappings. @@ -2830,6 +2913,17 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, return 0; } +static int vm_munmap_zap_rlock(unsigned long start, size_t len) +{ + int ret; + struct mm_struct *mm = current->mm; + LIST_HEAD(uf); + + ret = do_munmap_zap_rlock(mm, start, len, &uf); + userfaultfd_unmap_complete(mm, &uf); + return ret; +} + int vm_munmap(unsigned long start, size_t len) { int ret; @@ -2849,10 +2943,9 @@ int vm_munmap(unsigned long start, size_t len) SYSCALL_DEFINE2(munmap, unsigned long, addr, size_t, len) { profile_munmap(addr); - return vm_munmap(addr, len); + return vm_munmap_zap_rlock(addr, len); } - /* * Emulation of deprecated remap_file_pages() syscall. */ From patchwork Tue Sep 11 20:58:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10596317 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B838B112B for ; Tue, 11 Sep 2018 20:58:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8F6A29DCA for ; Tue, 11 Sep 2018 20:58:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9D19429DD2; Tue, 11 Sep 2018 20:58:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E06629DCA for ; Tue, 11 Sep 2018 20:58:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11BC28E0007; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0DAFA8E0008; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAE3B8E0007; Tue, 11 Sep 2018 16:58:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id A9B788E0001 for ; Tue, 11 Sep 2018 16:58:45 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id v9-v6so13446710pff.4 for ; Tue, 11 Sep 2018 13:58:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=BKnBaTX8iefsbR3W8x30Vgm3fDP89JaoPoxTGit5Fvo=; b=pKhCYQ2lJ2vdLKX+uKnPR59Og6TE5f5RIoNFn/cebTs1zNG5uNr12TcniCh7GvRu6v tvq6KtL00kx6G2d+fZ7PT6lCk5uYiKL0ebf7GBDw2fHj2jrNpHV/r1mUopJ5ZZntXgcf w/NiGjRr+aWn84p+mYAVTWmo2ihvWIoRn5+NRKOTpQy90ydkEZPJ+X2g+4X0ktF5snvc mjEWNMN+TykeEQYGjjac5Ein0Ar+RA6Nj+CSnQa/BAnvK8qXUrqq0XgvREz38c+5d8/b 9Pmh/WLT+DVSf0NmmC109WhVcem0a3CtqHp/3jJBbbG+Ab+fQB9QWOWT/2qG3AlAMD3d Lggw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: APzg51CLNEkhziPP2EYh7iQjc3ubRvbNS0AcEckSzI39uZvpDic/IcsQ 9GzVP/SBUjF2eW0D7e2ASJH/wjvkvjJiwCJ9JJZi2adMg6+yk7UgMkHE4DJ55ytrJva9S30E2zf vwfxsv7fzQs9bTv456VtiTsvOJ/laqAqI3YpG7K2eBvHZ7ntOioS51yvv8GNRpOfXgQ== X-Received: by 2002:a63:28c7:: with SMTP id o190-v6mr30199634pgo.84.1536699525363; Tue, 11 Sep 2018 13:58:45 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbTTb+SFYXU6OJW/bn7vNF6ftWYjGUyG/8iyZ6dn5S70oyDI8slzHeZ0w4Ke+agwKJyz+mE X-Received: by 2002:a63:28c7:: with SMTP id o190-v6mr30199598pgo.84.1536699524506; Tue, 11 Sep 2018 13:58:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536699524; cv=none; d=google.com; s=arc-20160816; b=dipMsI8qGPs7I9BaIaqM4arGKxsRSh0IgozMDa5NZrbAac8za/6lrES/u6GqLlhrxb 5LfFwaPg7XVPUGYFuxQUV0qJZuvP1miQoztL54ffBjKzBMe2x5eu36uhJ5N/O0ezdSlR Cm3utAWMFOBEqQcekEfw4mUhH4vWfa3cheCaD84onRTXtPuGH0syQHPWt/7l47Mu39Ub cT2f1aaXSxhUlarMg2qeSZJRmCtLvQl+X/DBfwx01U+m19CzY0jKNw8EiUX0tFLFhewK t+V0KIi0GCbGfzNW+t3IybmS5u1Ox5tFnViEpreM6PKAll6A2vttYfHG3AoTnYCl35WP vZlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=BKnBaTX8iefsbR3W8x30Vgm3fDP89JaoPoxTGit5Fvo=; b=bGC+FFiQqJqHrsyOKWoSzWONBBYiGa3hlkcmVIdkkag3J/BZdMDa8zOnbkvYGCC5IS ZAR4QIqjKh+oDZ8l4T/ocC37PeJ5m1pp5ZXRjwwUrU+q6+JdQ1f5bMTHBPYdHzyRw0zr fKxs0CLG3zmC+3T4MbF5Ex8ea5PGzdUyKqPG20lZEBCj+gpHiHAuJkq/lsZaW0DCDpXJ LOkko7YT5k2bUI2X0QwFST7d5Kkfri+9yggJZgcnOhTvCOqBOZIrVC9KcyWA3AbxlIc5 OBMLh9CUJtKDqB+vW5T0RO+xUosNLXUYpA/bUncwCG0mWm5ZhajhoxN56J/SJNbJXRNI zi5w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com. [115.124.30.131]) by mx.google.com with ESMTPS id ca2-v6si22871796plb.305.2018.09.11.13.58.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 13:58:44 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) client-ip=115.124.30.131; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e01451;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0T8Sg6Yy_1536699501; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T8Sg6Yy_1536699501) by smtp.aliyun-inc.com(127.0.0.1); Wed, 12 Sep 2018 04:58:30 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, vbabka@suse.cz, akpm@linux-foundation.org, dave.hansen@intel.com, oleg@redhat.com, srikar@linux.vnet.ibm.com Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v9 PATCH 3/4] mm: unmap VM_HUGETLB mappings with optimized path Date: Wed, 12 Sep 2018 04:58:12 +0800 Message-Id: <1536699493-69195-4-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> References: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When unmapping VM_HUGETLB mappings, vm flags need to be updated. Since the vmas have been detached, so it sounds safe to update vm flags with read mmap_sem. Cc: Michal Hocko Cc: Vlastimil Babka Signed-off-by: Yang Shi --- mm/mmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 937d2f2..086f8b5 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2812,12 +2812,12 @@ static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, } /* - * Unmapping vmas, which have VM_HUGETLB or VM_PFNMAP + * Unmapping vmas, which have VM_PFNMAP * need get done with write mmap_sem held since they may update * vm_flags. Deal with such mappings with regular do_munmap() call. */ for (vma = start_vma; vma && vma->vm_start < end; vma = vma->vm_next) { - if (vma->vm_flags & (VM_HUGETLB | VM_PFNMAP)) + if (vma->vm_flags & VM_PFNMAP) goto regular_path; } From patchwork Tue Sep 11 20:58:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 10596321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DC190112B for ; Tue, 11 Sep 2018 20:58:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC5C829DCA for ; Tue, 11 Sep 2018 20:58:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BEF8829DD2; Tue, 11 Sep 2018 20:58:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5080029DCA for ; Tue, 11 Sep 2018 20:58:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB9698E0008; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BF3B58E0001; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 988288E000B; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 33CD08E0008 for ; Tue, 11 Sep 2018 16:58:46 -0400 (EDT) Received: by mail-pg1-f197.google.com with SMTP id g9-v6so12895209pgc.16 for ; Tue, 11 Sep 2018 13:58:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=ZAgZ7q1tZ0obqE6duMZeoXRUPiPQC81VXykkld63HEs=; b=TXKq54V4g++GEH4u0HQzumQjrE2YupKbdOfVQJ3XY9LNYkW38tbQHxw38sEhFyXS4+ acve0GXGu+gtEjfx1mvn2VjlLRikXRYGgwJvDfajWMCREyhQJVtzLBCFw/jCZMjmW63m TtN1xOzUo49fl+Rdmx5/SJ8f4EM9XJQVoRMOZ0knDoS0puzf7pGijIUI4Z5r/0jjONkX GLb5uU6i4hNON+gLHt4rAvXpgad0Rtjxhz63bCJtpQBMY/4EeSqtn7FjkM+o5MMdfmdx SQzDD6xx5+HEJyarW5P0+q5YBS8SBDyzAq6OFDzCI7f4Wtqkkbo2mepsXIw6JY7EzCDv zPnA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Gm-Message-State: APzg51A67B+7wbn8jeE7wy5Br1bKf8Llfqm6wEyoADAhqV6AnuqBGtPw I5IqMUsoeWEAMrSBKv6Iy918oqc5rP37aIok0PBXOQE6uAhbFSCESw6fE3wJlA6H4ZoNDMZuSyN Hew7LQvrO+QNEgWUinHY6sFQTgmBFzR1LJzu0ApSVpVDbOY6eLMC6QHoCp+Cudyr3dw== X-Received: by 2002:a63:a619:: with SMTP id t25-v6mr30092742pge.288.1536699525899; Tue, 11 Sep 2018 13:58:45 -0700 (PDT) X-Google-Smtp-Source: ANB0Vda0eY7E7POgU0RfBS82w3w8GlN33zWdmofnOKOYg9v7D0OpD0drLx+p3xTCTnS66+85rGqS X-Received: by 2002:a63:a619:: with SMTP id t25-v6mr30092701pge.288.1536699524918; Tue, 11 Sep 2018 13:58:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536699524; cv=none; d=google.com; s=arc-20160816; b=UZjtqLiBfztgDbJdw5tx+yx8v3qO6iVA0AwuyJmzk3fcqqBpMSt3uEkBWA9CR22Hma 6cc5ZsSzM73Csxm1E1uvZFvk/ScPavGCOSnJMuJifoZm5RIzB+w8cMG1ehmM2QS/uaVN hk6RllgOhzE6es0z5VWoZ5tFGgPD59Z3a8NQrr9yaULhAYdb8s+C/cPLNsOFgVSbCJvl 3sLHbbz1RDdfJ8HFDUBESrkmVy91hHs90Sdxpr15rjLdzQUwDsIdOE1uzskzrTGa06OG Y84XPVcHGjlt5y0f7/l8cDz6rXIRa5N6gyMIBEXLo+wmduic6XliLCMrXm40acaCG46H vPIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=ZAgZ7q1tZ0obqE6duMZeoXRUPiPQC81VXykkld63HEs=; b=THK6BXk+lM2H7HHKfj1lx5cKPLtN0GVdzkUtmcw+tZLI8YoRpcbZ7HvEpkilwgX8QY BtLkq3Jaky1ZXLIiY/GTdfH+pHhJmUpk6op8boK5iDUt1gGmVvirfQwNK/EzAZFe+pmu 2o758IK9OtmIK/si4CtpRt5ad84Unzl3m5caz7Ug5DqOB9z41WKUIV1WQJUekAze5sr2 pHQC09yHFDkzGtlORg8pTIFAoi7UqT2OFvIJ0a1DEszvl6DK8s7uChRKuZ56Dy1+78jM mALKHD3Qss0am1B9Mh84XJalGqDyZtlMtvC4u4yRhMFMgw67mAsrFP6FSwpFjA08dSAF dhlw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from out30-131.freemail.mail.aliyun.com (out30-131.freemail.mail.aliyun.com. [115.124.30.131]) by mx.google.com with ESMTPS id e21-v6si20832405pgb.131.2018.09.11.13.58.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Sep 2018 13:58:44 -0700 (PDT) Received-SPF: pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) client-ip=115.124.30.131; Authentication-Results: mx.google.com; spf=pass (google.com: domain of yang.shi@linux.alibaba.com designates 115.124.30.131 as permitted sender) smtp.mailfrom=yang.shi@linux.alibaba.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R541e4;CH=green;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01f04452;MF=yang.shi@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0T8Sg6Yy_1536699501; Received: from e19h19392.et15sqa.tbsite.net(mailfrom:yang.shi@linux.alibaba.com fp:SMTPD_---0T8Sg6Yy_1536699501) by smtp.aliyun-inc.com(127.0.0.1); Wed, 12 Sep 2018 04:58:30 +0800 From: Yang Shi To: mhocko@kernel.org, willy@infradead.org, ldufour@linux.vnet.ibm.com, vbabka@suse.cz, akpm@linux-foundation.org, dave.hansen@intel.com, oleg@redhat.com, srikar@linux.vnet.ibm.com Cc: yang.shi@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC v9 PATCH 4/4] mm: unmap VM_PFNMAP mappings with optimized path Date: Wed, 12 Sep 2018 04:58:13 +0800 Message-Id: <1536699493-69195-5-git-send-email-yang.shi@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> References: <1536699493-69195-1-git-send-email-yang.shi@linux.alibaba.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP When unmapping VM_PFNMAP mappings, vm flags need to be updated. Since the vmas have been detached, so it sounds safe to update vm flags with read mmap_sem. Cc: Michal Hocko Cc: Vlastimil Babka Signed-off-by: Yang Shi --- mm/mmap.c | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 086f8b5..0b6b231 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2778,7 +2778,7 @@ static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, size_t len, struct list_head *uf) { unsigned long end; - struct vm_area_struct *start_vma, *prev, *vma; + struct vm_area_struct *start_vma, *prev; int ret = 0; if (!addr_ok(start, len)) @@ -2811,16 +2811,6 @@ static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, goto out; } - /* - * Unmapping vmas, which have VM_PFNMAP - * need get done with write mmap_sem held since they may update - * vm_flags. Deal with such mappings with regular do_munmap() call. - */ - for (vma = start_vma; vma && vma->vm_start < end; vma = vma->vm_next) { - if (vma->vm_flags & VM_PFNMAP) - goto regular_path; - } - /* Handle mlocked vmas */ if (mm->locked_vm) munlock_vmas(start_vma, end); @@ -2844,9 +2834,6 @@ static int do_munmap_zap_rlock(struct mm_struct *mm, unsigned long start, return 0; -regular_path: - ret = do_munmap(mm, start, len, uf); - out: up_write(&mm->mmap_sem); return ret;