From patchwork Mon Feb 24 20:30:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michel Lespinasse X-Patchwork-Id: 11401531 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE58114BC for ; Mon, 24 Feb 2020 20:32:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B100220675 for ; Mon, 24 Feb 2020 20:32:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Wym6MiWq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B100220675 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F322E6B00AE; Mon, 24 Feb 2020 15:32:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EE3B66B00AF; Mon, 24 Feb 2020 15:32:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF9C66B00B0; Mon, 24 Feb 2020 15:32:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id C63D66B00AE for ; Mon, 24 Feb 2020 15:32:00 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 82358824556B for ; Mon, 24 Feb 2020 20:32:00 +0000 (UTC) X-FDA: 76526167200.08.trail69_2e95aa577c521 X-Spam-Summary: 2,0,0,bb671e977c180149,d41d8cd98f00b204,3vjjuxgykcg0hlwvpyrzzrwp.nzxwtyfi-xxvglnv.zcr@flex--walken.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:973:988:989:1260:1277:1313:1314:1345:1359:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:3138:3139:3140:3141:3142:3152:3354:3865:3867:3868:3870:3871:4321:5007:6120:6261:6653:7875:7903:8660:9036:9969:10004:10400:10945:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12555:12683:12895:13148:13230:13972:14096:14097:14181:14659:14721:21080:21324:21444:21451:21627:21990:30003:30012:30054:30070,0,RBL:209.85.216.74:@flex--walken.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:0,LUA_SUMMARY:none X-HE-Tag: trail69_2e95aa577c521 X-Filterd-Recvd-Size: 5607 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Mon, 24 Feb 2020 20:32:00 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id w4so416009pjt.5 for ; Mon, 24 Feb 2020 12:31:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=868boSFkVmfEWD3v65FDJM67TMj+vIuExQ/AnL0vgiQ=; b=Wym6MiWqZmzwSo8teZ/nrQIO0/ndYZcSBNNXvBMUZO1qbH+q57OrxFvWTf5k+2SyEm G3oAkeyC0eQYoVh0V9we1M+InvlilMw7JPMwrcLKE6CjsMPfG0luFOw18bntRs9FQSqD +z7oFsA4GvJMjDv7V01/i6yCjYRsmJRZNhxGoOIP1fbgjMmniZlEKhadp/TCE69KLVzE waSHqz2h98aBJnIMJWdqkQSCFksLjAYS1dteYTbrNFmxp1fLPbdU8rpadyg9a/jyAMCp 4lKuEYftwhtd960un4oR4TvoTIq8+3luPInJo+pvuwF5G8YCpyLiComHHzEB/MHIFtJ0 sDgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=868boSFkVmfEWD3v65FDJM67TMj+vIuExQ/AnL0vgiQ=; b=M5vQFhpKBLK2IVP0Pjw1HIJ8MlqqtILzFxODLmc68re01pdXyecXNGSsXnGn46zOQD ySKXwzL70VKctxy2JpdckOs1g7U4dFUMkplQhpNvKGSJhv93eiQPpVZvTaDEIKmkSWN0 5/DLMLKYkuyf/w1lwYRPnU65leqp1d8Cfsb6v7e/tISUBAKrO6GK1esNEz41wr4jOz/m zlHiyNiYUnQHyPknI59i2uEarr17jvxufLiesqguQ6irxLeLcglgrGZgUxzQXVtzhVD8 CQ8DBL6eopBfn7dFNxmHfwwPTMweN9XdQCE+Wh8bDV9MCbbwq9jQNaV48k7G6sBPVohX zgLw== X-Gm-Message-State: APjAAAUyiNgf+dAi9SoLlAsvowRl8PgVtmtrsvQcqQwuohvS7q7Jto60 kG4gDKcOJRVgg4YRVPT0KMuTgLZqnME= X-Google-Smtp-Source: APXvYqx3WfIzf0fOBNhhodqZ0lP7yxpx/F8OmAWztQyoEdvxwhYukEX1o2JpV3Lvhnl55I5J+KPhl/U1ohw= X-Received: by 2002:a63:de54:: with SMTP id y20mr54906856pgi.79.1582576318907; Mon, 24 Feb 2020 12:31:58 -0800 (PST) Date: Mon, 24 Feb 2020 12:30:57 -0800 In-Reply-To: <20200224203057.162467-1-walken@google.com> Message-Id: <20200224203057.162467-25-walken@google.com> Mime-Version: 1.0 References: <20200224203057.162467-1-walken@google.com> X-Mailer: git-send-email 2.25.0.265.gbab2e86ba0-goog Subject: [RFC PATCH 24/24] do_mmap: implement easiest cases of fine grained locking From: Michel Lespinasse To: Peter Zijlstra , Andrew Morton , Laurent Dufour , Vlastimil Babka , Matthew Wilcox , "Liam R . Howlett" , Jerome Glisse , Davidlohr Bueso , David Rientjes Cc: linux-mm , Michel Lespinasse X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use a range lock in the easiest possible mmap case: - the mmap address is known; - there are no existing vmas within the mmap range; - there is no file being mapped. When these conditions are met, we can trivially support a fine grained range lock by just holding the mm_vma_lock accross the entire mmap operation. This is safe because the mmap only registers the new mapping using O(log N) operations, and does not have to call back into arbitrary code (such as file mmap handlers) or iterate over existing vmas and mapped pages. Signed-off-by: Michel Lespinasse --- mm/mmap.c | 36 +++++++++++++++++++++++++++++------- 1 file changed, 29 insertions(+), 7 deletions(-) diff --git mm/mmap.c mm/mmap.c index 75755f1cbd0b..5fa23f300e72 100644 --- mm/mmap.c +++ mm/mmap.c @@ -1372,6 +1372,7 @@ unsigned long do_mmap(struct file *file, unsigned long addr, unsigned long pgoff, bool locked, unsigned long *populate, struct list_head *uf) { + struct mm_lock_range mmap_range, *range = NULL; struct mm_struct *mm = current->mm; int pkey = 0; @@ -1406,8 +1407,18 @@ unsigned long do_mmap(struct file *file, unsigned long addr, if ((pgoff + (len >> PAGE_SHIFT)) < pgoff) return -EOVERFLOW; - if (!locked && mm_write_lock_killable(mm)) - return -EINTR; + if (!locked) { + if (addr && !file) { + mm_init_lock_range(&mmap_range, addr, addr + len); + range = &mmap_range; + } else + range = mm_coarse_lock_range(); + retry: + if (mm_write_range_lock_killable(mm, range)) + return -EINTR; + if (!mm_range_is_coarse(range)) + mm_vma_lock(mm); + } /* Too many mappings? */ if (mm->map_count > sysctl_max_map_count) { @@ -1422,12 +1433,20 @@ unsigned long do_mmap(struct file *file, unsigned long addr, if (IS_ERR_VALUE(addr)) goto unlock; - if (flags & MAP_FIXED_NOREPLACE) { + if ((flags & MAP_FIXED_NOREPLACE) || + (!locked && !mm_range_is_coarse(range))) { struct vm_area_struct *vma = find_vma(mm, addr); if (vma && vma->vm_start < addr + len) { - addr = -EEXIST; - goto unlock; + if (flags & MAP_FIXED_NOREPLACE) { + addr = -EEXIST; + goto unlock; + } else { + mm_vma_unlock(mm); + mm_write_range_unlock(mm, range); + range = mm_coarse_lock_range(); + goto retry; + } } } @@ -1587,8 +1606,11 @@ unsigned long do_mmap(struct file *file, unsigned long addr, *populate = len; unlock: - if (!locked) - mm_write_unlock(mm); + if (!locked) { + if (!mm_range_is_coarse(range)) + mm_vma_unlock(mm); + mm_write_range_unlock(mm, range); + } return addr; }