From patchwork Tue Jan 15 18:12:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10764931 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C33F6C5 for ; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E37F2D467 for ; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 722842D4D8; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10D3A2D467 for ; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE0C78E0005; Tue, 15 Jan 2019 13:13:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CC7BF8E0004; Tue, 15 Jan 2019 13:13:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B45318E0005; Tue, 15 Jan 2019 13:13:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 6F48C8E0004 for ; Tue, 15 Jan 2019 13:13:31 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id y88so2556467pfi.9 for ; Tue, 15 Jan 2019 10:13:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=mUM3N24aT9PyrD+2U70aULJmOFDBVISgGQ/w+hNmTgM=; b=KFDVrTOgBzWkuHN5ltAbVs+YYVLMBbKKkJuiO8CbA5GgryyJGPhuM9u7d43Gt/BGia n+sBLbSaao2A3jCMKql5n6OTaO/WjGgt6re9WLZvnXm+v072RLAobL2zGq5NzAVbZq8D lH+SfY2twaeToKl/gCbF88Qx13HxIIcAGr8lPgwGceVeNfJEACxF3bKgyDwDrL9QkJk/ hSEOvJeQDzoaHaVfPgTeOGY5h2DyKwE+8iZYEPZ2HQTKhzJr+sPyY9cfW3wpBxnmBhww jIwddmNzcA86DGyUOHZB27vQhsg8WajxkX2NbrAHGkwazE1Pai7DY81Tl6/2IBnf+S+T u/Pw== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukdgity4eaaiwFMGgcqkuLqGsMCW0TdsD+wAOen0vPLVygfoucjA 0stoaqT8nwmKVdlNcmr1fO2RcE6Odk4wvCfwuz0BNtNh7nFk+aw1gC3Ta0y/o/egbRGaM1MNq8i QjeVI3jtKjCyKopDH8TtUfUVB4CLDlvSCuln43OGIwOxMhl0sj6bFPrgx9Bq7CRE= X-Received: by 2002:a63:30c8:: with SMTP id w191mr5075329pgw.120.1547576011073; Tue, 15 Jan 2019 10:13:31 -0800 (PST) X-Google-Smtp-Source: ALg8bN4Dw8jAAcmipob90C7MYp6E8J2tt/mvtWxEoHJxzQojqgmfGXNhPw03pKy/csQKkUJH/Ozp X-Received: by 2002:a63:30c8:: with SMTP id w191mr5075264pgw.120.1547576010181; Tue, 15 Jan 2019 10:13:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547576010; cv=none; d=google.com; s=arc-20160816; b=oDCtjCGTz3Ylm1oGkYl+hgxDNZc8SdQWymz3T5lOrkyP492EZu+nMHUil2EI8aM5EM 01okuSQS6AbyTO9FMaI4uPLaieIhnc+n1mszZfzCYdaAp0kscoU6B7dkpuHPXtTQr9CS jhXWVbQ0PmkYb3vTtOtb4N9L2ciAG1WhHBx2FoHYVcGghjuJbNh+gdLU8bTR4rvvguMs MWN+mY4hV0mmYsIkTkXoFVsO8qbGaeln7VEDGuiRdv4nOpWco8Gb/TliB4cnl5z9vj4r U9AGOvICEab/P8cnxgg1nTVbrQejjD9fqLjvSaYbqoonsESIqpV8lAbrldEF31tJYfhl nmMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=mUM3N24aT9PyrD+2U70aULJmOFDBVISgGQ/w+hNmTgM=; b=u2xesUA7vv2CMc6d1TzGnph+11mjq/7yA1rtgxInz9de010pCvV/lP5nFXbx+4UnAU jiu2Pt7SI1UmnbKUsXopumN4p6IhiD4OhRPnf3uiqgRghTHR2HtpW53fIvYY1f3phnT3 V87pFfYtEjFxnTlj0wZTTcq8AASxHOpfwg2nn4prtLEipZ2Y38AZpmGJoBAOYgb3R8Gy BVj5y9banUOlGbPf6JOUm0mmWRHE7IuBlp6xav3bziikYcgi2HNEsmVTqJZgUximmFY+ hAL6dWlL1Rv1+EEH0lLVlyjffTW0Tt9RlHU6+C7jutYs6pSiJ3KPiMi1SFFXknGHn5NH pYfw== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id i1si3780777pgi.480.2019.01.15.10.13.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jan 2019 10:13:30 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Tue, 15 Jan 2019 11:13:24 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, dave@stgolabs.net, dennis.dalessandro@intel.com, mike.marciniszyn@intel.com, Davidlohr Bueso Subject: [PATCH 3/6] drivers/IB,qib: do not use mmap_sem Date: Tue, 15 Jan 2019 10:12:57 -0800 Message-Id: <20190115181300.27547-4-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190115181300.27547-1-dave@stgolabs.net> References: <20190115181300.27547-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The driver uses mmap_sem for both pinned_vm accounting and get_user_pages(). By using gup_fast() and letting the mm handle the lock if needed, we can no longer rely on the semaphore and simplify the whole thing as the pinning is decoupled from the lock. This also fixes a bug that __qib_get_user_pages was not taking into account the current value of pinned_vm. Cc: dennis.dalessandro@intel.com Cc: mike.marciniszyn@intel.com Signed-off-by: Davidlohr Bueso Reviewed-by: Ira Weiny --- drivers/infiniband/hw/qib/qib_user_pages.c | 67 ++++++++++-------------------- 1 file changed, 22 insertions(+), 45 deletions(-) diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c index 981795b23b73..4b7a5be782e6 100644 --- a/drivers/infiniband/hw/qib/qib_user_pages.c +++ b/drivers/infiniband/hw/qib/qib_user_pages.c @@ -49,43 +49,6 @@ static void __qib_release_user_pages(struct page **p, size_t num_pages, } } -/* - * Call with current->mm->mmap_sem held. - */ -static int __qib_get_user_pages(unsigned long start_page, size_t num_pages, - struct page **p) -{ - unsigned long lock_limit; - size_t got; - int ret; - - lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - - if (num_pages > lock_limit && !capable(CAP_IPC_LOCK)) { - ret = -ENOMEM; - goto bail; - } - - for (got = 0; got < num_pages; got += ret) { - ret = get_user_pages(start_page + got * PAGE_SIZE, - num_pages - got, - FOLL_WRITE | FOLL_FORCE, - p + got, NULL); - if (ret < 0) - goto bail_release; - } - - atomic_long_add(num_pages, ¤t->mm->pinned_vm); - - ret = 0; - goto bail; - -bail_release: - __qib_release_user_pages(p, got, 0); -bail: - return ret; -} - /** * qib_map_page - a safety wrapper around pci_map_page() * @@ -137,26 +100,40 @@ int qib_map_page(struct pci_dev *hwdev, struct page *page, dma_addr_t *daddr) int qib_get_user_pages(unsigned long start_page, size_t num_pages, struct page **p) { + unsigned long locked, lock_limit; + size_t got; int ret; - down_write(¤t->mm->mmap_sem); + lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + locked = atomic_long_add_return(num_pages, ¤t->mm->pinned_vm); - ret = __qib_get_user_pages(start_page, num_pages, p); + if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { + ret = -ENOMEM; + goto bail; + } - up_write(¤t->mm->mmap_sem); + for (got = 0; got < num_pages; got += ret) { + ret = get_user_pages_fast(start_page + got * PAGE_SIZE, + num_pages - got, + FOLL_WRITE | FOLL_FORCE, + p + got); + if (ret < 0) + goto bail_release; + } + return 0; +bail_release: + __qib_release_user_pages(p, got, 0); +bail: + atomic_long_sub(num_pages, ¤t->mm->pinned_vm); return ret; } void qib_release_user_pages(struct page **p, size_t num_pages) { - if (current->mm) /* during close after signal, mm can be NULL */ - down_write(¤t->mm->mmap_sem); - __qib_release_user_pages(p, num_pages, 1); - if (current->mm) { + if (current->mm) { /* during close after signal, mm can be NULL */ atomic_long_sub(num_pages, ¤t->mm->pinned_vm); - up_write(¤t->mm->mmap_sem); } }