From patchwork Thu Jun 13 10:45:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 10991561 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92D491398 for ; Thu, 13 Jun 2019 10:45:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EF7128B02 for ; Thu, 13 Jun 2019 10:45:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 72E7828B08; Thu, 13 Jun 2019 10:45:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC1AA28B02 for ; Thu, 13 Jun 2019 10:45:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06A936B000D; Thu, 13 Jun 2019 06:45:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F367B6B000E; Thu, 13 Jun 2019 06:45:51 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB1066B0010; Thu, 13 Jun 2019 06:45:51 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 9FBD46B000D for ; Thu, 13 Jun 2019 06:45:51 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id 140so14170264pfa.23 for ; Thu, 13 Jun 2019 03:45:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=f93IclPUPcs3Bt3TjEqqUNMcFkImDI5CNRnqLCqc+P8=; b=YkOJKUP3+YAi04z3VJyCWGC3lvMNay2Qj3JCOnoEwSJlF6qLRtGCwPziOE0ItPZYah XnY2EvgQI4FrFSzrn8DL1gH64nzeXqzkPQ3cb5tu0m38XNG2ScBIZW2UzwGyx0BsjDCG 0mvgOngT8Bk5xaFm/bCUngdwSHeXKC3eJqz7EX63Uq82/wzcI4foyyjqZLDX6pzYYyAF aeUKNlXh4lS3R3CxjWROtTqh2VWo4AyH9r6/bXVmgGP9B+GOH1G7qXBocaNNcLdPqqg2 B3wQ7baaesLQ8Os7Pu93IV956gIAWPWey/eH6G4lF0ZmsSHu4Kae4V2QzT5QQlh2ieqY Z1NA== X-Gm-Message-State: APjAAAWSmDTRvvBIfU0OD+FDl7NrZFAbthi35f3NqTiQugMGOffWcXI+ IkpPwxYUk58AwYGPt5OqD0JvaULW+8H3Ns/gFmRj3S98k4O+MyDG4UauBHm59O4juWDhBm4FrPc IYJgn5/BrABdFP1z9Z9InAoc5cZPo7cp+1RctF5yopGQIvwpw4UKonWMKZgeaY3AMcQ== X-Received: by 2002:a63:68b:: with SMTP id 133mr28665248pgg.385.1560422751159; Thu, 13 Jun 2019 03:45:51 -0700 (PDT) X-Received: by 2002:a63:68b:: with SMTP id 133mr28665133pgg.385.1560422750040; Thu, 13 Jun 2019 03:45:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1560422750; cv=none; d=google.com; s=arc-20160816; b=Dgcj20HzzbW7mHaLimwS9HzePDg9qeNP5TsnFCmw7PS6+UWJ31DuHKXF6nC18QM9Ok 0IFMOL8noQmxo0kj7Pa8Dl9+bVjeCj0eHhVHxa/jAHOfnavK7LSGHqx2UPLRP4Ar4TB4 gx14f1pgupAElF4FRn39cmWHPO9s2zOhmdExvTOTNz1sJ7y69V1N/l6+9OXmWGhbV0Sx EwAC5gg/Ep60bB3n9NUkgM1woQmo1t97+tn8I4HAwZxvShcLEb20HR6+UxutJ57WIr/K aou/Eh3zaUyawH7rN5Zvpsh0Q2McKW+JpUeyY7ZAF7Jjh9B1aufb5B7DaoO2W4yrq3rQ Y+xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=f93IclPUPcs3Bt3TjEqqUNMcFkImDI5CNRnqLCqc+P8=; b=toAEV2C3buT7M+g63HvV4QkNBQFRFxb+B7ihdLmJ2B5jsb7sHsluzYMDpxrgOWUc47 Ge+8ookuBjxt6R7GKTHJksCPYXKwDU3Mz53jfnHONL1MNfjsm58g/wtpFIsXtWrHML/u od7IiHWwiooOFpOR0lfkpUxmBQYZ+o1YVWuOIBY045q5pAx/+5wWLELjZglwwDGr9ssp SlqRkoh/rmeI89BM1SyVeoRAxZsElHoQIW1inHg/tkFetsWW+/IJBcbJSpBZwRjdsTjk CBkCYsZFp21s5/E4VUQWotwbFvZFJqUi6CXwxinqTpKgzw7QfO7qaPVO87Ls0vcFcWok NlaA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="BYU4x/H5"; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id q8sor2597362pll.16.2019.06.13.03.45.49 for (Google Transport Security); Thu, 13 Jun 2019 03:45:50 -0700 (PDT) Received-SPF: pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="BYU4x/H5"; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=f93IclPUPcs3Bt3TjEqqUNMcFkImDI5CNRnqLCqc+P8=; b=BYU4x/H5A/U1vDb2iAJXDaxX8mJ03+iBLBS2s7wEPWjYRW/6/REj+tyFh+ON5yPvJo tiCoiYOkqYP+kp556mYNnSX3QOkg5moTm3N6xbZw/Yrrj8L8vCoFdZF01ln2Kg1TSJ20 l8yyAkVj8u1bv0D1tNhG8nVVbabbFkU6NKgh91Yw/+sfinCHWixpfmjN3+L8OCqLFdT5 QtU1EFgQvcT1b2S+3dIO4h7MkzFXjGN/IczhQ4oDdb75vFwLc5ZAgem35ipiyzKI9zPX VpM54MBVxi/MPr7DE9sfD6fGln7FmUlJGPt2eA9X1qTtZGyKbN356+9p/3dkHHqR5suj WN0A== X-Google-Smtp-Source: APXvYqzjzG1IExx5QhEFhc3PhY3IWUPkhfXxSPgDIyhVJDO97u6bC7ct5htX5h5CIaSIaAPehBv3WA== X-Received: by 2002:a17:902:b695:: with SMTP id c21mr34964938pls.160.1560422749562; Thu, 13 Jun 2019 03:45:49 -0700 (PDT) Received: from mylaptop.redhat.com ([2408:8207:7825:dd90:9051:d949:55f9:678b]) by smtp.gmail.com with ESMTPSA id a13sm2813285pgh.6.2019.06.13.03.45.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Jun 2019 03:45:48 -0700 (PDT) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , Christoph Hellwig , Shuah Khan , linux-kernel@vger.kernel.org Subject: [PATCHv4 1/3] mm/gup: rename nr as nr_pinned in get_user_pages_fast() Date: Thu, 13 Jun 2019 18:45:00 +0800 Message-Id: <1560422702-11403-2-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1560422702-11403-1-git-send-email-kernelfans@gmail.com> References: <1560422702-11403-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To better reflect the held state of pages and make code self-explaining, rename nr as nr_pinned. Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: Christoph Hellwig Cc: Shuah Khan Cc: linux-kernel@vger.kernel.org Reviewed-by: Ira Weiny --- mm/gup.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index f173fcb..766ae54 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2216,7 +2216,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages) { unsigned long addr, len, end; - int nr = 0, ret = 0; + int nr_pinned = 0, ret = 0; start &= PAGE_MASK; addr = start; @@ -2231,25 +2231,25 @@ int get_user_pages_fast(unsigned long start, int nr_pages, if (gup_fast_permitted(start, nr_pages)) { local_irq_disable(); - gup_pgd_range(addr, end, gup_flags, pages, &nr); + gup_pgd_range(addr, end, gup_flags, pages, &nr_pinned); local_irq_enable(); - ret = nr; + ret = nr_pinned; } - if (nr < nr_pages) { + if (nr_pinned < nr_pages) { /* Try to get the remaining pages with get_user_pages */ - start += nr << PAGE_SHIFT; - pages += nr; + start += nr_pinned << PAGE_SHIFT; + pages += nr_pinned; - ret = __gup_longterm_unlocked(start, nr_pages - nr, + ret = __gup_longterm_unlocked(start, nr_pages - nr_pinned, gup_flags, pages); /* Have to be a bit careful with return values */ - if (nr > 0) { + if (nr_pinned > 0) { if (ret < 0) - ret = nr; + ret = nr_pinned; else - ret += nr; + ret += nr_pinned; } } From patchwork Thu Jun 13 10:45:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 10991563 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 215AD76 for ; Thu, 13 Jun 2019 10:46:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0CC9C28B02 for ; Thu, 13 Jun 2019 10:46:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F399A28B08; Thu, 13 Jun 2019 10:46:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F9D128B02 for ; Thu, 13 Jun 2019 10:46:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 721956B0010; Thu, 13 Jun 2019 06:46:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6AC2C6B0266; Thu, 13 Jun 2019 06:46:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54B6A6B0269; Thu, 13 Jun 2019 06:46:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 152EE6B0010 for ; Thu, 13 Jun 2019 06:46:00 -0400 (EDT) Received: by mail-pl1-f199.google.com with SMTP id s22so7714439plp.5 for ; Thu, 13 Jun 2019 03:46:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=uCwWLrWN30s9tONuGry7ZPwH/e/7xLPMQjqyo7O5eBk=; b=G4bbsHUvE12jV68huhwUgzoHH4opSGn266Q5FZzqB4YEEba/nWf+D0sh64ci/yjPk3 FhsLmfZwcUxvoWxT4N1rQm1YKTnuBU9HJHRWIdNmkORM7dDqQjwZ7diI54yBqf+mIDDp frA0MPxjmVOMIW0ng5Y7q6IZOBrmCFvjSSiqdHfd72/GpUmPnr3c2MomGB9ulnqn4wm2 vWnPb7N0VVwzcSxZUL/0/GgWY8sP4Fh7asNOmOe+IagIfS6dTgXVC18N3AgVlfZ0PsIu xD7thuMWSGdrqnf7RJDV4LIuAc5D0qqkeUBBqVO6wEtQCa3t+uzRuF605qWE4ELf5Afq qmVA== X-Gm-Message-State: APjAAAX3grM1gMOHr6IOaFlvoYRBrb/0S5H64JkxmqxqqOJWheBIQ3rj VHVeRV1ZCw/GKunwMqonlTDu4BuCVkHCp7cB9c7QuiKHyazV6mW7Wh1CywaJgr2gqN5FIzd3x7k avx+FEdaCqZXwxih8Few+AjTIOuhtTcKzF59blAbAqKQqxGlSZk8Pz4D4fzPll9bCaw== X-Received: by 2002:a62:5e06:: with SMTP id s6mr95120015pfb.193.1560422759757; Thu, 13 Jun 2019 03:45:59 -0700 (PDT) X-Received: by 2002:a62:5e06:: with SMTP id s6mr95119954pfb.193.1560422759188; Thu, 13 Jun 2019 03:45:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1560422759; cv=none; d=google.com; s=arc-20160816; b=0vaW532rLSzvNqfyRJsbS/8QDK2yBtJyVVyv3vEOvtFH4Pfhy+KM8tjndQKsFp+eeE suir04RbhoSwRAy286eDBTZGpoZeVgQFPXHh85h9EQauwRGtkw71PUXbschFjj6aqqpE 9HaU6MyXh0FnovPfQHTdD2xTtCbbsvMdD+W5cASwynqZpoQlI8mC1Qq9hO0sXN+CIjLk 6W2x2H0jF0XHncTb8JerJTZV/4uvIbHRBUvGLtozAfj4feIwqC2BP3Ueddze1QIHjGHt cXcXUCO4ytj2soSz7pqjbhrZzrh4NjWfzDqXlKcEpw/9d0T8XLDUkQUO+5LQFnd+JL/t uoZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=uCwWLrWN30s9tONuGry7ZPwH/e/7xLPMQjqyo7O5eBk=; b=UjO7jDkofulH2nVzZqgUczoqT7AdWHeoos/VxCb3t18nP+9eIDO42oOERafVmN6Pdg LpYS4NCR8T4oKwrzpHruIzbpF/4thmegwUNe0kcCCV3yKqzqHcbG8zwCR3wnjINuZods kLlj8gvN5ABt1wjG8Ka0SA5KC9n6lrw4UkYnpmXKI66VWEokhQ7kNxUqHK/0oDqzKetM EpETahMJGUOPw7LYHSzaglEDATyl61o2g6YQKZDBJAqzsi67dAC1aMZNWMI9LS1W3IPG Z56G19AS6d0KckDOqXp+/hcHrIf8QU4mVbfQ3QMDYTEiHDC2zzqPX74Zxt3/5vc49fZc LnxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=OP0W8yRl; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id m37sor2664059pla.6.2019.06.13.03.45.59 for (Google Transport Security); Thu, 13 Jun 2019 03:45:59 -0700 (PDT) Received-SPF: pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=OP0W8yRl; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=uCwWLrWN30s9tONuGry7ZPwH/e/7xLPMQjqyo7O5eBk=; b=OP0W8yRlUthFREqCxHMZcGfrR7ObjX7GSg3sDYQOctNsRHbm2Lqcb/opmbujosV95J Kh99aG4N5q3cD8Qe5BzWR9WJEMlx27hM+S6fiT/SGbKF6lFXXfH5hU2LKUg7g4jKajV7 i+CpumuA89UHiI+0KEwM0sEc8xCFxvGVPnOo/wSJ+eHrjcMmDLkXIKzRiOrMhUlW3xX2 0j2845eItLsOowqgOEDPMtCStDCTisuEjva0X94eD9UD6KLLzhjtMkuZ7kqhppDiaS4C nwXrzRFqbHigTy4SOi86TzPaELfID3cNgUHIFdnNBPTaW8jZPP1i+VQqes+JkhBDgzO0 JNAw== X-Google-Smtp-Source: APXvYqz9LfQKa/wrOUIWnFUFZ/wGYmdDsdGq/Xe0dgLwb9nKtqXnSonygBcQvBArP0L1tEzTp904NA== X-Received: by 2002:a17:902:a506:: with SMTP id s6mr12547789plq.87.1560422758789; Thu, 13 Jun 2019 03:45:58 -0700 (PDT) Received: from mylaptop.redhat.com ([2408:8207:7825:dd90:9051:d949:55f9:678b]) by smtp.gmail.com with ESMTPSA id a13sm2813285pgh.6.2019.06.13.03.45.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Jun 2019 03:45:58 -0700 (PDT) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , Christoph Hellwig , Shuah Khan , linux-kernel@vger.kernel.org Subject: [PATCHv4 2/3] mm/gup: fix omission of check on FOLL_LONGTERM in gup fast path Date: Thu, 13 Jun 2019 18:45:01 +0800 Message-Id: <1560422702-11403-3-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1560422702-11403-1-git-send-email-kernelfans@gmail.com> References: <1560422702-11403-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP FOLL_LONGTERM suggests a pin which is going to be given to hardware and can't move. It would truncate CMA permanently and should be excluded. FOLL_LONGTERM has already been checked in the slow path, but not checked in the fast path, which means a possible leak of CMA page to longterm pinned requirement through this crack. Place a check in gup_pte_range() in the fast path. Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: Christoph Hellwig Cc: Shuah Khan Cc: linux-kernel@vger.kernel.org --- mm/gup.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/mm/gup.c b/mm/gup.c index 766ae54..de1b03f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1757,6 +1757,14 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, VM_BUG_ON(!pfn_valid(pte_pfn(pte))); page = pte_page(pte); + /* + * FOLL_LONGTERM suggests a pin given to hardware. Prevent it + * from truncating CMA area + */ + if (unlikely(flags & FOLL_LONGTERM) && + is_migrate_cma_page(page)) + goto pte_unmap; + head = try_get_compound_head(page, 1); if (!head) goto pte_unmap; @@ -1900,6 +1908,12 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + if (unlikely(flags & FOLL_LONGTERM) && + is_migrate_cma_page(page)) { + *nr -= refs; + return 0; + } + head = try_get_compound_head(pmd_page(orig), refs); if (!head) { *nr -= refs; @@ -1941,6 +1955,12 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + if (unlikely(flags & FOLL_LONGTERM) && + is_migrate_cma_page(page)) { + *nr -= refs; + return 0; + } + head = try_get_compound_head(pud_page(orig), refs); if (!head) { *nr -= refs; @@ -1978,6 +1998,12 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + if (unlikely(flags & FOLL_LONGTERM) && + is_migrate_cma_page(page)) { + *nr -= refs; + return 0; + } + head = try_get_compound_head(pgd_page(orig), refs); if (!head) { *nr -= refs; From patchwork Thu Jun 13 10:45:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 10991565 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E110976 for ; Thu, 13 Jun 2019 10:46:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF41428B02 for ; Thu, 13 Jun 2019 10:46:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C34B428B08; Thu, 13 Jun 2019 10:46:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 379CA28B02 for ; Thu, 13 Jun 2019 10:46:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C5E26B0269; Thu, 13 Jun 2019 06:46:10 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 34F3F6B026A; Thu, 13 Jun 2019 06:46:10 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F0416B026B; Thu, 13 Jun 2019 06:46:10 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id D787D6B0269 for ; Thu, 13 Jun 2019 06:46:09 -0400 (EDT) Received: by mail-pl1-f197.google.com with SMTP id d2so11689382pla.18 for ; Thu, 13 Jun 2019 03:46:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=pd2UgfTEjIEqeOaghBSIw7YbVcG8PFXXSl39qx+Thxw=; b=WkXo1O/U3Nj4FIlwfvBlIjOhXxZuPVoWFyoxi7I7AnMi2p8urrH4G4JRJlZtnJagmG SM6wLkWuvHdGEhwT0jiQFz5EQ3AhsfVRt+twqYndjn0wteL4mvXMa6pK9j0rN2J1sr4I j4UiT6IlgULXU5OYv10buHsrFGj4zprcQy7Q+L7XcUX2Hv2AuDRfvAt7nmNlmRbYQmQt JBybvRgxjt8SxIbb6Z2kDRPexiQyQ+ynJi6EYqCzzLg6MRs6PgQFTRumtCIxHzGBytPf nvkmS1dVX5y6Ia0/pJfuO1VP5nNZ1LYBZnay57aqwgkw0AiwKagrcHkXXnmiqShA6Tmu 5bLA== X-Gm-Message-State: APjAAAVrXBkYB2NNJnKqD/vRfiWvdcr/6Ezv1Aiu+jOdsEeLXKzQvcZx fqZVi6NTJe+sXDBr4VPyx19EqVhsxo56uX6xwACC7GhzVq8tNUuIsy/oott9pRuZu476HfAKsSC QIUhc3Qx7+9qQX+tkqMgW+nsBTZHMVmytYk3FnJWmmDMBp8kZQTnlesgqC6NtvEsOBA== X-Received: by 2002:a17:90a:9905:: with SMTP id b5mr4953748pjp.70.1560422769554; Thu, 13 Jun 2019 03:46:09 -0700 (PDT) X-Received: by 2002:a17:90a:9905:: with SMTP id b5mr4953614pjp.70.1560422768382; Thu, 13 Jun 2019 03:46:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1560422768; cv=none; d=google.com; s=arc-20160816; b=jsuUGcyPrkhCxDeeQr7zpoqx7ATe2erpMpqta26e+4o/1BZu3pSssnB4PZgpU0yG3b oxrW49iDgsbJbHr3RxGNqyWSt6wulXZoW7wD7oPeHb/gQKHIp6J5Vy8xdTSG3fHPu+29 gZzEI8zHPIW3rz/7+GMPr4/uqb+Fjnpvw18UsALB0fSfdSDzlMge2CCca3rS4UWEZ9/p aBiioajfSaFXzSdeiHTMxF7msK16hyQwY7+5TjhHaC2s9hqrqX5JQA3/NmrGvPXI6co1 qL771NRei6+m9w84WgB77MUk57AUAM+ux6/XFkp7JV36H5renz0v765SzQFeCAfDPCfa zYGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=pd2UgfTEjIEqeOaghBSIw7YbVcG8PFXXSl39qx+Thxw=; b=wG+ANmgmdfLKSJG8rRUuwCxSqSpyHUt/6YgnF6Gmg4O8SMm71FHEHs3TQKuYwoZDPr QNR7oMc5Nv/Uv72q5y5fcRBzfrL0BrNCIHh2z1n0d53HnOaWBAdHyfy9v2a6Nj+7JmS0 7dFuc6mcXpNMuVvRIqxkuMEjrGsfKIvTO+bLi/S0HU0R8PQckcQixOnzAb3XaVabP0k8 6xNxpaJAWO7f/q9FVOmxwcpbyk3z4VxrAxf6OC93ImsiqjCmtTShlmdk2Y9uPyoQTGpI LDmL3/nwGb/Pvifn9el3VlBQOK/qyv41JkR1LKt/uXmtgs6/spi/fPj2Vkk3TQaSwytN v0tg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=C5JSD5n4; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id cc5sor2586581plb.73.2019.06.13.03.46.08 for (Google Transport Security); Thu, 13 Jun 2019 03:46:08 -0700 (PDT) Received-SPF: pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=C5JSD5n4; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pd2UgfTEjIEqeOaghBSIw7YbVcG8PFXXSl39qx+Thxw=; b=C5JSD5n4A9lSn7ivR28GTiLFczd6Q2dkFHL4BmaZvwF4xDTrtWHt8tMYismMcyg2uL y1rFeektXyRR2wpeaWJlCDw1NHIRjkzD34s2ptRzp0+/Kdf9Frgc3DRtKp5qJoZUw6qa MGIqdLmFUHu3zMfzfxWlOd0dAMEDV1H4hxiRRuzK5tN8W6+kg02efURm2aAv+duHDcf8 +IFdFr0OGdxP7AgFi8gJcL6BB204Je13VF+cOBZwXJouh1qpMrXK6UFIl/Czaepdx8av jlgK12fmk34hSjNQaRWPPXxw29ULIxRdwPKsqdpk0lUjzURt+yKoJfflmG/sTo6897ho QCxw== X-Google-Smtp-Source: APXvYqyzzGjKho8nXPrUGm5asBy3qMNefeXotTwKcWnKkde5GEq3MJbTFtzDQgF+NAQZL9mqvTXLkA== X-Received: by 2002:a17:902:583:: with SMTP id f3mr24314348plf.137.1560422767994; Thu, 13 Jun 2019 03:46:07 -0700 (PDT) Received: from mylaptop.redhat.com ([2408:8207:7825:dd90:9051:d949:55f9:678b]) by smtp.gmail.com with ESMTPSA id a13sm2813285pgh.6.2019.06.13.03.45.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Jun 2019 03:46:07 -0700 (PDT) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , Christoph Hellwig , Shuah Khan , linux-kernel@vger.kernel.org Subject: [PATCHv4 3/3] mm/gup_benchemark: add LONGTERM_BENCHMARK test in gup fast path Date: Thu, 13 Jun 2019 18:45:02 +0800 Message-Id: <1560422702-11403-4-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1560422702-11403-1-git-send-email-kernelfans@gmail.com> References: <1560422702-11403-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Introduce a GUP_LONGTERM_BENCHMARK ioctl to test longterm pin in gup fast path. Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: Christoph Hellwig Cc: Shuah Khan Cc: linux-kernel@vger.kernel.org Reviewed-by: Ira Weiny --- mm/gup_benchmark.c | 11 +++++++++-- tools/testing/selftests/vm/gup_benchmark.c | 10 +++++++--- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/mm/gup_benchmark.c b/mm/gup_benchmark.c index 7dd602d..83f3378 100644 --- a/mm/gup_benchmark.c +++ b/mm/gup_benchmark.c @@ -6,8 +6,9 @@ #include #define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) -#define GUP_LONGTERM_BENCHMARK _IOWR('g', 2, struct gup_benchmark) -#define GUP_BENCHMARK _IOWR('g', 3, struct gup_benchmark) +#define GUP_FAST_LONGTERM_BENCHMARK _IOWR('g', 2, struct gup_benchmark) +#define GUP_LONGTERM_BENCHMARK _IOWR('g', 3, struct gup_benchmark) +#define GUP_BENCHMARK _IOWR('g', 4, struct gup_benchmark) struct gup_benchmark { __u64 get_delta_usec; @@ -53,6 +54,11 @@ static int __gup_benchmark_ioctl(unsigned int cmd, nr = get_user_pages_fast(addr, nr, gup->flags & 1, pages + i); break; + case GUP_FAST_LONGTERM_BENCHMARK: + nr = get_user_pages_fast(addr, nr, + (gup->flags & 1) | FOLL_LONGTERM, + pages + i); + break; case GUP_LONGTERM_BENCHMARK: nr = get_user_pages(addr, nr, (gup->flags & 1) | FOLL_LONGTERM, @@ -96,6 +102,7 @@ static long gup_benchmark_ioctl(struct file *filep, unsigned int cmd, switch (cmd) { case GUP_FAST_BENCHMARK: + case GUP_FAST_LONGTERM_BENCHMARK: case GUP_LONGTERM_BENCHMARK: case GUP_BENCHMARK: break; diff --git a/tools/testing/selftests/vm/gup_benchmark.c b/tools/testing/selftests/vm/gup_benchmark.c index c0534e2..ade8acb 100644 --- a/tools/testing/selftests/vm/gup_benchmark.c +++ b/tools/testing/selftests/vm/gup_benchmark.c @@ -15,8 +15,9 @@ #define PAGE_SIZE sysconf(_SC_PAGESIZE) #define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) -#define GUP_LONGTERM_BENCHMARK _IOWR('g', 2, struct gup_benchmark) -#define GUP_BENCHMARK _IOWR('g', 3, struct gup_benchmark) +#define GUP_FAST_LONGTERM_BENCHMARK _IOWR('g', 2, struct gup_benchmark) +#define GUP_LONGTERM_BENCHMARK _IOWR('g', 3, struct gup_benchmark) +#define GUP_BENCHMARK _IOWR('g', 4, struct gup_benchmark) struct gup_benchmark { __u64 get_delta_usec; @@ -37,7 +38,7 @@ int main(int argc, char **argv) char *file = "/dev/zero"; char *p; - while ((opt = getopt(argc, argv, "m:r:n:f:tTLUSH")) != -1) { + while ((opt = getopt(argc, argv, "m:r:n:f:tTlLUSH")) != -1) { switch (opt) { case 'm': size = atoi(optarg) * MB; @@ -54,6 +55,9 @@ int main(int argc, char **argv) case 'T': thp = 0; break; + case 'l': + cmd = GUP_FAST_LONGTERM_BENCHMARK; + break; case 'L': cmd = GUP_LONGTERM_BENCHMARK; break;