From patchwork Mon Jun 3 06:34:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 10972397 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4952D13AD for ; Mon, 3 Jun 2019 06:34:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2CA5828852 for ; Mon, 3 Jun 2019 06:34:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19AC628861; Mon, 3 Jun 2019 06:34:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9875628852 for ; Mon, 3 Jun 2019 06:34:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35D5C6B000C; Mon, 3 Jun 2019 02:34:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2E6916B0269; Mon, 3 Jun 2019 02:34:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1397C6B026A; Mon, 3 Jun 2019 02:34:33 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id CF9886B000C for ; Mon, 3 Jun 2019 02:34:32 -0400 (EDT) Received: by mail-pg1-f199.google.com with SMTP id s3so9275848pgv.12 for ; Sun, 02 Jun 2019 23:34:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id; bh=CBsIkBvODy2SMAroMU4TmKj7JDPTiCcRWf3bIgO8CHE=; b=hfZJ7RlHbvB5iuGF7muNetEMGbfVX9sVPUeSSNhqXjggKy1u2svzgYUrP4aySib4ns ZM8bUI35sbJTgrLwXYZ7f9kMeabsGb15OIHGpNmWBgGi2fnqlDswRply629raqT09oQn Q40pRJiks0GszAE1ccBAGl5iW6fEwM2z4WupRk6wTXbwXHqRql+V2Q8LGbg4BUa426o5 yseBMd9D4jzPecu4UIQAkdcQkAV0FAi/vGOHJG563wcsgVtoo6Gi2z7Ylk8D+j/PkZmT NarPtjo0Fw3dF9fShGEZimORlvd3x87jA/AVdW3Hox1b7Cnng8t4A6VhWoKAAWePA/LX lvlg== X-Gm-Message-State: APjAAAXUK/xTxtv53Ur0kUpGaE5xhRZKjZW9uCStllyqwZ3CbcEV6Uxa vh8KHk7kIw0Tmxx+4yC4Rh+vrvAZwwvZbTcO1n9vpnLUK94adIrEPrb9IWXq1p4S827hya6VBeK wVefA2Eagd4Qg5jt3FL5hOrqxzjcO8Z+0XLPbVOCBFPN9ZpJCioyvZU9X55ihkDiGXg== X-Received: by 2002:a63:18e:: with SMTP id 136mr26302268pgb.277.1559543672336; Sun, 02 Jun 2019 23:34:32 -0700 (PDT) X-Received: by 2002:a63:18e:: with SMTP id 136mr26302223pgb.277.1559543671350; Sun, 02 Jun 2019 23:34:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559543671; cv=none; d=google.com; s=arc-20160816; b=qvFdmEa3cQAT+qkcvPoHWAnf+F00xJ5WyjGfA3wTPHvAs/ZPooJIEVYnmC9ZSGkdIA 1XlHJmMmX0L9KsnGdzUrr4wavKJTkERu+Etw0heJqjAvjRpbbykK6POTK8tFyhZTy5H2 XnJgAzOd95hrnNsiBHozSQ3RWlbB5s6sDlCZpCTVcl/V1eAEP7DMyQkcM62XsWms1NQZ PmCvg7fwekKY+6zaiLzvUFBMyv37ReEJdjaZDfrIxT4GgjBLGUFBSN62PW4Rv5BHww7p m4HbXq7r5b4J/X70nKHiLfYG982cJRso/+mrk7a2szvreo0um3ER+zalTzz+ULina5Na lVow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature; bh=CBsIkBvODy2SMAroMU4TmKj7JDPTiCcRWf3bIgO8CHE=; b=a2qFVQTu8Wm9AjD7L8tX6+0/TQLsrp7o7j1heYYOcmGYTEuO7WTFU2Jj3lYTY6x0+o aAdnZuLLqaa1ZoXD4/whTHbJ9Q7IOTNMpMJmdsDam1sUWUSUmLpTuPukEyREseHBoxe2 iGLcht+AKiZUYUXs21xb8dc6zU19I5FMCs6afzdxOSaVt/Cr+x4vacJ9fgvrmPHYFnTa mHFDgKIiL0bx0SxTwZ6+kTUpNyqXRKsMVTsfCPXHfMomQM7LsarVWZ93/JldbgMV2sKd Tfz+ws5DJlJ83hBTGzM5exbvYW4YfnHmY5VyC5lvmqgz50h6srBSGHq/Edx13zwkV6hz pVfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BhMamTU0; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id o5sor13939217pgk.82.2019.06.02.23.34.31 for (Google Transport Security); Sun, 02 Jun 2019 23:34:31 -0700 (PDT) Received-SPF: pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BhMamTU0; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=CBsIkBvODy2SMAroMU4TmKj7JDPTiCcRWf3bIgO8CHE=; b=BhMamTU0Mc/0EVu+/M0cSdxLSLtdEF4XdEDqq7cwIDqJ7R77OQ6DzkQx8nK1FEbT1z 4eBywr+SqVgYb04iLdG+/a8DSBrlu5KiO5U47jSQVrXKBW1a0G34o5ggzIhiREoNDvlk 7M2jOQlgXRYFa/bV2ss2jvJvHnZyl4wT68FX+ihBeaCkm09Yb4Ju5jSl+ni/0sz7yb14 XSCCuNn4L/L0f3REr3TwEheSyYh34Nfgs9BmZaviqWTlA6thycR71efgcT7PEou8I13m 9OPKCF0lbheyuixt8G7ahGo4tYBS65QA2LNGZU6zYTHpXQKOStN9Cu5ITqVI21372TMD Q8Fw== X-Google-Smtp-Source: APXvYqwQ1TToPxYTZgvBH7nocMRSyjS+MAPRo/77b8NkuXiJ+gfrhgYYiGq1j9IXEVMHW9lP+JJuTw== X-Received: by 2002:a65:518d:: with SMTP id h13mr25383136pgq.186.1559543670700; Sun, 02 Jun 2019 23:34:30 -0700 (PDT) Received: from mylaptop.nay.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id j14sm13859027pfe.10.2019.06.02.23.34.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 02 Jun 2019 23:34:30 -0700 (PDT) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , linux-kernel@vger.kernel.org Subject: [PATCHv2 1/2] mm/gup: fix omission of check on FOLL_LONGTERM in get_user_pages_fast() Date: Mon, 3 Jun 2019 14:34:12 +0800 Message-Id: <1559543653-13185-1-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP As for FOLL_LONGTERM, it is checked in the slow path __gup_longterm_unlocked(). But it is not checked in the fast path, which means a possible leak of CMA page to longterm pinned requirement through this crack. Place a check in the fast path. Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: linux-kernel@vger.kernel.org Reviewed-by: Ira Weiny --- mm/gup.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/mm/gup.c b/mm/gup.c index f173fcb..6fe2feb 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2196,6 +2196,29 @@ static int __gup_longterm_unlocked(unsigned long start, int nr_pages, return ret; } +#if defined(CONFIG_CMA) +static inline int reject_cma_pages(int nr_pinned, unsigned int gup_flags, + struct page **pages) +{ + if (unlikely(gup_flags & FOLL_LONGTERM)) { + int i = 0; + + for (i = 0; i < nr_pinned; i++) + if (is_migrate_cma_page(pages[i])) { + put_user_pages(pages + i, nr_pinned - i); + return i; + } + } + return nr_pinned; +} +#else +static inline int reject_cma_pages(int nr_pinned, unsigned int gup_flags, + struct page **pages) +{ + return nr_pinned; +} +#endif + /** * get_user_pages_fast() - pin user pages in memory * @start: starting user address @@ -2236,6 +2259,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, ret = nr; } + nr = reject_cma_pages(nr, gup_flags, pages); if (nr < nr_pages) { /* Try to get the remaining pages with get_user_pages */ start += nr << PAGE_SHIFT; From patchwork Mon Jun 3 06:34:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 10972399 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9DE1C933 for ; Mon, 3 Jun 2019 06:34:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CDD728852 for ; Mon, 3 Jun 2019 06:34:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 80FAC28861; Mon, 3 Jun 2019 06:34:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 074D928852 for ; Mon, 3 Jun 2019 06:34:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9B496B026A; Mon, 3 Jun 2019 02:34:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B247F6B026B; Mon, 3 Jun 2019 02:34:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A2266B026C; Mon, 3 Jun 2019 02:34:36 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 5A5F86B026A for ; Mon, 3 Jun 2019 02:34:36 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id k22so12874105pfg.18 for ; Sun, 02 Jun 2019 23:34:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=R+hQLAUcVNWROrMbU90MxMOWiIFmUVyPvMT6jkT1W4o=; b=VK9NS0vl0h7CjGOFImH7u1tmrsdQI+0XlVEfHxIbGumcLs7ZL+3vTd2b4TwF49Eqbj 08n7oA9yw92PZyXiw7xP54Bh5i0SN0KZd5/X1mjiRN0mzwQ1dHad6+T9hHY8I5TfhOk8 7r8W5I9PfJ0nPQ80FUzl08A4nxP1QBLHdxcdoiE0yqyr1dWiNIY6gXQ42emWOXAQ07k5 SzsY7Romye2T6/oxfVS6lD1/2MI+VBamZHdOzEUGpef8qhN/F9NhtT1bGGUNMVEHVIzX mKdcqni6Bb8UxlhvE9Yb+AUGIVEfmiJY6hptXIsqhuniRfC8gOyJoKpeHszCyJItINym qN9Q== X-Gm-Message-State: APjAAAWzmJOvzIhQYc6qLgW6YTE6OUS8Lw57HHpowQ1iH5WWCPxuPtrz mpm0sOio4KYOKCJfPVXCXqBNqBuUf1pqhk7pqhWBXXD9gm3XRt8XCcF9Ts3qC1mpT3fDzNf5/uq 1cHOmr8eNMDrVwQkLuoUamt/vbJ5Krpr2FN5qFUIXxEvCfNIAHuDMD2y3fYF6QDp78w== X-Received: by 2002:a17:90a:9504:: with SMTP id t4mr28210629pjo.100.1559543675901; Sun, 02 Jun 2019 23:34:35 -0700 (PDT) X-Received: by 2002:a17:90a:9504:: with SMTP id t4mr28210588pjo.100.1559543674819; Sun, 02 Jun 2019 23:34:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559543674; cv=none; d=google.com; s=arc-20160816; b=gA1evp4Ku/ddNsFYC2trjdHXnpFEn4wRulzQWkhe+mfNOJlVX2e1elQvhdpoKlcRcj a1oasiQVRV0IY1PyvxE4t01EyNsha7Y2+rCtKFVVXRwak9D7Ba8GWR4dbEcyebWL6sei Ba23kunPap42JTmZgXOrOHVamfNUiDb+IajJvHYc8jML/I29p1guBxtabYP+4y3UTTcV vtVhJH0o8vKWNnQ89mYc/TXLO7UdabkVmGiEz6dlXUheTVRpp7vPatZpFXY/lpf4IA8n 3qhe7kixBKx7hhXd0baLPD2KzrP4Q6oy6u96NoEfQ5R5XOfqc4RLcXEtNWXcKYF89wRW kllQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=R+hQLAUcVNWROrMbU90MxMOWiIFmUVyPvMT6jkT1W4o=; b=pJLdt2SJHjmDwfoR97sRiY4yPKV6tkBC37VFn0fMZihVIiL8tLHcfn72CtU4lWmECb 1VYiZDfRoxQc9Fxw/VE/oRoTg2sG2uZKwvxbnqq3ST6/u20GsET+IMerjAQCWjINY7s5 hhvRltI30gbviGS5jnL5kgIAgYB+zzkAj/B6PgMk//7abC38oTQ6o06NDFN3USA6xH9O bH/GrPoj1EjaXrIQNQBx1GynA3vaYsvKlCunQ0ULh6G+dW6Q7NSlw9hCOVHMQTPCMBww xHHDLz8xhNFwk1Q1fJxSZQLfXhKifWeZby5weX78AcogKtwKplvgR6BuoepBUuEV5N1h a79Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=JeweOGjj; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 59sor10177662plb.14.2019.06.02.23.34.34 for (Google Transport Security); Sun, 02 Jun 2019 23:34:34 -0700 (PDT) Received-SPF: pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=JeweOGjj; spf=pass (google.com: domain of kernelfans@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=kernelfans@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=R+hQLAUcVNWROrMbU90MxMOWiIFmUVyPvMT6jkT1W4o=; b=JeweOGjjlqVIH0KxKJuMpIq1jSl5UooWsZl6FxhuW8sb2MrVdCeODLdOoT53EObqUZ 4qjqklXB8sKCdnrChJYqiO2zwJFomiMOlfWPl/vGQHgSbOXLj/ZLI63GKQJPmNc16MiQ Gtu2Zndm8bSdcTbZ5IgIceYQ0+Y0UKAGwEDJEBVLuPfmUidBnMkozY/geDaUTJLyDzRS 9Ad79TVjhULakP38inxlO4HWvIVyy/C0Z2vuFCmg8zNuXIz0ARsxoWQ1M1PdLU0fVXQ8 +bV2Y1lvl8+TzKbz1LCnhDqSfNI1mwsALbONIGIuSbcLgLiaYXVT+Dh4vKyRBKAGJS9L wOdA== X-Google-Smtp-Source: APXvYqzm7+kJNYspgi7GV1P/I+D5ucOoLPIk/Mc2TPSNdzWzx+ipIDi9/crR7STfyeeK+Yteix/HFg== X-Received: by 2002:a17:902:4381:: with SMTP id j1mr27518004pld.286.1559543674435; Sun, 02 Jun 2019 23:34:34 -0700 (PDT) Received: from mylaptop.nay.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id j14sm13859027pfe.10.2019.06.02.23.34.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 02 Jun 2019 23:34:34 -0700 (PDT) From: Pingfan Liu To: linux-mm@kvack.org Cc: Pingfan Liu , Ira Weiny , Andrew Morton , Mike Rapoport , Dan Williams , Matthew Wilcox , John Hubbard , "Aneesh Kumar K.V" , Keith Busch , linux-kernel@vger.kernel.org Subject: [PATCHv2 2/2] mm/gup: rename nr as nr_pinned in get_user_pages_fast() Date: Mon, 3 Jun 2019 14:34:13 +0800 Message-Id: <1559543653-13185-2-git-send-email-kernelfans@gmail.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1559543653-13185-1-git-send-email-kernelfans@gmail.com> References: <1559543653-13185-1-git-send-email-kernelfans@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To better reflect the held state of pages and make code self-explaining, rename nr as nr_pinned. Signed-off-by: Pingfan Liu Cc: Ira Weiny Cc: Andrew Morton Cc: Mike Rapoport Cc: Dan Williams Cc: Matthew Wilcox Cc: John Hubbard Cc: "Aneesh Kumar K.V" Cc: Keith Busch Cc: linux-kernel@vger.kernel.org Reviewed-by: Ira Weiny --- mm/gup.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 6fe2feb..106ab22 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2239,7 +2239,7 @@ int get_user_pages_fast(unsigned long start, int nr_pages, unsigned int gup_flags, struct page **pages) { unsigned long addr, len, end; - int nr = 0, ret = 0; + int nr_pinned = 0, ret = 0; start &= PAGE_MASK; addr = start; @@ -2254,26 +2254,26 @@ int get_user_pages_fast(unsigned long start, int nr_pages, if (gup_fast_permitted(start, nr_pages)) { local_irq_disable(); - gup_pgd_range(addr, end, gup_flags, pages, &nr); + gup_pgd_range(addr, end, gup_flags, pages, &nr_pinned); local_irq_enable(); - ret = nr; + ret = nr_pinned; } - nr = reject_cma_pages(nr, gup_flags, pages); - if (nr < nr_pages) { + nr_pinned = reject_cma_pages(nr_pinned, gup_flags, pages); + if (nr_pinned < nr_pages) { /* Try to get the remaining pages with get_user_pages */ - start += nr << PAGE_SHIFT; - pages += nr; + start += nr_pinned << PAGE_SHIFT; + pages += nr_pinned; - ret = __gup_longterm_unlocked(start, nr_pages - nr, + ret = __gup_longterm_unlocked(start, nr_pages - nr_pinned, gup_flags, pages); /* Have to be a bit careful with return values */ - if (nr > 0) { + if (nr_pinned > 0) { if (ret < 0) - ret = nr; + ret = nr_pinned; else - ret += nr; + ret += nr_pinned; } }