From patchwork Wed May 24 11:54:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Punit Agrawal X-Patchwork-Id: 9745847 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3ABF460209 for ; Wed, 24 May 2017 11:56:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2890028942 for ; Wed, 24 May 2017 11:56:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D20A28963; Wed, 24 May 2017 11:56:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8E49528942 for ; Wed, 24 May 2017 11:56:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Bj8vs85kbdcNBWq6IWWJPWGb23NZuIvcb8UQUQOScZU=; b=cJqOSikIJhqGxIKkKYP5LM/5OM BDYKpqo6GoSOi3eKkw6NpeJMoIyAkKrfaNO02mXwO5qfbw1dGv6J4jRjbmkwwAXnUmc5zc579oM/q dxQHpUapiinlt9Hyo6hxycj7KaRiXddP9XZl0wW+npW4hsCI8K1aP8RxHfH0SJTIhjL6XGr05aIKa Itf8Wqvs8t6vBsiPWWdxQju4F1Jd+LrihnPTc2hwCLnbEeBDV9nLEkxK91C06N1vjZMfdxfhD26ML DZO7D41ImuAVlwMj5GXxkvkTVMOm3258MfMc0zqqIeVLwBuW4wWHZWTIB2h7wjMqxc1YrzeurPp2S PE9lb7Kg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1dDUuH-0002fd-O7; Wed, 24 May 2017 11:56:49 +0000 Received: from merlin.infradead.org ([2001:4978:20e::2]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1dDUtU-00020K-LL for linux-arm-kernel@bombadil.infradead.org; Wed, 24 May 2017 11:56:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=PK59JUH6gWQUFozAm/hR/cU3VJw3x/BUILymsUNohrw=; b=LPmyAjS1pKU3978DuPnh1nOps u+RMgdZIZeYFg7EKrwguLFIvhjfX0MHOSledO+UUdWv5iKIr6nuSeK1O3OqwVdf+0MkUmKKuUCVRE J48+zcz5sfSGLJ20ZH8VDdUEslYGoCnwsKTWfeEd4NzL6qC+GMOP7ZdYo0uxFkVhxDfxrjqpohWe4 J2XRUIEAQAVVSpLObmnqwoCeUtY+SAb/IzlTTkEwhkWJifHOvFAkTl8k2VD9zzKcwTw2g7znk6XRt 31ujp7fgd3eqcUp5/kSsE8qgkc8OzJWK02YAh8/TsYCjUyxmbQbOV/pwcX3iq4j9Xlgs7CUEhyizP zJtaYCAwg==; Received: from foss.arm.com ([217.140.101.70]) by merlin.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1dDUtS-0008BA-Jo for linux-arm-kernel@lists.infradead.org; Wed, 24 May 2017 11:55:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D49D315BF; Wed, 24 May 2017 04:55:36 -0700 (PDT) Received: from localhost (e105922-lin.cambridge.arm.com [10.1.207.56]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A2D863F578; Wed, 24 May 2017 04:55:36 -0700 (PDT) From: Punit Agrawal To: akpm@linux-foundation.org Subject: [PATCH v4 4/8] mm, gup: Ensure real head page is ref-counted when using hugepages Date: Wed, 24 May 2017 12:54:05 +0100 Message-Id: <20170524115409.31309-5-punit.agrawal@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170524115409.31309-1-punit.agrawal@arm.com> References: <20170524115409.31309-1-punit.agrawal@arm.com> X-ARM-No-Footer: FoSSMail X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170524_075558_809469_58063161 X-CRM114-Status: GOOD ( 11.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, linux-arch@vger.kernel.org, Michal Hocko , steve.capper@arm.com, aneesh.kumar@linux.vnet.ibm.com, catalin.marinas@arm.com, Punit Agrawal , will.deacon@arm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kirill.shutemov@linux.intel.com, n-horiguchi@ah.jp.nec.com, linux-arm-kernel@lists.infradead.org, mike.kravetz@oracle.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP When speculatively taking references to a hugepage using page_cache_add_speculative() in gup_huge_pmd(), it is assumed that the page returned by pmd_page() is the head page. Although normally true, this assumption doesn't hold when the hugepage comprises of successive page table entries such as when using contiguous bit on arm64 at PTE or PMD levels. This can be addressed by ensuring that the page passed to page_cache_add_speculative() is the real head or by de-referencing the head page within the function. We take the first approach to keep the usage pattern aligned with page_cache_get_speculative() where users already pass the appropriate page, i.e., the de-referenced head. Apply the same logic to fix gup_huge_[pud|pgd]() as well. Signed-off-by: Punit Agrawal Acked-by: Steve Capper Cc: Michal Hocko Cc: "Kirill A. Shutemov" Cc: Aneesh Kumar K.V --- mm/gup.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index ccf8cb38234f..be67996513be 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1358,8 +1358,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, return __gup_device_huge_pmd(orig, addr, end, pages, nr); refs = 0; - head = pmd_page(orig); - page = head + ((addr & ~PMD_MASK) >> PAGE_SHIFT); + page = pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1367,6 +1366,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(page); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1396,8 +1396,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, return __gup_device_huge_pud(orig, addr, end, pages, nr); refs = 0; - head = pud_page(orig); - page = head + ((addr & ~PUD_MASK) >> PAGE_SHIFT); + page = pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1405,6 +1404,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(page); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0; @@ -1433,8 +1433,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, BUILD_BUG_ON(pgd_devmap(orig)); refs = 0; - head = pgd_page(orig); - page = head + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); + page = pgd_page(orig) + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); do { pages[*nr] = page; (*nr)++; @@ -1442,6 +1441,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, refs++; } while (addr += PAGE_SIZE, addr != end); + head = compound_head(page); if (!page_cache_add_speculative(head, refs)) { *nr -= refs; return 0;