From patchwork Mon May 28 18:04:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 10433937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 064A260327 for ; Mon, 28 May 2018 18:04:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA38128485 for ; Mon, 28 May 2018 18:04:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DE6AC2848B; Mon, 28 May 2018 18:04:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2AD9028485 for ; Mon, 28 May 2018 18:04:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38C946B0007; Mon, 28 May 2018 14:04:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 317A06B0008; Mon, 28 May 2018 14:04:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DED66B000A; Mon, 28 May 2018 14:04:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199]) by kanga.kvack.org (Postfix) with ESMTP id B58B46B0007 for ; Mon, 28 May 2018 14:04:25 -0400 (EDT) Received: by mail-wr0-f199.google.com with SMTP id k18-v6so10865350wrm.6 for ; Mon, 28 May 2018 11:04:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:mime-version; bh=irOdRywaZCHET+e3UcnFOUB9j+6lApaaFjw2mh4fYBs=; b=bRTute4k1subzp0UlvOYAhssKZp4HZPucDp65ql606PwpJBthabybvEwrKS0Y01cqA 0arxv1wzsR55F7gh8cl7c6j4ii7xMwMzrBe6tJ04L6qDheWkbtmVlV21owULbBUDryRi dz8PD8OyZjkSry1kvvATTlOv81oD5ZVSWSWyzMz/YYyXP9PXT5uHjB/3xwoT6jbRJj8k bfpDLXBD3vZ1Roc2836Y3zcd2rrxXhkWWwAfRafCe4BpMUwyeS8HTw7HzDGgrNgKYGYX QPhtww/yR0VjvWz8rUBZQBKYuQgdIPPMnKI+J+BLzl36t1Kkhho1g8uvcfvUAvdlpejt +WlQ== X-Gm-Message-State: ALKqPweCzQm2Aeg3kM8yTcQOMyg6mjbQ8hf0vP9XXMS+fj0NXZ0szHiT +DlApCg6FTRxzj3cwlVBEPSOBc7OHW2AGcsbl4n42zAnCk+M5UOoVciRR1AJuiU/3F+VaCMfDC/ 0Ve8CnDmZmsmT2XLE6ei6Sf96BKSmQfZWURV5aaE5BlKWUZrzVbLdzNUkIzX9D2eYKg== X-Received: by 2002:a1c:c10b:: with SMTP id r11-v6mr10221603wmf.120.1527530665280; Mon, 28 May 2018 11:04:25 -0700 (PDT) X-Google-Smtp-Source: ADUXVKK347fb79PM49UR49I8ZrGcNJTAemZpEu7YFSPjxRRNi6tddy/ap4hbYM6putjqrCr0ENMN X-Received: by 2002:a1c:c10b:: with SMTP id r11-v6mr10221556wmf.120.1527530664437; Mon, 28 May 2018 11:04:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527530664; cv=none; d=google.com; s=arc-20160816; b=KxwJ8BT6M3oMt8ZIZgIG1N88rdRCoWJDQTOAukguiv/sYRuB8MabWft+sG0GyWoh1U jv/RNHpgyxI3WOwjQii2KlUUReB10Gh0oe9WDQDbXR8edL41po5I/g4Tja+HT2zL4gzx PIFf7vPWZUL0iZ3lepkS8xqVxtXIlmYu807Z8iPWvICJPKyEmtt0b1pPXeZGBEuuJnL/ UI8njyn681STmsOaBPH6y0WIdktTPPvI6FJGY1SUWBla2sQ2LNHYU87IZf7Thf73nP7Y AvvaOvJfjzhOGN70I9Bsgsve6pLyNF7l7y+aAzh926uNG1Rl7wm1B34J3u2gkfTTAmt9 3/8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:smtp-origin-cluster:cc:to :smtp-origin-hostname:from:smtp-origin-hostprefix:dkim-signature :arc-authentication-results; bh=irOdRywaZCHET+e3UcnFOUB9j+6lApaaFjw2mh4fYBs=; b=d+tLwMlaKUGuYVRf/DQMiiBqLY3VPJV4UDBGwlL9vinQDkTEHM7bDtmjVz7QbxOIkd ZcjvJkXyKzwzDIPPO3YIbw5+UTYkYuYkT4rPkkSXEvko7fEP0SMWx6cDe9DCeMRJBM9O 3rdZvbl2LcvxFAeiU+HabO6+kMxzXzgjZgSmQjALaQX6z9bRT8fExB6xqX1zBz9gqOqU Q+vYmdJIIxQTDGHguhBMf/VsaKl3FMo/qyvipOdu5uWEHS/rAA7AnlaAZMO5iZQKEINz AAIQpGFivvZxIcBG2KaHh6Gd8+qNh4i+IMRbl2KlH8ex2J7gEqBd8xrbYBABGrcK9H4i rBxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=pqt50etv; spf=pass (google.com: domain of prvs=968665f4dc=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom=prvs=968665f4dc=songliubraving@fb.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id a62-v6si10020355wmf.135.2018.05.28.11.04.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 May 2018 11:04:24 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=968665f4dc=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=pqt50etv; spf=pass (google.com: domain of prvs=968665f4dc=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom=prvs=968665f4dc=songliubraving@fb.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4SI3rIl007248 for ; Mon, 28 May 2018 11:04:23 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=facebook; bh=irOdRywaZCHET+e3UcnFOUB9j+6lApaaFjw2mh4fYBs=; b=pqt50etvi4TdnF6V+cT5lr/VBVa2TBXsXPu6/PT95slQxhDogSmpXrb86MTaBi0qJLoD 049iRl8l31Ry02wgmnTN6G4W1usUQG+VduDUKOXCKlX31HP8J3AD+OuiL+swB7eD/UB/ Q8fesuRkT29apmYjW0nx1mu5vzpV05fcVIA= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2j8k8mrd12-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 28 May 2018 11:04:23 -0700 Received: from mx-out.facebook.com (192.168.52.123) by PRN-CHUB13.TheFacebook.com (192.168.16.23) with Microsoft SMTP Server id 14.3.361.1; Mon, 28 May 2018 11:04:22 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 1CE9D62E0E0A; Mon, 28 May 2018 11:04:20 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: CC: , Song Liu , , , , , Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v3] mm/THP: use hugepage_vma_check() in khugepaged_enter_vma_merge() Date: Mon, 28 May 2018 11:04:12 -0700 Message-ID: <20180528180412.529686-1-songliubraving@fb.com> X-Mailer: git-send-email 2.9.5 X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-28_11:, , signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP khugepaged_enter_vma_merge() is using a different approach to check whether a vma is valid for khugepaged_enter(): if (!vma->anon_vma) /* * Not yet faulted in so we will register later in the * page fault if needed. */ return 0; if (vma->vm_ops || (vm_flags & VM_NO_KHUGEPAGED)) /* khugepaged not yet working on file or special mappings */ return 0; This check has some problems. One of the obvious problems is that it doesn't check shmem_file(), so that vma backed with shmem files will not call khugepaged_enter(). Here is an example of failed madvise(): /* mount /dev/shm with huge=advise: * mount -o remount,huge=advise /dev/shm */ /* create file /dev/shm/huge */ #define HUGE_FILE "/dev/shm/huge" fd = open(HUGE_FILE, O_RDONLY); ptr = mmap(NULL, FILE_SIZE, PROT_READ, MAP_PRIVATE, fd, 0); ret = madvise(ptr, FILE_SIZE, MADV_HUGEPAGE); madvise() will return 0, but this memory region is never put in huge page (check from /proc/meminfo: ShmemHugePages). This patch fixes these problems by reusing hugepage_vma_check() in khugepaged_enter_vma_merge(). vma->vm_flags is not yet updated in khugepaged_enter_vma_merge(), so we need to pass the new vm_flags to hugepage_vma_check() through a separate argument. Signed-off-by: Song Liu --- mm/khugepaged.c | 53 ++++++++++++++++++++++++----------------------------- 1 file changed, 24 insertions(+), 29 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d7b2a4b..e0a3bdf 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -430,18 +430,32 @@ int __khugepaged_enter(struct mm_struct *mm) return 0; } +static bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags) +{ + if ((!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) || + (vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) + return false; + if (shmem_file(vma->vm_file)) { + if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE)) + return false; + return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, + HPAGE_PMD_NR); + } + if (!vma->anon_vma || vma->vm_ops) + return false; + if (is_vma_temporary_stack(vma)) + return false; + return !(vm_flags & VM_NO_KHUGEPAGED); +} + int khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long vm_flags) { unsigned long hstart, hend; - if (!vma->anon_vma) - /* - * Not yet faulted in so we will register later in the - * page fault if needed. - */ - return 0; - if (vma->vm_ops || (vm_flags & VM_NO_KHUGEPAGED)) - /* khugepaged not yet working on file or special mappings */ + + if (!hugepage_vma_check(vma, vm_flags)) return 0; hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; hend = vma->vm_end & HPAGE_PMD_MASK; @@ -819,25 +833,6 @@ khugepaged_alloc_page(struct page **hpage, gfp_t gfp, int node) } #endif -static bool hugepage_vma_check(struct vm_area_struct *vma) -{ - if ((!(vma->vm_flags & VM_HUGEPAGE) && !khugepaged_always()) || - (vma->vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - if (shmem_file(vma->vm_file)) { - if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE)) - return false; - return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, - HPAGE_PMD_NR); - } - if (!vma->anon_vma || vma->vm_ops) - return false; - if (is_vma_temporary_stack(vma)) - return false; - return !(vma->vm_flags & VM_NO_KHUGEPAGED); -} - /* * If mmap_sem temporarily dropped, revalidate vma * before taking mmap_sem. @@ -862,7 +857,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, hend = vma->vm_end & HPAGE_PMD_MASK; if (address < hstart || address + HPAGE_PMD_SIZE > hend) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma)) + if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; return 0; } @@ -1694,7 +1689,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma)) { + if (!hugepage_vma_check(vma, vma->vm_flags)) { skip: progress++; continue;