From patchwork Tue May 22 19:44:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 10419527 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 01A896032C for ; Tue, 22 May 2018 19:44:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6D6528ECB for ; Tue, 22 May 2018 19:44:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DAE3F28F28; Tue, 22 May 2018 19:44:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D1E928ECB for ; Tue, 22 May 2018 19:44:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0AAE76B0003; Tue, 22 May 2018 15:44:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0576E6B0005; Tue, 22 May 2018 15:44:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8A0C6B0006; Tue, 22 May 2018 15:44:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr0-f199.google.com (mail-wr0-f199.google.com [209.85.128.199]) by kanga.kvack.org (Postfix) with ESMTP id 894E06B0003 for ; Tue, 22 May 2018 15:44:37 -0400 (EDT) Received: by mail-wr0-f199.google.com with SMTP id 3-v6so15411752wry.0 for ; Tue, 22 May 2018 12:44:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:mime-version; bh=SaEyGMhhZ+l+KaSb1AbE+NZtAmjNhuC2b1yPDC9zRmc=; b=EKZe0cLUlFs74ub547aN2G99BshrfWTueIPRyWVqJcw8Va7Ob+8NRaTMzXCH052Oks iiQehqzemTTf9cGQWXgplj0uuy0abHG/3RphC0LMnq4/OKIWQh1cRxZ68FDvnlp/gqJj r3xXdDAgxogn7M8o+fh4yuVon6B89vc9/p3p7FzFkjuTARg8vC7Biltwmd3/VcHD/KyW d32TpeYBSwTsY8VXPpuR3Zd+Q5kQWXoiQ5cYvME32BjZ/g/EWEIgKXBuSB5krNV7QRQb lxBx8WI8OKucUiQaqp8NhYkqK8SyO+omeq6rLml86xGqmCUv6/H03rKXtonwBfvnOLP8 cPVw== X-Gm-Message-State: ALKqPwegMxT5prPpiINNjGgBXxb9uJPqRxFnPCh6OR+Khi44mtMxGBjQ Dg76qLz/o/uesVO3vZ/lerLMVV/VTf2jRr4nsyFQEWuiNEGHLdzVFj1hzVIxHREZu04seYgeUXo lpxOZS2Pa8AtSVUhP+ZgjU6ansJaQb8dAma4BC7G6R/do+EKTkb47+aHybVXnH9H2hg== X-Received: by 2002:aa7:c657:: with SMTP id z23-v6mr4005570edr.9.1527018276986; Tue, 22 May 2018 12:44:36 -0700 (PDT) X-Google-Smtp-Source: AB8JxZrsVquJHSMzxzv3VX9EZ7dNpptmzP3++qQW7hC1m1llH3KOhCDyB4A2tJu6yxyyQAcUlat3 X-Received: by 2002:aa7:c657:: with SMTP id z23-v6mr4005523edr.9.1527018276000; Tue, 22 May 2018 12:44:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527018275; cv=none; d=google.com; s=arc-20160816; b=ewkiCKX4zNEJx6GmZEhEJxSmNGRy2pmNecOi0k5FXMr4uJMIW0N4qzvqv8WrWxRnyS Pc8XpZvoR+24mJVMtj144ctEmEQXVAHMcRDA6nHveHtgcEh6L6D9X0buPJYnDeJYDxjp hCcJTxn5bli851ra7wJ2pi8Gnis3zjWJhRN9ONi+yZ2HVBe1evriC5thxsRimoW1PzZ/ LafeNsPoDBbItb6ySYxxfwujGK9nWuhyEBFrS9QjZSAaz2e0J5ce3nNWporLaTZULm7s dmg+4vmsqQ5h8ZcRhtAKq6ZnNo5ZWJ7Q59XTpK6bpawN4aXeUceKaShuKaw23YpmXr1O 03hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:smtp-origin-cluster:cc:to :smtp-origin-hostname:from:smtp-origin-hostprefix:dkim-signature :arc-authentication-results; bh=SaEyGMhhZ+l+KaSb1AbE+NZtAmjNhuC2b1yPDC9zRmc=; b=rep+aySPJUKafbfzZIAaGBZsktVXq7EzdCciFLxNEWzB96kFnMHKYVprPg85/JecdT Z+P+EmGJwFNy2qIQ960MEV/Kh1EuIXE8Z5Z+7LYYaKxzmeXucgfwBA67ZkXRpsPONoSX XBluXu2s5Mmd0aI/D2zkYXvK575nauliow3PfT1BC1O4+9xl6LaMrhsSRPLt1YgxfUal XwPH3niOcAwVETN8HpmTRYymSkdLVspGX+Ja0yjlKPUoWvp1NiVp/O8clQ7FSycTEm/f 8dWNFmi1R58tSQyMKfYWxGd5z/mb7HdozR0Bnr+m9oYzgf0+C6ETI5PF6YKGUY+Itvko CvMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=MecQXEx5; spf=pass (google.com: domain of prvs=86801d0329=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom=prvs=86801d0329=songliubraving@fb.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id t55-v6si447727edb.420.2018.05.22.12.44.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 May 2018 12:44:35 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=86801d0329=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=MecQXEx5; spf=pass (google.com: domain of prvs=86801d0329=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom=prvs=86801d0329=songliubraving@fb.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4MJeMCS013681 for ; Tue, 22 May 2018 12:44:33 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=facebook; bh=SaEyGMhhZ+l+KaSb1AbE+NZtAmjNhuC2b1yPDC9zRmc=; b=MecQXEx5eWmxQEBFlZtsFJ3WldYIwQ+DvYPrrxodGf6G36W+99s1PVTDV6jqYSZNEIpO +XFGBFPo6OJ+TxTTsrzZZtwCQRyVY+9FGEyEEOwQrLsNxjc0I34MV93getQpOTqQeO6v vCsC2Na97lI1G5q0qtI5KZmfXMRr1k4+eJk= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2j4r8788tr-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Tue, 22 May 2018 12:44:33 -0700 Received: from mx-out.facebook.com (192.168.52.123) by PRN-CHUB01.TheFacebook.com (192.168.16.11) with Microsoft SMTP Server id 14.3.361.1; Tue, 22 May 2018 12:44:32 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 5F92E62E1098; Tue, 22 May 2018 12:44:32 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: CC: , Song Liu , , , , , Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v2] mm/THP: use hugepage_vma_check() in khugepaged_enter_vma_merge() Date: Tue, 22 May 2018 12:44:30 -0700 Message-ID: <20180522194430.426688-1-songliubraving@fb.com> X-Mailer: git-send-email 2.9.5 X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-22_06:, , signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP khugepaged_enter_vma_merge() is using a different approach to check whether a vma is valid for khugepaged_enter(): if (!vma->anon_vma) /* * Not yet faulted in so we will register later in the * page fault if needed. */ return 0; if (vma->vm_ops || (vm_flags & VM_NO_KHUGEPAGED)) /* khugepaged not yet working on file or special mappings */ return 0; This check has some problems. One of the obvious problems is that it doesn't check shmem_file(), so that vma backed with shmem files will not call khugepaged_enter(). Here is an example of failed madvise(): /* mount /dev/shm with huge=advise: * mount -o remount,huge=advise /dev/shm */ /* create file /dev/shm/huge */ #define HUGE_FILE "/dev/shm/huge" fd = open(HUGE_FILE, O_RDONLY); ptr = mmap(NULL, FILE_SIZE, PROT_READ, MAP_PRIVATE, fd, 0); ret = madvise(ptr, FILE_SIZE, MADV_HUGEPAGE); madvise() will return 0, but this memory region is never put in huge page (check from /proc/meminfo: ShmemHugePages). This patch fixes these problems by reusing hugepage_vma_check() in khugepaged_enter_vma_merge(). vma->vm_flags is not yet updated in khugepaged_enter_vma_merge(), so we need to pass the new vm_flags to hugepage_vma_check() through a separate argument. Signed-off-by: Song Liu --- mm/khugepaged.c | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d7b2a4b..9f74e51 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -430,18 +430,15 @@ int __khugepaged_enter(struct mm_struct *mm) return 0; } +static bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags); + int khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long vm_flags) { unsigned long hstart, hend; - if (!vma->anon_vma) - /* - * Not yet faulted in so we will register later in the - * page fault if needed. - */ - return 0; - if (vma->vm_ops || (vm_flags & VM_NO_KHUGEPAGED)) - /* khugepaged not yet working on file or special mappings */ + + if (!hugepage_vma_check(vma, vm_flags)) return 0; hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; hend = vma->vm_end & HPAGE_PMD_MASK; @@ -819,10 +816,11 @@ khugepaged_alloc_page(struct page **hpage, gfp_t gfp, int node) } #endif -static bool hugepage_vma_check(struct vm_area_struct *vma) +static bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags) { - if ((!(vma->vm_flags & VM_HUGEPAGE) && !khugepaged_always()) || - (vma->vm_flags & VM_NOHUGEPAGE) || + if ((!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) || + (vm_flags & VM_NOHUGEPAGE) || test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; if (shmem_file(vma->vm_file)) { @@ -835,7 +833,7 @@ static bool hugepage_vma_check(struct vm_area_struct *vma) return false; if (is_vma_temporary_stack(vma)) return false; - return !(vma->vm_flags & VM_NO_KHUGEPAGED); + return !(vm_flags & VM_NO_KHUGEPAGED); } /* @@ -862,7 +860,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, hend = vma->vm_end & HPAGE_PMD_MASK; if (address < hstart || address + HPAGE_PMD_SIZE > hend) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma)) + if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; return 0; } @@ -1694,7 +1692,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma)) { + if (!hugepage_vma_check(vma, vma->vm_flags)) { skip: progress++; continue;