From patchwork Fri Jul 26 05:46:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11060335 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5420112C for ; Fri, 26 Jul 2019 05:53:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 946C828A0B for ; Fri, 26 Jul 2019 05:53:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8846F28A4F; Fri, 26 Jul 2019 05:53:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0F20428A24 for ; Fri, 26 Jul 2019 05:53:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 548636B0006; Fri, 26 Jul 2019 01:53:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 396F46B0007; Fri, 26 Jul 2019 01:53:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 212C08E0002; Fri, 26 Jul 2019 01:53:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f71.google.com (mail-yw1-f71.google.com [209.85.161.71]) by kanga.kvack.org (Postfix) with ESMTP id F2B036B0006 for ; Fri, 26 Jul 2019 01:52:59 -0400 (EDT) Received: by mail-yw1-f71.google.com with SMTP id i63so38830233ywc.1 for ; Thu, 25 Jul 2019 22:52:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=YpPGU3ZiU++0VQgyDS3QSdQ0lQHIOIN4ED9AD9AHh/Y=; b=HWLW4NZvi7CxPdvKx5+cfMPaAZPo17xS1v3zpxY82gvmzGt2hEKEj/idZgloQyZnuw VH5GxNuzisTkN3bLd7EpxDGd0nEkTT6dNQoMejZ7Rzyn3KY/mMZCROjCR8COsqhObHT1 IUN/tvJjcgyehjXuiD20Gp+oII+pTXjeJlC495I16R845O0z5aCPyWa40KcfwJYrzxZs 0H8mJKQ/Px4U64eryvVgecvcnbpup45NF6ZBffAiSw8d6SvmD9OA5yb5KhOpkoaXdbMZ pRrTHqfoc4Zr5dpPtxG1tDlT0OLu8QIYbwlWFrkKfGcrTmDpgYMj2n2b56m1VGR8djwq PIdQ== X-Gm-Message-State: APjAAAVRPStsXuzIe/AwdeiLoXe7EL0lsBLAj/hypItJPgfIK2CFGEpc gvPMVGloiODjncAukny0hwxrapk709L072u3h+cTChidVNxqRu4iTicY1caC4TH0qp/daagN7Wv exCVerGw4g23wFNEVNdy/emVlPGLYzwhWYkdC8NY49UZQjqROI59KKTb4dwCNjgn0hg== X-Received: by 2002:a25:d289:: with SMTP id j131mr58027468ybg.91.1564120379773; Thu, 25 Jul 2019 22:52:59 -0700 (PDT) X-Google-Smtp-Source: APXvYqwx5fe38ZVzL5adCv5QGj4Q2/vTh+d7KxV4/ZKSQUckfqM+m2FWXFbX4ToheQSl51Klewl6 X-Received: by 2002:a25:d289:: with SMTP id j131mr58027453ybg.91.1564120379298; Thu, 25 Jul 2019 22:52:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564120379; cv=none; d=google.com; s=arc-20160816; b=Xk+0JkWamcBBr/amNPPhxr9c7Z/Cg6HlTOCs1vpLhtPqostQCft82i1jsCVPDwcLNA Y+cxyJMgGC9sHkf5jThSJ6b9ujOmLqiNgJkUoEJCdLsBfRiTkYH4ub1IIaRpkL8UWGrS 9Idcj0PCQC7+eP0ZdzPyorhSiq8bn5VEM7bfA5La+R9gSX5rePSu3UIiEz3k1QDElk4w qVRU2f+Yt85NJfFErlFTzVgV+St4qbX65I5OhBbUfoA/y2DP/YDcNX+D8Gw9HV2ePvaz a7dwR03Q7+SS6HxecMbRMFiouLOcoWZ+EHyq/ekK9tcZVaflEfjnyG+cgCPeSxGBrKjM pJDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=YpPGU3ZiU++0VQgyDS3QSdQ0lQHIOIN4ED9AD9AHh/Y=; b=aO6Zmzc1SyC4QG1IBwSZlEf41YxmwpEpX9OUuDDBeCx2CaoSsXyfHdMeX2/SBU1vWQ eHWRyWkNF2OKn+0mOHnlSotVwzma0RYEi5fZZSSSLOixeBqJ4XFqvaFshsIctU86EIFN +4FyKghr1boi1mFPHZ0F9Qqkgd0UkxA+5XEz7qBU3pkIm2RQY+ewy9rf0Rcso8jNaU0o toDbhKb6Otus5uaA/sTjEfpylX/6RyHpjkNJwnjw/GdpCdbhbYEd6vUn98Kp6KSycKIH sAkU/gAQRkK9+UeY/BKotsOLy7EKxZPiyeDK1YSUFAZQqbEawfsbqeS1nFtbQcI6hFLc hz5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=jndW4AKC; spf=pass (google.com: domain of prvs=21101f516b=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=21101f516b=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id n3si17925157ywd.189.2019.07.25.22.52.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Jul 2019 22:52:59 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=21101f516b=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=jndW4AKC; spf=pass (google.com: domain of prvs=21101f516b=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=21101f516b=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.27/8.16.0.27) with SMTP id x6Q5qC5u013835 for ; Thu, 25 Jul 2019 22:52:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=YpPGU3ZiU++0VQgyDS3QSdQ0lQHIOIN4ED9AD9AHh/Y=; b=jndW4AKCC04H2racbu/QH+nObjPHzUAhRUY5hpwh+DjuvLEh/+O50CTcsYrb+Ri9Wwgn DtjHtmInDofnmNuig8VfsT4ieBGgohij/gOjxG2OQYV0UWXm/MvpocuH8B2tLrmAcYHN TETuNiDLvMlUujmnmCTg2IX7ekQVoRjMQ+M= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by m0001303.ppops.net with ESMTP id 2tye8mawqw-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 25 Jul 2019 22:52:58 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Thu, 25 Jul 2019 22:52:55 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 1DFB362E3049; Thu, 25 Jul 2019 22:47:18 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v9 3/4] mm, thp: introduce FOLL_SPLIT_PMD Date: Thu, 25 Jul 2019 22:46:53 -0700 Message-ID: <20190726054654.1623433-4-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190726054654.1623433-1-songliubraving@fb.com> References: <20190726054654.1623433-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-07-26_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1907260078 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patches introduces a new foll_flag: FOLL_SPLIT_PMD. As the name says FOLL_SPLIT_PMD splits huge pmd for given mm_struct, the underlining huge page stays as-is. FOLL_SPLIT_PMD is useful for cases where we need to use regular pages, but would switch back to huge page and huge pmd on. One of such example is uprobe. The following patches use FOLL_SPLIT_PMD in uprobe. Acked-by: Kirill A. Shutemov Signed-off-by: Song Liu --- include/linux/mm.h | 1 + mm/gup.c | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index f189176dabed..74db879711eb 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2614,6 +2614,7 @@ struct page *follow_page(struct vm_area_struct *vma, unsigned long address, #define FOLL_COW 0x4000 /* internal GUP flag */ #define FOLL_ANON 0x8000 /* don't do file mappings */ #define FOLL_LONGTERM 0x10000 /* mapping lifetime is indefinite: see below */ +#define FOLL_SPLIT_PMD 0x20000 /* split huge pmd before returning */ /* * NOTE on FOLL_LONGTERM: diff --git a/mm/gup.c b/mm/gup.c index 98f13ab37bac..3c514e223ce3 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -399,7 +399,7 @@ static struct page *follow_pmd_mask(struct vm_area_struct *vma, spin_unlock(ptl); return follow_page_pte(vma, address, pmd, flags, &ctx->pgmap); } - if (flags & FOLL_SPLIT) { + if (flags & (FOLL_SPLIT | FOLL_SPLIT_PMD)) { int ret; page = pmd_page(*pmd); if (is_huge_zero_page(page)) { @@ -408,7 +408,7 @@ static struct page *follow_pmd_mask(struct vm_area_struct *vma, split_huge_pmd(vma, pmd, address); if (pmd_trans_unstable(pmd)) ret = -EBUSY; - } else { + } else if (flags & FOLL_SPLIT) { if (unlikely(!try_get_page(page))) { spin_unlock(ptl); return ERR_PTR(-ENOMEM); @@ -420,6 +420,10 @@ static struct page *follow_pmd_mask(struct vm_area_struct *vma, put_page(page); if (pmd_none(*pmd)) return no_page_table(vma, flags); + } else { /* flags & FOLL_SPLIT_PMD */ + spin_unlock(ptl); + split_huge_pmd(vma, pmd, address); + ret = pte_alloc(mm, pmd); } return ret ? ERR_PTR(ret) :