From patchwork Thu Dec 7 03:09:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Yu X-Patchwork-Id: 13482644 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5291C10F04 for ; Thu, 7 Dec 2023 03:10:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CDBE6B0089; Wed, 6 Dec 2023 22:10:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 02F896B008A; Wed, 6 Dec 2023 22:10:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E12656B008C; Wed, 6 Dec 2023 22:10:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D1EC56B0089 for ; Wed, 6 Dec 2023 22:10:03 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9E564C0308 for ; Thu, 7 Dec 2023 03:10:03 +0000 (UTC) X-FDA: 81538543086.23.A12C2E8 Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) by imf03.hostedemail.com (Postfix) with ESMTP id 8B61E20008 for ; Thu, 7 Dec 2023 03:10:01 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf03.hostedemail.com: domain of xuyu@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=xuyu@linux.alibaba.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701918602; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FwDSACcA/GE3WNPhmiD7yE8NRGmZSdvnOdYpaQgUDLA=; b=H5cgRUNhPgM4uYusS6N203/YKnYd0//KEuge5Ar+z3Q41MuBMdeEaBZBrYr5eHZoE3sW+a c0yvcUFFw0IIC9fq5hLkv/ypbGT/m9mPuDYmcR4+x94gh0cQAkGNigpoLEEaFl29nZr2t9 R79Op4VW/Cm992Qp1xyr54bfJSR+50Y= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf03.hostedemail.com: domain of xuyu@linux.alibaba.com designates 115.124.30.133 as permitted sender) smtp.mailfrom=xuyu@linux.alibaba.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701918602; a=rsa-sha256; cv=none; b=MiXbQ5zcpw1f1Y/rTA9+buuPwJeN/Je6dd0+U1trzc7de0n8zOgIUeg5VwvFacPOPo+IWj QeqZ0J+47wezStfmBrRGxM+qiDIplVAMICct1dInFq8YCcCI5zlnSQk0YdJNAaGxkqMybL e9JA00iAHY1wugkGpO4gMG2qjs236d8= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R181e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046060;MF=xuyu@linux.alibaba.com;NM=1;PH=DS;RN=2;SR=0;TI=SMTPD_---0Vy-Jdtv_1701918597; Received: from localhost(mailfrom:xuyu@linux.alibaba.com fp:SMTPD_---0Vy-Jdtv_1701918597) by smtp.aliyun-inc.com; Thu, 07 Dec 2023 11:09:58 +0800 From: Xu Yu To: linux-mm@kvack.org Cc: david@redhat.com Subject: [PATCH v2 2/2] mm/khugepaged: add case for mapping anonymous pte-mapped THPs by pmds Date: Thu, 7 Dec 2023 11:09:46 +0800 Message-Id: <94575367298bc13d36937f15ac2cd1652684272f.1701917546.git.xuyu@linux.alibaba.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 8B61E20008 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: w5miy1qemg6965i14ezc81kc3cjfbri1 X-HE-Tag: 1701918601-209528 X-HE-Meta: U2FsdGVkX1+cVlgkvMl2U5TGvhY00UxXW9nKrgbDOIjZDfbQu4yao2CxQ9XDNNLeefmg2X9S9Xn23AKfxuJkghHJiignbk2UQKYONNn0cgC3wzgcmZldEK1kcjy6cC0N0OHaaYPWfCOYcOIo0ILlW1YIl89MXAP21Od0pOCH/NrlBEWoh3Ru+BCHbvhTvP/MNdJ4GIrB0jgvmazPyn7jKqGDFCtFwbMCxoUcG2dvpFkPvyggRsWrjVshLmrXCpMkqTOcW7C8tWZozEh4MSjA1jLuZp8CDhZHbsOoKtgap3ptmD0c27trK29r1vTU2oQRQz7AegFfu7oJdmcQjsiQ/OrPnwHVOwKfOL1dMXA5SsmzmUrlqdwyncEkKc7qE8mKw7Oclss/t3BsuGCUi6ovEsIt+WoiAerMdlgfpwEidfTYyQPZnDV23FhlYarZgIAkd01R5XxThb4j06w9wrJpOp/skjSzjFnvtgLbXmaN4TGidU0oYhhkKZujtk1DEwRLZmNz6+BsguB1oevgusWPPWpJQQnaczTKR/vQZfUfkAem7JGQByQSUjf4x+Fi9kkeAlaE7aoSilhqP6nVnx5byhDuw5t8v5Xxb0sbu9/TUNAxM+pH0tlC0Vd7Yn3KMSmqwatHTENnl1VsXBo4pj54IITU0FbHPG+VC9HI0Ar6/fXCZ2z1pvkhDyGMYZgoRlzFivBAvDFARj+jdgrNkuud8Yq/EFtQdHgrjXAzC/My79Xo8+nS9GPbdN0tJF4WM2h38UGCc2YX7VuU9L8uyDJ8uESqL4ffyVIPCRAOLaZUMDGupfNxJRuztDyo1wDl/G2JLRjJMXagJ4i/YnCeh7FtAdI/VRPiQXyoFM8A08An40ZxXRa1j+/n4diOQOwz0Pag4omHxVmXO17WJHJ3OJPvXcmICB7vkkCIm3yfdr2MO1ZC7IkDNGl4nVfgQPd7HbeEuxo+A+tgbGhBrviyCXw yFJ0Y2pi toW78Ep0Qgj/1TRmLDEQDOEK1I352KDaH5om8HrFaj1n4+M6yxXpZZqM6tmrQnsS03LOUD19/zr5QKK4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This adds another case, as David points out, which is suitable for mapping anonymous pte-mapped THPs by pmds. When all subpages are PageAnonExclusive (PTEs may be either R/O or R/W), we can clear PageAnonExclusive on all tail pages but the first (head) page and collapse to a R/W PMD with VM_WRITE or a R/O PMD without VM_WRITE. Signed-off-by: Xu Yu --- mm/khugepaged.c | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 85c7a2ab44ce..5c51b09cb291 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1366,8 +1366,14 @@ static int collapse_pte_mapped_anon_thp(struct mm_struct *mm, * Case 1: * No subpages are PageAnonExclusive (PTEs must be R/O), we can * collapse into a R/O PMD without further action. + * + * Case 2: + * All subpages are PageAnonExclusive (PTEs may be either R/O or R/W), + * we clear PageAnonExclusive on all tail pages but the head page and + * collapse to a R/W PMD with VM_WRITE or a R/O PMD without VM_WRITE. */ - if (!(exclusive == 0 && !writable)) + if (!((exclusive == 0 && !writable) || + (exclusive == HPAGE_PMD_NR))) goto drop_hpage; /* Collapse pmd entry */ @@ -1396,12 +1402,21 @@ static int collapse_pte_mapped_anon_thp(struct mm_struct *mm, page = vm_normal_page(vma, addr, pteval); page_remove_rmap(page, vma, false); + + if (exclusive == HPAGE_PMD_NR) + ClearPageAnonExclusive(page); } pte_unmap_unlock(start_pte, ptl); /* Install pmd entry */ pgtable = pmd_pgtable(pmdval); pmdval = mk_huge_pmd(hpage, vma->vm_page_prot); + + if (exclusive == HPAGE_PMD_NR) { + SetPageAnonExclusive(hpage); + pmdval = maybe_pmd_mkwrite(pmd_mkdirty(pmdval), vma); + } + spin_lock(pml); page_add_anon_rmap(hpage, vma, haddr, RMAP_COMPOUND); pgtable_trans_huge_deposit(mm, pmd, pgtable); @@ -1596,7 +1611,9 @@ static int hpage_collapse_scan_pmd(struct mm_struct *mm, out_unmap: pte_unmap_unlock(pte, ptl); - if (is_hpage && (exclusive == 0 && !writable)) { + if (is_hpage && + ((exclusive == 0 && !writable) || + (exclusive == HPAGE_PMD_NR))) { int res; res = collapse_pte_mapped_anon_thp(mm, vma, address,