From patchwork Fri Jun 17 07:05:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 12885119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA075C43334 for ; Fri, 17 Jun 2022 07:06:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 458886B0071; Fri, 17 Jun 2022 03:06:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E0D36B0073; Fri, 17 Jun 2022 03:06:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25A286B0074; Fri, 17 Jun 2022 03:06:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 134C46B0071 for ; Fri, 17 Jun 2022 03:06:08 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E135160A02 for ; Fri, 17 Jun 2022 07:06:07 +0000 (UTC) X-FDA: 79586843574.12.C1C4671 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by imf30.hostedemail.com (Postfix) with ESMTP id 9054C80099 for ; Fri, 17 Jun 2022 07:06:07 +0000 (UTC) Received: by mail-pg1-f170.google.com with SMTP id s135so3315418pgs.10 for ; Fri, 17 Jun 2022 00:06:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=rtNddg8+QR7i3/tqYYzEhkG55nHCGbEucwWJ0AzfxSY=; b=Y3Ikp1g3yXZryhp2RN3/d/ZQz2VkG9Sg8fgoWH6JRf2HFKVC6tNNx9Pv7JTbKCWr6r 0xBJ77YN7pfONa+9Hs/W6SVxVtMEoQ/+9fJZT51QNovBNoK0qeWib1Fu9bKoOBv9Mvgd M3RovELRnuXDCp48MIhKVMP2IMn4RUa5a1/4wr+3i3k8J63jJf8O3eHUI9l9q/t7E3dF HEORQSEZm2AHlS5MbPd4jho75QGyPwdoMUGRilokh9EpJhHfRSxdrVaHjbuQl2VWqk+J y1f7wHrCxJC9h/X9KlBo4ooC3JaFY/58aDIMSkznEBHgZ3AwJQUpD4L7bYvrPIXK22bZ Z12A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=rtNddg8+QR7i3/tqYYzEhkG55nHCGbEucwWJ0AzfxSY=; b=SWE/H+V4iwI+v4tV+/Kb9hDGbMXjoZXMR+18vuSAf3XMflqyURsAP24EqEh2SRvobK LLCXN5tHMPkKIs5Mw+N05nO5Z9mz4BxJcBw6u3vhiDgo7YsOUtKMXXPhIca1Q+vi3y2d aTL2P7LfQMqIF2QsFIbIAdw0sxk005sS09IVLYnC2ITZfxFbNRsmPiDHyYdNe+dXX5kH n+pcNXA81FJeTpySCmyXEpeO/KdA/o9rmvYHcwbludx7y3sISUnI0y9ay6+RB/8a94EZ tHTOJJ76g4l1vAFjOsY+Mu16k2LO1Ma9IYEBLNGWwa3BWIpcnstx79wBrxtf96Utaudw hjOQ== X-Gm-Message-State: AJIora+FcYY3hyig7uO5/sm2ujMy6ryqPa9Yx1ucWipUmMPdrnovSjwQ DU+BObR/j/Ht+NlTcQ5PigM= X-Google-Smtp-Source: AGRyM1uh5RH5fUU2TfkQKcXpEKfQHxSECH3JtIM2R0gAvOptpcEOcZuljlHoaBgseEZvPx4NRx+xkA== X-Received: by 2002:a63:77cc:0:b0:408:e3bc:6294 with SMTP id s195-20020a6377cc000000b00408e3bc6294mr7747507pgc.154.1655449566551; Fri, 17 Jun 2022 00:06:06 -0700 (PDT) Received: from localhost.localdomain (47-72-206-164.dsl.dyn.ihug.co.nz. [47.72.206.164]) by smtp.gmail.com with ESMTPSA id y11-20020a170902d64b00b00161955fe0d5sm2743670plh.274.2022.06.17.00.06.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Jun 2022 00:06:06 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, torvalds@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, huzhanyuan@oppo.com, Barry Song , Yu Zhao , Will Deacon , Alex Van Brunt , Shaohua Li Subject: [RFC PATCH] mm: rmap: Don't flush TLB after checking PTE young for page reference Date: Fri, 17 Jun 2022 19:05:55 +1200 Message-Id: <20220617070555.344368-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Y3Ikp1g3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.215.170 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655449567; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=rtNddg8+QR7i3/tqYYzEhkG55nHCGbEucwWJ0AzfxSY=; b=8rEH8thfzwJ4FeHxeUaJEebA4sLEEGTHMA5sMJ0tP1kmtSFR1+78pXLQYQUCZ0I/Bw1P2V TfW4RQ12wARf+V6m++ls45yvRkL6ceYqkO+Jeu1SZt/cxMfYGQ+4Mi+dFUaNAfVcAvQIYN bBHNg2o8b0/F1yl/RCWsCTJB2WE64Fk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655449567; a=rsa-sha256; cv=none; b=kHdsuSHcfg+242PgOEAJEoz9RVzjNaWfoXWzbrjDpQMjcPC1TCTvK4SZue07EBThPSXJy6 KsjB12eosbE2kfUQjIbvFHw/xNWul89/mdCrp5+n79LK4HhYY3bQ56i0B+0du1QqGRQ/6c /qlZk7VX0s4Dy0IiTuCv+0hZ5cksc8U= X-Rspam-User: X-Stat-Signature: jrq6nzndh3etn3dbrqwxioof7da9z7hz X-Rspamd-Queue-Id: 9054C80099 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Y3Ikp1g3; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.215.170 as permitted sender) smtp.mailfrom=21cnbao@gmail.com X-Rspamd-Server: rspam05 X-HE-Tag: 1655449567-336035 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Barry Song Flushing TLB is usually quite expensive through hardware or software. Both x86 and arm64 have tried to decrease the overhead by either removing TLB flush and deferring it in ptep_clear_flush_young(). Removing the tlb flush gives about 20% ~ 30% swapout speedup on x86 according to commit b13b1d2d8692b ("x86/mm: In the PTE swapout page reclaim case clear the accessed bit instead of flushing the TLB"). Similar result was also reported on arm64 by commit 3403e56b41c1(" arm64: mm: Don't wait for completion of TLB invalidation when page aging"). While platforms like x86 and arm64 have noticed the problem and resolved it by modifying ptep_clear_flush_young() to drop flush by some means, most platforms are still doing TLB flush. In LRU, it seems pointless to do TLB broadcast simply because of update access bit. Dropping flush in general LRU code seems be a proper way than removing TLB flush in ptep_clear_flush_young() in all kind of platforms as the name of the function is implying flush should be included. Removing flush in a function who is named by flush sounds vague. So this patch moves to ptep_clear_young_notify() clearly without flush in LRU code. This will help decrease the cost of TLB broadcast due to access bit in LRU. The side effect is some minor lose in the accuracy of PTE young data, but this has been proven to be not harmful by those mainstream platforms like x86 and arm64. Cc: Yu Zhao Cc: Will Deacon Cc: Alex Van Brunt Cc: Shaohua Li Signed-off-by: Barry Song --- This RFC is inspired by the discussion in Yu Zhao's MGLRU: https://lore.kernel.org/lkml/CAOUHufYvH2LaGyAJZFQNOsGDBKD2++aFnTV6=qaVtcNrKjS_bA@mail.gmail.com/ mm/rmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/rmap.c b/mm/rmap.c index 5bcb334cd6f2..7ce6f0b6c330 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -830,7 +830,7 @@ static bool folio_referenced_one(struct folio *folio, } if (pvmw.pte) { - if (ptep_clear_flush_young_notify(vma, address, + if (ptep_clear_young_notify(vma, address, pvmw.pte)) { /* * Don't treat a reference through