From patchwork Mon Aug 8 14:56:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Aaron Lu X-Patchwork-Id: 12938858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56EFAC00140 for ; Mon, 8 Aug 2022 14:57:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D60C8E0001; Mon, 8 Aug 2022 10:57:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8854C6B0072; Mon, 8 Aug 2022 10:57:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74CEF8E0001; Mon, 8 Aug 2022 10:57:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5EF7A6B0071 for ; Mon, 8 Aug 2022 10:57:35 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2AB76140EE4 for ; Mon, 8 Aug 2022 14:57:35 +0000 (UTC) X-FDA: 79776729270.31.8ABF845 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf01.hostedemail.com (Postfix) with ESMTP id DD0B840171 for ; Mon, 8 Aug 2022 14:57:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1659970653; x=1691506653; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=CGF9ZvLWxny2IyMvhFj9EY7L6o3JSecFMpYExyr+FLo=; b=M9JNaku+ldOhTMtXuy7FID4NC47FO06OgtXXl12pKRdgcmE8fahtnreK naK43dgNdYlSgicdYDSvuu3mUs2wymjCxrRIngqF3HqUiE6Np+LGK837j ROUUYQYHLFC4u2U7o9RYQ2rsv8GwEMNBVInj36p2Nj6Fk8SxluEIKd4+V NlmvYiuce1HPRYlDE8tRt5fJNgrC8blwBBCe8+utZDjMUHDJpOkuawYdE FhFcqBr4OoqoSk/5g7l8uzHNy2EqOuUnSy1EogklWicthaOF3lDlWOfNB zprhs5S+HWIMnWz80sL2DSSA+mRnUK7DjtkiXC1Yir3pKkYU6FgkSkTGA Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10433"; a="376904732" X-IronPort-AV: E=Sophos;i="5.93,222,1654585200"; d="scan'208";a="376904732" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2022 07:57:32 -0700 X-IronPort-AV: E=Sophos;i="5.93,222,1654585200"; d="scan'208";a="663980478" Received: from ziqianlu-desk2.sh.intel.com ([10.238.2.76]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2022 07:57:30 -0700 From: Aaron Lu To: Dave Hansen , Rick Edgecombe Cc: Song Liu , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 0/4] x86/mm/cpa: merge small mappings whenever possible Date: Mon, 8 Aug 2022 22:56:45 +0800 Message-Id: <20220808145649.2261258-1-aaron.lu@intel.com> X-Mailer: git-send-email 2.37.1 MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659970654; a=rsa-sha256; cv=none; b=iQVmWMj8+sJIbwVx+z2zJfdcTFYjUqm73eYB4NiBh5z68Hmtn4KiT+xt2XkIrO75g0/PM0 PhETrAolSoP5fiAdCutal8f7QtKsJCZzY+TwSsJgKpVkwLZcIrZxKATJ/Y6/He32mUYadg auGEzufIzoW5x4vRrcxvBcP9A4u1fso= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659970654; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=cRiTNbTQv7UDrmmvueTEouJOsGEh3qe5gepeIAUi7bU=; b=MAV0D8IXwhWXRmbsU2Sye5hRc6jUknxKUQ6VkkhGrZwXdWIqycXoYzYa/nhHKs9HDl/gvu qt5RtlQNKfwjlMNBR7xxGZs+b1SwNfvI7qvX1fOtU1I93PTKyTadO51y4zILXuaEZVzulO feOx42VXbzYHxtC9LVcZACwHj96lYws= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=M9JNaku+; spf=pass (imf01.hostedemail.com: domain of aaron.lu@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=aaron.lu@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Stat-Signature: mox1mxrsfebe4556goq44tnffwysf9r7 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DD0B840171 Authentication-Results: imf01.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=M9JNaku+; spf=pass (imf01.hostedemail.com: domain of aaron.lu@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=aaron.lu@intel.com; dmarc=pass (policy=none) header.from=intel.com X-HE-Tag: 1659970653-21205 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is an early RFC. While all reviews are welcome, reviewing this code now will be a waste of time for the x86 subsystem maintainers. I would, however, appreciate a preliminary review from the folks on the to and cc list. I'm posting it to the list in case anyone else is interested in seeing this early version. Dave Hansen: I need your ack before this goes to the maintainers. Here it goes: On x86_64, Linux has direct mapping of almost all physical memory. For performance reasons, this mapping is usually set as large page like 2M or 1G per hardware's capability with read, write and non-execute protection. There are cases where some pages have to change their protection to RO and eXecutable, like pages that host module code or bpf prog. When these pages' protection are changed, the corresponding large mapping that cover these pages will have to be splitted into 4K first and then individual 4k page's protection changed accordingly, i.e. unaffected pages keep their original protection as RW and NX while affected pages' protection changed to RO and X. There is a problem due to this split: the large mapping will remain splitted even after the affected pages' protection are changed back to RW and NX, like when the module is unloaded or bpf progs are freed. After system runs a long time, there can be more and more large mapping being splitted, causing more and more dTLB misses and overall system performance getting hurt[1]. For this reason, people tried some techniques to reduce the harm of large mapping beling splitted, like bpf_prog_pack[2] which packs multiple bpf progs into a single page instead of allocating and changing one page's protection for each bpf prog. This approach made large mapping split happen much fewer. This patchset addresses this problem in another way: it merges splitted mappings back to a large mapping when protections of all entries of the splitted small mapping page table become same again, e.g. when the page whose protection was changed to RO+X now has its protection changed back to RW+NX due to reasons like module unload, bpf prog free, etc. and all other entries' protection are also RW+NX. One final note is, with features like bpf_prog_pack etc., there can be much fewer large mapping split IIUC; also, this patchset can not help when the page which has its protection changed keeps in use. So my take on this large mapping split problem is: to get the most value of keeping large mapping intact, features like bpf_prog_pack is important. This patchset can help to further reduce large mapping split when in use page that has special protection set finally gets released. [1]: http://lkml.kernel.org/r/CAPhsuW4eAm9QrAxhZMJu-bmvHnjWjuw86gFZzTHRaMEaeFhAxw@mail.gmail.com [2]: https://lore.kernel.org/lkml/20220204185742.271030-1-song@kernel.org/ Aaron Lu (4): x86/mm/cpa: restore global bit when page is present x86/mm/cpa: merge splitted direct mapping when possible x86/mm/cpa: add merge event counter x86/mm/cpa: add a test interface to split direct map arch/x86/mm/pat/set_memory.c | 411 +++++++++++++++++++++++++++++++++- include/linux/mm_types.h | 6 + include/linux/page-flags.h | 6 + include/linux/vm_event_item.h | 2 + mm/vmstat.c | 2 + 5 files changed, 420 insertions(+), 7 deletions(-)