From patchwork Fri Aug 12 10:12:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12942102 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8769C25B0F for ; Fri, 12 Aug 2022 10:12:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDF0D8E0003; Fri, 12 Aug 2022 06:12:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8EC28E0001; Fri, 12 Aug 2022 06:12:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D30378E0003; Fri, 12 Aug 2022 06:12:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C656F8E0001 for ; Fri, 12 Aug 2022 06:12:08 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AB15E1C5FF2 for ; Fri, 12 Aug 2022 10:12:08 +0000 (UTC) X-FDA: 79790525136.03.42AC649 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf25.hostedemail.com (Postfix) with ESMTP id 49187A018C for ; Fri, 12 Aug 2022 10:12:08 +0000 (UTC) Received: by mail-pg1-f174.google.com with SMTP id r69so447186pgr.2 for ; Fri, 12 Aug 2022 03:12:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=3RiuHL8cuPFNRdOQLrbL2ARJQLP3txMsqxB/DqBpHYo=; b=PO+YfNP7KvjF2zdmrmg/gzcou8Nbp4XOJlJjIiqFLzBP2VVWJK4AujsgLDH3VUlo41 xuxdRdR1edLV2Oixgvzo/tOfW6ht3FHrXcrb0KPTEu9xVgySElv0RJzAtXuwC+DDOAIi 21Zl59qjzNZUvkVy/dFUGG1qkbMPxjfNe++YhnttjiOdl8WZf77H3BRopKPYKfc/x7UU 732zYyCXLdqk63zY3974mToWDa13wP/rKKl2X+hpiDCLqpHiboKmEotx26W8+146voUV 5zm1r+XfHr1NYw7FgeOVDG701wh4B+fOu1ybyI3kKcFgKOewsk4VTJR++CKRUcVy5yFc tHhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=3RiuHL8cuPFNRdOQLrbL2ARJQLP3txMsqxB/DqBpHYo=; b=u1N64dKTw4YfQJlrwJS65lccFkB92jydjHWg+Ggp1lJdPfpQxmVKJGWsJIZ3Rpdka6 csbFH2QUNp30LJm8ik32WrJn9ps9+cGtIkRigKdtygsBiGsTcWIDGVBFPvMjmR5KGBIs CkbHKSN4NLkldeE2/s72Wz1HRLr7dFIazcPZQ1VhfShu6sb32qw8sco3pCWl6zF6y0qg O8ED085nhVlPaska7giDDO/ht4jmV3XS4oPS0UxOJuMivmgRKc7bVOSAWRo2l9lFrCxo +OrCiDzufCxkzhybHNan5H2KJRETtIMu5P4mAZybNfROUE7Wk1WFSD+U1Sx0V8LW/Y9U +a/w== X-Gm-Message-State: ACgBeo0GZlTNQAR2d3QDmyTRvtnYmFv40NVEN+NB40nS7M8jUz2tU4uU oTqUdoBdCDsJ9HGkcnLNFZw= X-Google-Smtp-Source: AA6agR7Fd93QkM030LuiG9mHSEbbXsNDiHX6pJcZ8w61nSyMQdFIuvFz43JGwYRt9ZJeuAcXovAWyg== X-Received: by 2002:a05:6a00:17a8:b0:52e:6e3e:9ff with SMTP id s40-20020a056a0017a800b0052e6e3e09ffmr3217306pfg.42.1660299127293; Fri, 12 Aug 2022 03:12:07 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id mp2-20020a17090b190200b001f319e9b9e5sm5134115pjb.16.2022.08.12.03.12.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 03:12:06 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org, willy@infradead.org Cc: hughd@google.com, izik.eidus@ravellosystems.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xu.xin16@zte.com.cn, CGEL Subject: [PATCH v2 2/5] ksm: implement scan-enhanced algorithm Date: Fri, 12 Aug 2022 10:12:02 +0000 Message-Id: <20220812101202.41533-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220812101102.41422-1-xu.xin16@zte.com.cn> References: <20220812101102.41422-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660299128; a=rsa-sha256; cv=none; b=09jsS+twxsv4DRruBJbZgqGpgxCjIBUYra5/VE+zif6zgXk/pLWND9pbxCsPAvuwnxOyic l7dka2LUmt6xf6pomz1rshhM6Df0XSyXrEhF4sMiSxkIJyjD/Q2ogH1+DwcfkdvUZ1QqHB p8hr0CKGuB0OMLuQ6OnSlO+7gId9IKw= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PO+YfNP7; spf=pass (imf25.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660299128; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3RiuHL8cuPFNRdOQLrbL2ARJQLP3txMsqxB/DqBpHYo=; b=FEE7mtqxWHpkNku5setUROd0AT7y2ehjBG4tuQYHHX+6HPUBHBllNJQKnCgRK4kXLtwdel xFTK1jtIO7ih6lHLQg5iqZH+MvLnprU8rahJiFWXXon9c5pHuBiyrlLjVURHhD6NDSEaVU RUsJE+SicX2cx4r12jfGmWW8aJdp5DA= X-Stat-Signature: m7cins3opwdj1rphkko6t58c4cbp6pu6 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 49187A018C Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PO+YfNP7; spf=pass (imf25.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-HE-Tag: 1660299128-647574 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin Implement the scan-enhanced algorithm of auto mode. In this algorithm, after every time of scanning, if new ksm pages are obtained, it will double pages_to_scan for the next scanning until the general multiplying factor is not less than max_scanning_factor. If no new ksm pages are obtained, then reset pages_to_scan to the default value. We add the sysfs klob of max_scanning_factor to limit scanning factor's excessive growth. Signed-off-by: CGEL Signed-off-by: xu xin --- mm/ksm.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 98 insertions(+), 2 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index c80d908221a4..f416f168a6da 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -131,6 +131,10 @@ struct mm_slot { * @address: the next address inside that to be scanned * @rmap_list: link to the next rmap to be scanned in the rmap_list * @seqnr: count of completed full scans (needed when removing unstable node) + * @new_ksmpages: count of the new merged KSM pages in the current scanning + * of mm_lists (cleared after every turn of ksm_do_scan() ends) + * @prev_ksmpages: the record of the new merged KSM pages in the last turn of + * scanning by ksm_do_scan(). * * There is only the one ksm_scan instance of this cursor structure. */ @@ -139,6 +143,8 @@ struct ksm_scan { unsigned long address; struct rmap_item **rmap_list; unsigned long seqnr; + unsigned long new_ksmpages; + unsigned long prev_ksmpages; }; /** @@ -277,6 +283,19 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; +/* + * Work in auto-mode. + * The multiplicative factor of pages_to_scan. + * Real pages to scan equals to the product of scanning_factor + * and pages_to_scan + */ +#define INIT_SCANNING_FACTOR 1 +static unsigned int scanning_factor = INIT_SCANNING_FACTOR; + +/* The upper limit of scanning_factor */ +#define DEFAULT_MAX_SCANNING_FACTOR 16 +static unsigned int max_scanning_factor = DEFAULT_MAX_SCANNING_FACTOR; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -2031,6 +2050,8 @@ static void stable_tree_append(struct rmap_item *rmap_item, rmap_item->address |= STABLE_FLAG; hlist_add_head(&rmap_item->hlist, &stable_node->hlist); + ksm_scan.new_ksmpages++; + if (rmap_item->hlist.next) ksm_pages_sharing++; else @@ -2422,6 +2443,41 @@ static int ksmd_should_run(void) return 0; } +/* + * Work in auto mode, the scan-enhanced algorithm. + * current_factor: the current scanning_factor. + * return: the scanning_factor caculated by scan-enhanced algorithm. + */ +static unsigned int scan_enhanced_algorithm(unsigned int current_factor) +{ + unsigned int next_factor; + unsigned int max, min; + + /* + * The calculation is divied into three cases as follows: + * + * Case 1: when new_ksmpages > prev_ksmpages * 1/2, get the + * next factor by double the current factor. + * Case 2: when 0 < new_ksmpages < prev_ksmpages * 1/2, keep + * the factor unchanged. + * Case 3: when new_ksmpages equals 0, then get the next + * factor by halfing the current factor. + */ + max = READ_ONCE(max_scanning_factor); + min = INIT_SCANNING_FACTOR; + if (ksm_scan.new_ksmpages * 2 > ksm_scan.prev_ksmpages) { + next_factor = current_factor << 1; /* Doubling */ + if (next_factor > max) + next_factor = max; + } else if (ksm_scan.new_ksmpages == 0) { + next_factor = current_factor >> 1; /* Halfing */ + next_factor = next_factor < min ? min : next_factor; + } else + next_factor = current_factor; + + return next_factor; +} + static int ksm_scan_thread(void *nothing) { unsigned int sleep_ms; @@ -2432,8 +2488,19 @@ static int ksm_scan_thread(void *nothing) while (!kthread_should_stop()) { mutex_lock(&ksm_thread_mutex); wait_while_offlining(); - if (ksmd_should_run()) - ksm_do_scan(ksm_thread_pages_to_scan); + if (ksmd_should_run()) { + if (ksm_run & KSM_RUN_AUTO) { + ksm_do_scan(ksm_thread_pages_to_scan * scanning_factor); + + scanning_factor = scan_enhanced_algorithm(scanning_factor); + /* + * Reset ksm_scan.new_ksmpages after + * updating scanning_factor by scan_enhanced_algorithm. + */ + ksm_scan.new_ksmpages = 0; + } else + ksm_do_scan(ksm_thread_pages_to_scan); + } mutex_unlock(&ksm_thread_mutex); try_to_freeze(); @@ -2904,6 +2971,34 @@ static ssize_t pages_to_scan_store(struct kobject *kobj, } KSM_ATTR(pages_to_scan); +static ssize_t max_scanning_factor_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%u\n", max_scanning_factor); +} + +static ssize_t max_scanning_factor_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + unsigned int value, max; + int err; + + err = kstrtouint(buf, 10, &value); + if (err) + return -EINVAL; + + max = totalram_pages() / ksm_thread_pages_to_scan; + + if (value < 1 && value > max) + return -EINVAL; + + max_scanning_factor = value; + + return count; +} +KSM_ATTR(max_scanning_factor); + static ssize_t run_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3161,6 +3256,7 @@ KSM_ATTR_RO(full_scans); static struct attribute *ksm_attrs[] = { &sleep_millisecs_attr.attr, &pages_to_scan_attr.attr, + &max_scanning_factor_attr.attr, &run_attr.attr, &pages_shared_attr.attr, &pages_sharing_attr.attr,