From patchwork Thu May 7 00:42:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Yznaga X-Patchwork-Id: 11532167 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF01281 for ; Thu, 7 May 2020 00:45:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7DA1821655 for ; Thu, 7 May 2020 00:45:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="NaEwX/P0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7DA1821655 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0EF5490001C; Wed, 6 May 2020 20:45:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 07A4390001B; Wed, 6 May 2020 20:45:17 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC0D090001C; Wed, 6 May 2020 20:45:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0019.hostedemail.com [216.40.44.19]) by kanga.kvack.org (Postfix) with ESMTP id B3135900003 for ; Wed, 6 May 2020 20:45:17 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 83680181AEF30 for ; Thu, 7 May 2020 00:45:17 +0000 (UTC) X-FDA: 76788079074.19.cable31_2e257af74ff2f X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,anthony.yznaga@oracle.com,,RULES_HIT:4423:30034:30054:30064:30069:30079,0,RBL:156.151.31.86:@oracle.com:.lbl8.mailshell.net-64.10.201.10 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: cable31_2e257af74ff2f X-Filterd-Recvd-Size: 9191 Received: from userp2130.oracle.com (userp2130.oracle.com [156.151.31.86]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Thu, 7 May 2020 00:45:16 +0000 (UTC) Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 0470dTvg076311; Thu, 7 May 2020 00:44:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=J/IiCj+3kST83E/VcM8iyZaL98ROPYAwK19raHrmTZw=; b=NaEwX/P0ZZI39WJCV8xbxOOGMpPaIk3KdKPqmf8xqw4W7aDYRJW8HFEbpNiMORAFOvdg Pw0JMz+eZIw1ILS1MUToUdL98fMRvPTPn5RInXZTH3CXNINmo4dd871gbwXTMtprIi5c OY/w/M9UUV003jvnhTMeGaLUnH/GW8q+W3Ps1wOZYgvxGqADYIXsF1ml9k2aP35nIE65 AJqv83Hztpww5XQcAIl/iTQk4+71FS6FE5+fB/xoAxqaagg+rcsnPUwprV29UckCzZ3Z 29bXAE5twy3CU1lwfYpHznoWgzK+Pd7y9lXfIJChI/rgDlTz1AoTZKiuLeSxgkiiZcSB Hw== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 30s09rdfcg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 07 May 2020 00:44:42 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 0470bTQT098629; Thu, 7 May 2020 00:44:41 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3020.oracle.com with ESMTP id 30sjnma5t8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 07 May 2020 00:44:41 +0000 Received: from abhmp0014.oracle.com (abhmp0014.oracle.com [141.146.116.20]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 0470ibRQ026192; Thu, 7 May 2020 00:44:37 GMT Received: from ayz-linux.localdomain (/68.7.158.207) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 06 May 2020 17:44:36 -0700 From: Anthony Yznaga To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: willy@infradead.org, corbet@lwn.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, rppt@linux.ibm.com, akpm@linux-foundation.org, hughd@google.com, ebiederm@xmission.com, masahiroy@kernel.org, ardb@kernel.org, ndesaulniers@google.com, dima@golovin.in, daniel.kiper@oracle.com, nivedita@alum.mit.edu, rafael.j.wysocki@intel.com, dan.j.williams@intel.com, zhenzhong.duan@oracle.com, jroedel@suse.de, bhe@redhat.com, guro@fb.com, Thomas.Lendacky@amd.com, andriy.shevchenko@linux.intel.com, keescook@chromium.org, hannes@cmpxchg.org, minchan@kernel.org, mhocko@kernel.org, ying.huang@intel.com, yang.shi@linux.alibaba.com, gustavo@embeddedor.com, ziqian.lzq@antfin.com, vdavydov.dev@gmail.com, jason.zeng@intel.com, kevin.tian@intel.com, zhiyuan.lv@intel.com, lei.l.li@intel.com, paul.c.lai@intel.com, ashok.raj@intel.com, linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, kexec@lists.infradead.org Subject: [RFC 34/43] shmem: PKRAM: multithread preserving and restoring shmem pages Date: Wed, 6 May 2020 17:42:00 -0700 Message-Id: <1588812129-8596-35-git-send-email-anthony.yznaga@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1588812129-8596-1-git-send-email-anthony.yznaga@oracle.com> References: <1588812129-8596-1-git-send-email-anthony.yznaga@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9613 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 adultscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 malwarescore=0 spamscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2005070001 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9613 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 lowpriorityscore=0 spamscore=0 adultscore=0 clxscore=1015 suspectscore=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2005070001 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Improve performance by multithreading the work to preserve and restore shmem pages. Add 'pkram_max_threads=' kernel option to specify the maximum number of threads to use to preserve or restore the pages of a shmem file. The default is 16. When preserving pages each thread saves chunks of a file to a pkram_obj until no more no more chunks are available. When restoring pages each thread loads pages using a copy of a pkram_stream initialized by pkram_prepare_load_obj(). Under the hood each thread ends up fetching and operating on pkram_link pages. Signed-off-by: Anthony Yznaga --- include/linux/pkram.h | 2 + mm/shmem_pkram.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 101 insertions(+), 2 deletions(-) diff --git a/include/linux/pkram.h b/include/linux/pkram.h index e71ccb91d6a6..bf2e138b044e 100644 --- a/include/linux/pkram.h +++ b/include/linux/pkram.h @@ -13,6 +13,8 @@ struct pkram_stream { struct pkram_node *node; struct pkram_obj *obj; + int error; + struct pkram_link *link; /* current link */ unsigned int entry_idx; /* next entry in link */ diff --git a/mm/shmem_pkram.c b/mm/shmem_pkram.c index 2f4d0bdf3e05..4992b6c3e54e 100644 --- a/mm/shmem_pkram.c +++ b/mm/shmem_pkram.c @@ -126,6 +126,16 @@ static int save_file_content_range(struct address_space *mapping, return err; } +/* Completion tracking for do_save_file_content_thr() threads */ +static atomic_t pkram_save_n_undone; +static DECLARE_COMPLETION(pkram_save_all_done_comp); + +static inline void pkram_save_report_one_done(void) +{ + if (atomic_dec_and_test(&pkram_save_n_undone)) + complete(&pkram_save_all_done_comp); +} + static int do_save_file_content(struct pkram_stream *ps) { int ret; @@ -142,11 +152,55 @@ static int do_save_file_content(struct pkram_stream *ps) return ret; } +static int do_save_file_content_thr(void *data) +{ + struct pkram_stream *ps = data; + struct pkram_stream pslocal = *ps; + int ret; + + ret = do_save_file_content(&pslocal); + if (ret && !ps->error) + ps->error = ret; + + pkram_save_report_one_done(); + return 0; +} +#define PKRAM_DEFAULT_MAX_THREADS 16 + +static int pkram_max_threads = PKRAM_DEFAULT_MAX_THREADS; + +static int __init set_pkram_max_threads(char *str) +{ + unsigned int val; + + if (kstrtouint(str, 0, &val)) + return 1; + + pkram_max_threads = val; + + return 1; +} +__setup("pkram_max_threads=", set_pkram_max_threads); + static int save_file_content(struct pkram_stream *ps) { + unsigned int thr, nr_threads; + + nr_threads = num_online_cpus() - 1; + nr_threads = clamp_val(pkram_max_threads, 1, nr_threads); + ps->max_idx = DIV_ROUND_UP(i_size_read(ps->mapping->host), PAGE_SIZE); - return do_save_file_content(ps); + if (nr_threads == 1) + return do_save_file_content(ps); + + atomic_set(&pkram_save_n_undone, nr_threads); + for (thr = 0; thr < nr_threads; thr++) + kthread_run(do_save_file_content_thr, ps, "pkram_save%d", thr); + + wait_for_completion(&pkram_save_all_done_comp); + + return ps->error; } static int save_file(struct dentry *dentry, struct pkram_stream *ps) @@ -248,7 +302,17 @@ int shmem_save_pkram(struct super_block *sb) return err; } -static int load_file_content(struct pkram_stream *ps) +/* Completion tracking for do_load_file_content_thr() threads */ +static atomic_t pkram_load_n_undone; +static DECLARE_COMPLETION(pkram_load_all_done_comp); + +static inline void pkram_load_report_one_done(void) +{ + if (atomic_dec_and_test(&pkram_load_n_undone)) + complete(&pkram_load_all_done_comp); +} + +static int do_load_file_content(struct pkram_stream *ps) { unsigned long index; struct page *page; @@ -266,6 +330,39 @@ static int load_file_content(struct pkram_stream *ps) return err; } +static int do_load_file_content_thr(void *data) +{ + struct pkram_stream *ps = data; + struct pkram_stream pslocal = *ps; + int ret; + + ret = do_load_file_content(&pslocal); + if (ret && !ps->error) + ps->error = ret; + + pkram_load_report_one_done(); + return 0; +} + +static int load_file_content(struct pkram_stream *ps) +{ + unsigned int thr, nr_threads; + + nr_threads = num_online_cpus() - 1; + nr_threads = clamp_val(pkram_max_threads, 1, nr_threads); + + if (nr_threads == 1) + return do_load_file_content(ps); + + atomic_set(&pkram_load_n_undone, nr_threads); + for (thr = 0; thr < nr_threads; thr++) + kthread_run(do_load_file_content_thr, ps, "pkram_load%d", thr); + + wait_for_completion(&pkram_load_all_done_comp); + + return ps->error; +} + static int load_file(struct dentry *parent, struct pkram_stream *ps, char *buf, size_t bufsize) {