From patchwork Thu May 28 23:53:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 11577277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3822690 for ; Thu, 28 May 2020 23:53:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EDE33206E2 for ; Thu, 28 May 2020 23:53:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LaoKiP+c" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDE33206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1BF76800B9; Thu, 28 May 2020 19:53:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1957080010; Thu, 28 May 2020 19:53:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05C00800B9; Thu, 28 May 2020 19:53:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id DFBDD80010 for ; Thu, 28 May 2020 19:53:04 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id ABFE62C8B for ; Thu, 28 May 2020 23:53:04 +0000 (UTC) X-FDA: 76867781088.11.cream27_8f37fd8cd7c2e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 87AF6180F8B80 for ; Thu, 28 May 2020 23:53:04 +0000 (UTC) X-Spam-Summary: 2,0,0,81ba1603fe3114f7,d41d8cd98f00b204,3307qxg0kclof2jqwfxrzxxjslttlqj.htrqnsz2-rrp0fhp.twl@flex--axelrasmussen.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:973:988:989:1260:1277:1313:1314:1345:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:3138:3139:3140:3141:3142:3152:3354:3622:3865:3870:4118:4321:4385:5007:6261:6653:6737:6742:6743:7875:7903:7904:9969:10004:10400:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12555:12683:12895:12986:14096:14097:14110:14181:14394:14659:14721:21080:21444:21451:21627:21990:30054,0,RBL:209.85.219.201:@flex--axelrasmussen.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: cream27_8f37fd8cd7c2e X-Filterd-Recvd-Size: 7599 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Thu, 28 May 2020 23:53:04 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id d69so884495ybc.22 for ; Thu, 28 May 2020 16:53:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=2yUKX5hqzt1oHPQ4zNvGMcIYmUYCvNSNOVGMDvCOr3Q=; b=LaoKiP+cg9anMNFNg9Kd4pZYQAtw0FOInUmBUGQu2wKaKQMBIhjiAIspR0+jNfFIUR Wy/vpw0/UipeJjuws5qV8Ox9SMPtVfFnRuwoOi0V8A+Wocp5oYNImICPhZelh6PWwS4/ So2FoMAKYaJb/PO12JSLeHAPj6EbyM/HTreSXbWscfXxhtBb9vvaPGTHXg1Hk7AqxaVp //w3S/CyuQKBwFdks95jIwGGVJLHBtOmyW1cF22iv8nXTF4fUV8kosw00ToCNEDBxtQ7 HQ/W+gonCfXZjojOvbE9fmMDE18lM57VoyhIBV2QG//7DAKMV0qfrn4KKYJ15WbdM4u4 PiiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=2yUKX5hqzt1oHPQ4zNvGMcIYmUYCvNSNOVGMDvCOr3Q=; b=DxRTKZ+0QvTHFRfvDOYh4eUuPh/WDAwRZJo2xIYTCCOd3xNGaUgrTvJXM5/+pvX9H+ EK8FpRFWJhxjnepnHAteG0/YLRqV5M0+55EcRBUOo1yoNat2iRKTb10wMt9TxnpZSTNe w3eRpvMTq2F2WwLhgg8qg0kMtsDfVKkbhitXy2QqI2nSDbGHqufihJeWz8gUNDXQDBPS jbjlX0M+G4F24e4748jgH++4GDY76Kw1cDJSkY/QxcORRh3rJLLVccPau1zOmC48U6Ms 5Y5ft6xQG8r0/xcVkgc2NTfOp50zmtEwbL6gzTvOmiguamVcitfrWH9VR1eo1Z29kQ68 uysA== X-Gm-Message-State: AOAM533LWBRn2KWw0vUIVs+uYYhENI5L75Qi7bfAfuPuWwWvsxBjC46I reXKHISdHEyc/Xm+Ii6TlfLZeCNcbs9nZIvq0wO+ X-Google-Smtp-Source: ABdhPJxTDy0JwP7FWRktZyHchnosXxFw+yM7WVZbQCJl7DvaCtN9Tv/xctTb9glc5ZOrQcq9e7nC5UJyNd/5xZElQ7UW X-Received: by 2002:a25:e7cf:: with SMTP id e198mr820935ybh.188.1590709983105; Thu, 28 May 2020 16:53:03 -0700 (PDT) Date: Thu, 28 May 2020 16:53:00 -0700 Message-Id: <20200528235300.77135-1-axelrasmussen@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652-goog Subject: [PATCH v2 4/7] mmap_lock: allocate histogram (if enabled) in mm_init From: Axel Rasmussen To: Andrew Morton , David Rientjes , Davidlohr Bueso , Ingo Molnar , Ingo Molnar , Jerome Glisse , Laurent Dufour , "Liam R . Howlett" , Matthew Wilcox , Michel Lespinasse , Peter Zijlstra , Vlastimil Babka , Will Deacon Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, AKASHI Takahiro , Aleksa Sarai , Alexander Potapenko , Alexey Dobriyan , Al Viro , Andrei Vagin , Ard Biesheuvel , Brendan Higgins , chenqiwu , Christian Brauner , Christian Kellner , Corentin Labbe , Daniel Jordan , Dan Williams , David Gow , "David S. Miller" , "Dmitry V. Levin" , "Eric W. Biederman" , Eugene Syromiatnikov , Jamie Liu , Jason Gunthorpe , John Garry , John Hubbard , Jonathan Adams , Junaid Shahid , Kees Cook , "Kirill A. Shutemov" , Konstantin Khlebnikov , Krzysztof Kozlowski , Mark Rutland , Masahiro Yamada , Masami Hiramatsu , Mathieu Desnoyers , Michal Hocko , Mikhail Zaslonko , Petr Mladek , Ralph Campbell , Randy Dunlap , Roman Gushchin , Shakeel Butt , Steven Rostedt , Tal Gilboa , Thomas Gleixner , " =?utf-8?q?Uwe_Kleine-K=C3=B6nig?= " , Vincenzo Frascino , Yang Shi , Yu Zhao , Axel Rasmussen X-Rspamd-Queue-Id: 87AF6180F8B80 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In effect, allocate a histogram for each process. On fork, initialize a new empty histogram (don't keep recorded samples from the parent). Signed-off-by: Axel Rasmussen --- kernel/fork.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/kernel/fork.c b/kernel/fork.c index e702e84897fa..afb036d81a73 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -94,6 +94,7 @@ #include #include #include +#include #include #include @@ -975,6 +976,58 @@ __setup("coredump_filter=", coredump_filter_setup); #include +#ifdef CONFIG_MMAP_LOCK_HISTOGRAMS +static const u64 mmap_lock_contention_buckets[] = { + 1, /* 0ns - uncontended acquisitions */ + 250, /* 250 ns */ + 375, /* 375 ns */ + 500, /* 500 ns */ + 1000, /* 1 us */ + 10000, /* 10 us */ + 100000, /* 100 us */ + 500000, /* 500 us */ + 1000000, /* 1 ms */ + 5000000, /* 5 ms */ + 10000000, /* 10 ms */ + 50000000, /* 50 ms */ + 100000000, /* 100 ms */ + 500000000, /* 500 ms */ + 1000000000, /* 1 s */ + 5000000000UL, /* 5 s */ + 10000000000UL, /* 10 s */ + ~0 /* > 10s */ +}; +#endif + +static void mm_init_histograms(struct mm_struct *mm) +{ +#ifdef CONFIG_MMAP_LOCK_HISTOGRAMS + mm->mmap_lock_contention = + kmalloc(sizeof(struct histogram_rcu), GFP_KERNEL); + if (unlikely(!mm->mmap_lock_contention)) + return; + + if (unlikely(histogram_init_rcu( + mm->mmap_lock_contention, mmap_lock_contention_buckets, + ARRAY_SIZE(mmap_lock_contention_buckets)))) { + kfree(mm->mmap_lock_contention); + mm->mmap_lock_contention = NULL; + } +#endif +} + +static void mm_free_histograms(struct mm_struct *mm) +{ +#ifdef CONFIG_MMAP_LOCK_HISTOGRAMS + if (unlikely(!mm->mmap_lock_contention)) + return; + + histogram_destroy_rcu(mm->mmap_lock_contention); + kfree(mm->mmap_lock_contention); + mm->mmap_lock_contention = NULL; +#endif +} + static void mm_init_aio(struct mm_struct *mm) { #ifdef CONFIG_AIO @@ -1015,6 +1068,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, atomic_set(&mm->mm_users, 1); atomic_set(&mm->mm_count, 1); mmap_init_lock(mm); + mm_init_histograms(mm); INIT_LIST_HEAD(&mm->mmlist); mm->core_state = NULL; mm_pgtables_bytes_init(mm); @@ -1078,6 +1132,7 @@ static inline void __mmput(struct mm_struct *mm) { VM_BUG_ON(atomic_read(&mm->mm_users)); + mm_free_histograms(mm); uprobe_clear_state(mm); exit_aio(mm); ksm_exit(mm);