From patchwork Thu May 28 23:52:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 11577269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F0F8F90 for ; Thu, 28 May 2020 23:52:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AFF4E2075F for ; Thu, 28 May 2020 23:52:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="m3HnaNZO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AFF4E2075F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DA4D18001A; Thu, 28 May 2020 19:52:45 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D55DA80010; Thu, 28 May 2020 19:52:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C1D668001A; Thu, 28 May 2020 19:52:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id AB3BD80010 for ; Thu, 28 May 2020 19:52:45 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 62F802496 for ; Thu, 28 May 2020 23:52:45 +0000 (UTC) X-FDA: 76867780290.03.baby22_8c69c4b073c1f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 4470228A4E8 for ; Thu, 28 May 2020 23:52:45 +0000 (UTC) X-Spam-Summary: 2,0,0,62d036e5e530c891,d41d8cd98f00b204,3y07qxg0kckygdkrxgysayyktmuumrk.iusrotad-ssqbgiq.uxm@flex--axelrasmussen.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:967:968:973:988:989:1260:1277:1313:1314:1345:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:1801:2393:2525:2559:2564:2682:2685:2859:2911:2912:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3354:3743:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4250:4321:4425:4605:5007:6261:6653:6737:6742:6743:7875:7903:7904:8908:9025:9969:10004:10400:10903:11026:11232:11658:11914:12043:12048:12257:12291:12296:12297:12438:12555:12679:12683:12895:13137:13150:13161:13229:13230:13231:13869:14039:14096:14097:14181:14394:14659:14721:21080:21324:21444:21451:21627:21740:21788:30005:30012:30034:30054:30070,0,RBL:209.85.219.202:@flex--axelrasmussen.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100,CacheIP:none,Bayesian: 0.5,0.5, X-HE-Tag: baby22_8c69c4b073c1f X-Filterd-Recvd-Size: 7973 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Thu, 28 May 2020 23:52:44 +0000 (UTC) Received: by mail-yb1-f202.google.com with SMTP id f130so926350yba.9 for ; Thu, 28 May 2020 16:52:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=PLCUzj5PnnceGXj3+QD8e5tY1RCBcE3OsFzkcSuFg6M=; b=m3HnaNZOl+tSHhl7t5o3rvY68ZjkKMh+Q+wDk423m3JAa2vvjXH0jhVqjxY+3VvOoq 0UCCyQLVRrhoDle9lx75mrRVXUn1QLPO7CVt1gFsg1tMN+QFbEnB5NcrD/dAEkdGOO+6 ePfvgiYwwuz6XH482uL/Hjkfx3U1wymgh45uVOVaHaAPvECtmzod5H7LEgfnE0SF5zYs yOccLpvtkQAGUm8lwDyX2iDjOXDkdwBwnFT2zZBxG5NnfR0mJHuksgLoyIi2pEfjk0HP 0jsEzwXI3nxlQXHMCWDDb/BNnoNT0c8TwfJI5ylPMtQ99akdUKV+dOKBIqXQo4W80QW+ d1Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=PLCUzj5PnnceGXj3+QD8e5tY1RCBcE3OsFzkcSuFg6M=; b=g1Ault2iGcFPrZHyVZaOwiYo2LB+JoPJy+Y29o8Fv3r2oqoPSiziAwIjUIzEYt+V1y r8UJTjdOFn82Y6fQReAh7yRPZnxiPRUXeG/t/LO1RdD4FgByjvUoCSCZ5FsW0bext9qB vnfGcsB/gKEWhL9RXoCVJpa9o3Ly8r46zD3Hzsz4HTBuAjaK4Df0gvikR9TKKLrLUXOD uTfyqTkMmwGHR9jtNqJzMtsIbpC0hF9xnKZT4EJhitAS1rPBNwoYapimecxdqIBptSep g1ttxLb5QF1D/qLuNrgxsC3BYV9rn1fbxOyVWu+qyP4jy0uxmh4m0NKu3KRT20Rs4vex ZT9A== X-Gm-Message-State: AOAM530YMj1V1Yu58fqL+NbAqwcppCvV07TDsiXqxNKBzleE3/Ft6OnG gnLr5//y8reV1lRpkeMmlK49tb8GM5PbpzDirDiE X-Google-Smtp-Source: ABdhPJyfkvStXNh/Yb1rvcJn8bGxQZLI/kp/fuH3e4D0yr3elo+M1CUYFIiuSZVGUyDjNHkMVEGQUPJjRC9rBMiZbkCv X-Received: by 2002:a25:9344:: with SMTP id g4mr8667405ybo.157.1590709963856; Thu, 28 May 2020 16:52:43 -0700 (PDT) Date: Thu, 28 May 2020 16:52:38 -0700 Message-Id: <20200528235238.74233-1-axelrasmussen@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652-goog Subject: [PATCH v2 0/7] Add histogram measuring mmap_lock contention latency From: Axel Rasmussen To: Andrew Morton , David Rientjes , Davidlohr Bueso , Ingo Molnar , Ingo Molnar , Jerome Glisse , Laurent Dufour , "Liam R . Howlett" , Matthew Wilcox , Michel Lespinasse , Peter Zijlstra , Vlastimil Babka , Will Deacon Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, AKASHI Takahiro , Aleksa Sarai , Alexander Potapenko , Alexey Dobriyan , Al Viro , Andrei Vagin , Ard Biesheuvel , Brendan Higgins , chenqiwu , Christian Brauner , Christian Kellner , Corentin Labbe , Daniel Jordan , Dan Williams , David Gow , "David S. Miller" , "Dmitry V. Levin" , "Eric W. Biederman" , Eugene Syromiatnikov , Jamie Liu , Jason Gunthorpe , John Garry , John Hubbard , Jonathan Adams , Junaid Shahid , Kees Cook , "Kirill A. Shutemov" , Konstantin Khlebnikov , Krzysztof Kozlowski , Mark Rutland , Masahiro Yamada , Masami Hiramatsu , Mathieu Desnoyers , Michal Hocko , Mikhail Zaslonko , Petr Mladek , Ralph Campbell , Randy Dunlap , Roman Gushchin , Shakeel Butt , Steven Rostedt , Tal Gilboa , Thomas Gleixner , " =?utf-8?q?Uwe_Kleine-K=C3=B6nig?= " , Vincenzo Frascino , Yang Shi , Yu Zhao , Axel Rasmussen X-Rspamd-Queue-Id: 4470228A4E8 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The overall goal of this patchset is to add a latency histogram which measures `mmap_lock` acquisition time. This is useful to measure the impact of ongoing work like maple trees and range locks (https://lwn.net/Articles/787629/), and it is also useful to debug userspace processes which experience long waits due to lock contention. This patchset is built upon walken@google.com's new `mmap_lock` API (https://lkml.org/lkml/2020/4/21/1307). In its current form, it should apply cleanly to a 5.7-rc7 tree to which Michel's patchset has already been applied. To summarize the changes being made at a high level: - Add a histogram library: a `struct histogram` is effectively an array of thresholds (i.e., buckets), and an array of per-cpu `u64` counts of the number of samples in each bucket. - Modify Michel's mmap_lock API to record samples in a histogram, owned by the `mm_struct`, on each lock acquisition. For contended lock acquisitions, we compute the amount of time spent waiting, which determines the bucket. - For uncontended cases, we still record a sample, but with "0" latency. The reasoning for this is, a) we don't want to incur the overhead of actually measuring the time, but b) we still want to end up with an accurate count of acquisition attempts, as this lets us compute latency percentiles (e.g., "x% of lock acquisitions completed in <= y ns"). Changes since v1 (sent to a few folks within Google for initial review): - Added a tracepoint to the contended case. - Modified `mmap_write_lock_nested` to split the {un,}contended cases. - Removed support for having more than one histogram in `mm_struct`. - Removed any histogram code not explicitly used in this patchset. - Whitespace cleanups. Axel Rasmussen (7): histogram: add struct histogram histogram: add helper function to expose histograms to userspace mmap_lock: add a histogram structure to struct mm_struct mmap_lock: allocate histogram (if enabled) in mm_init mmap_lock: add /proc//mmap_lock_contention interface mmap_lock: increment histogram whenever mmap_lock is acquired mmap_lock: add a tracepoint to contended acquisitions fs/proc/base.c | 25 +++ include/linux/histogram.h | 293 +++++++++++++++++++++++++++++++ include/linux/mm_types.h | 11 ++ include/linux/mmap_lock.h | 92 +++++++++- include/trace/events/mmap_lock.h | 34 ++++ kernel/fork.c | 55 ++++++ kernel/locking/rwsem.c | 4 +- lib/Kconfig | 3 + lib/Makefile | 2 + lib/histogram.c | 212 ++++++++++++++++++++++ mm/Kconfig | 13 ++ mm/Makefile | 1 + mm/mmap_lock.c | 46 +++++ 13 files changed, 782 insertions(+), 9 deletions(-) create mode 100644 include/linux/histogram.h create mode 100644 include/trace/events/mmap_lock.h create mode 100644 lib/histogram.c create mode 100644 mm/mmap_lock.c --- 2.27.0.rc0.183.gde8f92d652-goog