From patchwork Sun Jul 11 10:41:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leo Yan X-Patchwork-Id: 12369105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5686C07E96 for ; Sun, 11 Jul 2021 10:51:33 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 89FC061179 for ; Sun, 11 Jul 2021 10:51:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 89FC061179 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=B+p/3nGSAHj7znlY9NZNInWBorvZNJW4BLNtb5VNl0I=; b=uYDoovKUcmiL7f CCT4qmQmpiPNKS5mx9rL9sKBUkSRwGI3o4v4XqXSo7fR74lFjkaSZJimXgNBCg21I+1CobLs4YCPu cSi8oYZ2PflBWykifrRzdDHv2G0A0RsbsO0IygCWqPK7/knCl3u5KgAi4ugCmc2R+JZB4mpg66m8n 6VnHJBEzR21OgrGBj+QBaNXjg6iAJWhPoTwAKLniIyBQd4UrVY1bZzju5sl3hqozhQkRAkzsMV5f+ j5NqtW7ssM5mbEdswKZPK8y2ZuNE8fql4mIT67BfVeulLNYcunwztLSduM4ovKofGea037SGito69 UVewd5GD2OJLGFUKSuAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m2X1l-004vDl-7a; Sun, 11 Jul 2021 10:49:38 +0000 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m2WuD-004sOM-Dw for linux-arm-kernel@lists.infradead.org; Sun, 11 Jul 2021 10:41:51 +0000 Received: by mail-pg1-x530.google.com with SMTP id h4so14969322pgp.5 for ; Sun, 11 Jul 2021 03:41:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r/tGyqhrBoK91o4eCemyjadg6GlA9KSC6VamX5cgheU=; b=eFEcv+8r6OlhUaVPY7jy/IUCvqioh3wDxBp8UB6a6cxPKtq2AdIghdUkDBI7pMTDI3 TO7wKKfEWqmjQyPzh1I4DEb30+sVaHqfj09ksvynYZUN/MzRCXr85cfRBwcvxLLUUtOC /dEn3LUrOo/nwNUROrdfcv+59/VVU7n/na/fhM2NQyLGzuu6Ew2c4VmIHf5Pq3yGX9tL FNPEcwlIo62hlmuLXdC43OE5EISgH4eEuH1xQJl12c6awQWoU8pIDHbhlvXoHtFed/9O CTdA0E+i+/5szy48J0ZDSi+/EBH9urStYc3kjpb/c+uocuCAcE8wadWo35FWION6VQlp XXuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r/tGyqhrBoK91o4eCemyjadg6GlA9KSC6VamX5cgheU=; b=mIoZojGN/Iar2lkdvDsJS0Y6qHIq0RPaeTd5xfC4LQRjUJNxLcTWbe02fr3NxCnion P3FA9bLAKtz95W9X7weuogMu7Pp7o9B7iKDa7oyFXMkvqOwgFdShf7n9YpjQTG3wlKzX dB0FEczqDIBNDbCn9bpMksECvSqGsOWqLrUuS9yjw5Cnso9MAoFypZPa8jYBVK7jFji1 KliiriyvCCMAw+4+WB3H/rE62np8K/IwxDd3zMnxO0bGefl8DswYpJ4IoDBQFVgYIkd9 NIBcjQ7lxz2+pgK0djU9MZ91OPSxGUWmlZXwoIv+r1zBx5Z1I4QSchYi/VjpFIoeUkO2 olGQ== X-Gm-Message-State: AOAM533SuCLjC6EEsrAUfU3Em1zg1LfiAkJcBn6zfPls7U0CpqM5idB5 RX7fkUL8chTe9W90S8YGTzv7jg== X-Google-Smtp-Source: ABdhPJwAoXkLeUcaQpHcDt4kwAI6BVMAiNJjmO8p0o2fJKXs7tVEnzR/EhPW0ZB3udHnBB5QMHLlNw== X-Received: by 2002:a63:515f:: with SMTP id r31mr47275292pgl.406.1626000108577; Sun, 11 Jul 2021 03:41:48 -0700 (PDT) Received: from localhost ([103.127.241.250]) by smtp.gmail.com with ESMTPSA id s33sm1628476pfw.158.2021.07.11.03.41.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Jul 2021 03:41:48 -0700 (PDT) From: Leo Yan To: Arnaldo Carvalho de Melo , Adrian Hunter , Peter Zijlstra , Ingo Molnar , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Thomas Gleixner , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" , Mathieu Poirier , Suzuki K Poulose , Mike Leach , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, coresight@lists.linaro.org, linux-arm-kernel@lists.infradead.org Cc: Leo Yan Subject: [PATCH v4 11/11] perf auxtrace: Add compat_auxtrace_mmap__{read_head|write_tail} Date: Sun, 11 Jul 2021 18:41:05 +0800 Message-Id: <20210711104105.505728-12-leo.yan@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210711104105.505728-1-leo.yan@linaro.org> References: <20210711104105.505728-1-leo.yan@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210711_034149_535545_AAC97708 X-CRM114-Status: GOOD ( 31.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When perf runs in compat mode (kernel in 64-bit mode and the perf is in 32-bit mode), the 64-bit value atomicity in the user space cannot be assured, E.g. on some architectures, the 64-bit value accessing is split into two instructions, one is for the low 32-bit word accessing and another is for the high 32-bit word. This patch introduces two functions compat_auxtrace_mmap__read_head() and compat_auxtrace_mmap__write_tail(), as their naming indicates, when perf tool works in compat mode, it uses these two functions to access the AUX head and tail. These two functions can allow the perf tool to work properly in certain conditions, e.g. when perf tool works in snapshot mode with only using AUX head pointer, or perf tool uses the AUX buffer and the incremented tail is not bigger than 4GB. When perf tool cannot handle the case when the AUX tail is bigger than 4GB, the function compat_auxtrace_mmap__write_tail() returns -1 and tells the caller to bail out for the error. Suggested-by: Adrian Hunter Signed-off-by: Leo Yan --- tools/perf/util/auxtrace.c | 9 ++-- tools/perf/util/auxtrace.h | 94 +++++++++++++++++++++++++++++++++++++- 2 files changed, 98 insertions(+), 5 deletions(-) diff --git a/tools/perf/util/auxtrace.c b/tools/perf/util/auxtrace.c index 6a63be8b2430..d6fc250fbf97 100644 --- a/tools/perf/util/auxtrace.c +++ b/tools/perf/util/auxtrace.c @@ -1766,10 +1766,13 @@ static int __auxtrace_mmap__read(struct mmap *map, mm->prev = head; if (!snapshot) { - auxtrace_mmap__write_tail(mm, head); - if (itr->read_finish) { - int err; + int err; + err = auxtrace_mmap__write_tail(mm, head); + if (err < 0) + return err; + + if (itr->read_finish) { err = itr->read_finish(itr, mm->idx); if (err < 0) return err; diff --git a/tools/perf/util/auxtrace.h b/tools/perf/util/auxtrace.h index d68a5e80b217..66de7b6e65ec 100644 --- a/tools/perf/util/auxtrace.h +++ b/tools/perf/util/auxtrace.h @@ -18,6 +18,8 @@ #include #include +#include "env.h" + union perf_event; struct perf_session; struct evlist; @@ -440,23 +442,111 @@ struct auxtrace_cache; #ifdef HAVE_AUXTRACE_SUPPORT +/* + * In the compat mode kernel runs in 64-bit and perf tool runs in 32-bit mode, + * 32-bit perf tool cannot access 64-bit value atomically, which might lead to + * the issues caused by the below sequence on multiple CPUs: when perf tool + * accesses either the load operation or the store operation for 64-bit value, + * on some architectures the operation is divided into two instructions, one + * is for accessing the low 32-bit value and another is for the high 32-bit; + * thus these two user operations can give the kernel chances to access the + * 64-bit value, and thus leads to the unexpected load values. + * + * kernel (64-bit) user (32-bit) + * + * if (LOAD ->aux_tail) { --, LOAD ->aux_head_lo + * STORE $aux_data | ,---> + * FLUSH $aux_data | | LOAD ->aux_head_hi + * STORE ->aux_head --|-------` smp_rmb() + * } | LOAD $data + * | smp_mb() + * | STORE ->aux_tail_lo + * `-----------> + * STORE ->aux_tail_hi + * + * For this reason, it's impossible for the perf tool to work correctly when + * the AUX head or tail is bigger than 4GB (more than 32 bits length); and we + * can not simply limit the AUX ring buffer to less than 4GB, the reason is + * the pointers can be increased monotonically (e.g in snapshot mode), whatever + * the buffer size it is, at the end the head and tail can be bigger than 4GB + * and carry out to the high 32-bit. + * + * To mitigate the issues and improve the user experience, we can allow the + * perf tool working in certain conditions and bail out with error if detect + * any overflow cannot be handled. + * + * For reading the AUX head, it reads out the values for three times, and + * compares the high 4 bytes of the values between the first time and the last + * time, if there has no change for high 4 bytes injected by the kernel during + * the user reading sequence, it's safe for use the second value. + * + * When update the AUX tail and detects any carrying in the high 32 bits, it + * means there have two store operations in user space and it cannot promise + * the atomicity for 64-bit write, so return '-1' in this case to tell the + * caller an overflow error has happened. + */ +static inline u64 compat_auxtrace_mmap__read_head(struct auxtrace_mmap *mm) +{ + struct perf_event_mmap_page *pc = mm->userpg; + u64 first, second, last; + u64 mask = (u64)(UINT32_MAX) << 32; + + do { + first = READ_ONCE(pc->aux_head); + /* Ensure all reads are done after we read the head */ + smp_rmb(); + second = READ_ONCE(pc->aux_head); + /* Ensure all reads are done after we read the head */ + smp_rmb(); + last = READ_ONCE(pc->aux_head); + } while ((first & mask) != (last & mask)); + + return second; +} + +static inline int compat_auxtrace_mmap__write_tail(struct auxtrace_mmap *mm, + u64 tail) +{ + struct perf_event_mmap_page *pc = mm->userpg; + u64 mask = (u64)(UINT32_MAX) << 32; + + if (tail & mask) + return -1; + + /* Ensure all reads are done before we write the tail out */ + smp_mb(); + WRITE_ONCE(pc->aux_tail, tail); + return 0; +} + static inline u64 auxtrace_mmap__read_head(struct auxtrace_mmap *mm) { struct perf_event_mmap_page *pc = mm->userpg; - u64 head = READ_ONCE(pc->aux_head); + u64 head; + +#if BITS_PER_LONG == 32 + if (kernel_is_64_bit) + return compat_auxtrace_mmap__read_head(mm); +#endif + head = READ_ONCE(pc->aux_head); /* Ensure all reads are done after we read the head */ smp_rmb(); return head; } -static inline void auxtrace_mmap__write_tail(struct auxtrace_mmap *mm, u64 tail) +static inline int auxtrace_mmap__write_tail(struct auxtrace_mmap *mm, u64 tail) { struct perf_event_mmap_page *pc = mm->userpg; +#if BITS_PER_LONG == 32 + if (kernel_is_64_bit) + return compat_auxtrace_mmap__write_tail(mm, tail); +#endif /* Ensure all reads are done before we write the tail out */ smp_mb(); WRITE_ONCE(pc->aux_tail, tail); + return 0; } int auxtrace_mmap__mmap(struct auxtrace_mmap *mm,