From patchwork Fri Jul 8 14:07:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 12911206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F347C43334 for ; Fri, 8 Jul 2022 14:07:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9BFE06B0072; Fri, 8 Jul 2022 10:07:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 96F6D6B0073; Fri, 8 Jul 2022 10:07:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85E1F6B0074; Fri, 8 Jul 2022 10:07:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 785976B0072 for ; Fri, 8 Jul 2022 10:07:49 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 56AD7120FF2 for ; Fri, 8 Jul 2022 14:07:49 +0000 (UTC) X-FDA: 79664111058.16.DB2115F Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf31.hostedemail.com (Postfix) with ESMTP id 0A3B820051 for ; Fri, 8 Jul 2022 14:07:47 +0000 (UTC) Received: by mail-ej1-f53.google.com with SMTP id os14so7672912ejb.4 for ; Fri, 08 Jul 2022 07:07:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=UmQMSw/Lo/aUgwdf6X3tLXsFA1/lan16IQ3/ce55QGc=; b=e//7/zevFojSgOmv1iUZPcR9RxpOxiTq/3m9/uN6nMSXeI3cXH4ixaWg5fpOBb1Xyq 2/6CsOMlmIQ90xk3ZqpS9GmPsMEN77fRzrXujYwyvsFbFyV+Yr6pt7yDy8oh4YLBJHuI J4kwdndVmwW26wL2r1fIHKQXzEUE7DZIyQAZGyj6Jt0fO9fs6YuDgnPOK9CNdckL5ovW 9ckRgnYGvYqeB3OyMnm6b6nYTos1wFBo2zRVCQH6TzwBZY1iiWIj/wS0HJu578+a/37m CBMaGj+SfkKAEihV4KGfQTD6lzxvioUDTGkr2PHCMSz63svr+8UiaWd4Huyh6CIUuIWQ Gt0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=UmQMSw/Lo/aUgwdf6X3tLXsFA1/lan16IQ3/ce55QGc=; b=N/4qrrJsTH0AK98qxRzQiNrgg1wKHpMbzbedop5eXNs9Idr9r4IkHUVDC+AWenPpGz ZOvLB3dqKtexmC+KDbmk8t5ocUGD3T8bdQAXEJw1FCvfExTZQvw4+MP5ZhfYKuSNqT/2 BurLQEaidoG5266w5akJKXUQjxEbBiHdUZ7FD7QhYcnsNUF83/NxrMWJdaKhbAcNAdfp 7xrhClEnP5cB4eZzHBxFNuWArQtP35wwpxuQwV137Ln4INL4Xq6mRhjBsEJt+vK2fZVw nSBpBpk8CoU5WoTs0HWgRuzpmDZqwyZGYEXitVhfe5XXtFDuxytyhz3g9NYUPkIcpm3R Sqcg== X-Gm-Message-State: AJIora+vmAgx1pSTqpqd6J8CChAnHetxUD6jmk26HjmjWX5BoKc/kTfj O+bSeQvFXbKsShXTmqOdLc8fFTlFM30= X-Google-Smtp-Source: AGRyM1t1DjCzfOwLiRgELsKcXKYuqKjMDw7UMRA/ZepHVRPjeRcW0Dz4HWw2KyJ1desnWjuhUEvOGw== X-Received: by 2002:a17:906:e16:b0:726:a026:27ff with SMTP id l22-20020a1709060e1600b00726a02627ffmr3646857eji.573.1657289266405; Fri, 08 Jul 2022 07:07:46 -0700 (PDT) Received: from localhost.localdomain ([46.248.82.114]) by smtp.gmail.com with ESMTPSA id 1-20020a170906308100b007030c97ae62sm9533959ejv.191.2022.07.08.07.07.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Jul 2022 07:07:45 -0700 (PDT) From: Uros Bizjak To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Andrew Morton Subject: [PATCH] mm/page_alloc: use try_cmpxchg in set_pfnblock_flags_mask Date: Fri, 8 Jul 2022 16:07:36 +0200 Message-Id: <20220708140736.8737-1-ubizjak@gmail.com> X-Mailer: git-send-email 2.35.3 MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657289268; a=rsa-sha256; cv=none; b=8TiYoV++oVupQnC1thl0iukHx8Z6WvOGfWzLzBh4O6wZs7iNvT0WV/sJGIqESte4zbJofs Cbxv/pFsX/PIZ/7bkeqslVGG4jVEerytfaqT+8CiTsW7VlBpw5MYYDoHyaXczptsEpzEDd m/I0ckqnHqGsIFyRhIIhMtUOK50IY4Y= ARC-Authentication-Results: i=1; imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="e//7/zev"; spf=pass (imf31.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657289268; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=UmQMSw/Lo/aUgwdf6X3tLXsFA1/lan16IQ3/ce55QGc=; b=2IVHlbdmiAbEsceUrk5E4OwiXeTz1OhLI5wKg+Wt3Hit2GNqL2HugV/aV3KH60nTVRPXGX SjWw2jAqiiyAZ69VWYKG/yqvqNuCvjKaDLWwNzmUHBd3olWBNVcPiTCMuR9TfJDvfNUCCI d5DV6QlYZUdBvvSnajCF0iRxfqi3yWU= X-Stat-Signature: rq4zffcahe6ts1ju9kukej937bhzsnue X-Rspamd-Queue-Id: 0A3B820051 Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="e//7/zev"; spf=pass (imf31.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1657289267-978234 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use try_cmpxchg instead of cmpxchg in set_pfnblock_flags_mask. x86 CMPXCHG instruction returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). The main loop improves from: 1c5d: 48 89 c2 mov %rax,%rdx 1c60: 48 89 c1 mov %rax,%rcx 1c63: 48 21 fa and %rdi,%rdx 1c66: 4c 09 c2 or %r8,%rdx 1c69: f0 48 0f b1 16 lock cmpxchg %rdx,(%rsi) 1c6e: 48 39 c1 cmp %rax,%rcx 1c71: 75 ea jne 1c5d <...> to: 1c60: 48 89 ca mov %rcx,%rdx 1c63: 48 21 c2 and %rax,%rdx 1c66: 4c 09 c2 or %r8,%rdx 1c69: f0 48 0f b1 16 lock cmpxchg %rdx,(%rsi) 1c6e: 75 f0 jne 1c60 <...> Signed-off-by: Uros Bizjak Cc: Andrew Morton --- mm/page_alloc.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e008a3df0485..d415d5b539b7 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -524,7 +524,7 @@ void set_pfnblock_flags_mask(struct page *page, unsigned long flags, { unsigned long *bitmap; unsigned long bitidx, word_bitidx; - unsigned long old_word, word; + unsigned long word; BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 4); BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); @@ -540,12 +540,8 @@ void set_pfnblock_flags_mask(struct page *page, unsigned long flags, flags <<= bitidx; word = READ_ONCE(bitmap[word_bitidx]); - for (;;) { - old_word = cmpxchg(&bitmap[word_bitidx], word, (word & ~mask) | flags); - if (word == old_word) - break; - word = old_word; - } + do { + } while (!try_cmpxchg(&bitmap[word_bitidx], &word, (word & ~mask) | flags)); } void set_pageblock_migratetype(struct page *page, int migratetype)