From patchwork Tue May 28 14:43:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uros Bizjak X-Patchwork-Id: 13676931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBF81C25B78 for ; Tue, 28 May 2024 14:44:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19FE36B009B; Tue, 28 May 2024 10:44:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 128E76B009C; Tue, 28 May 2024 10:44:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE35F6B009E; Tue, 28 May 2024 10:44:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D36326B009B for ; Tue, 28 May 2024 10:44:07 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 80D8514059F for ; Tue, 28 May 2024 14:44:07 +0000 (UTC) X-FDA: 82168074534.02.6D38396 Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) by imf05.hostedemail.com (Postfix) with ESMTP id AE5FA10002B for ; Tue, 28 May 2024 14:44:05 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AGWjoUAP; spf=pass (imf05.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716907445; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YNJyQBEZIlvY5qkox0e9QbOH8UJx4eH8/3j/qt71lDE=; b=o6tyjC12wXjr6tWYQUAEarQBD7jS7cynE2FiAT4LSXMbcT9ncUO0FkdMuMfkhvNEz2CIBl 1BlLJIUz4dJ4mE5sByvb9gL0EWxl1otPO4rPUxC2yYUHyS/z/XhkTSiB3LU72uKrjfErKc VAQjDylUh0UJ5X9IAC+xt0HS47oNyOk= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=AGWjoUAP; spf=pass (imf05.hostedemail.com: domain of ubizjak@gmail.com designates 209.85.208.175 as permitted sender) smtp.mailfrom=ubizjak@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716907445; a=rsa-sha256; cv=none; b=b0jsc5MVuo90Wb8ub3rpEw3+HhaYZzj8Vq6/YVhEZs4WkQIx+UTJKWpAg9dy7tLER2Xe6o dogBr5591mL5RpGGRPOsX/fymjxQcgudFYGv31q9Pa8+0Zmh71eJrdMcUXPwYXDtIm8vZN M9W1CQufa8TAmNScEAf9OeDvLem+9s8= Received: by mail-lj1-f175.google.com with SMTP id 38308e7fff4ca-2e9819a630fso14255521fa.1 for ; Tue, 28 May 2024 07:44:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1716907444; x=1717512244; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YNJyQBEZIlvY5qkox0e9QbOH8UJx4eH8/3j/qt71lDE=; b=AGWjoUAPKnlSzrJbxQhG+sZMWsSMAAwqTfkj14DOJA82wI58y0n2Joi4j1D7Jkv7Y2 5w44VOnYVVF96m1GAMVPEfswAOf0BXrN7XCwLWUDxtwAtwrTaLrlteKoG6+SCKyNe56+ 9JspCfXXt5pXGK6Uiuw79y6NGasskdhdEYv49/6rp+Uh/vMakcfwYVBKkBaA6pHwtlz0 prLKHia5y1qI7YPgj6GzuEo5RZVGDx3NxVvy3TfDyQgF/dPolBYizLLfDGl83zDTPDi0 RKZrB42fjcdQTshbZcE8k2ZyoIW2iYPmdblFtyC7DjaslCnrrAGvuYCZW2D35sFRT/Bo Ce6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716907444; x=1717512244; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YNJyQBEZIlvY5qkox0e9QbOH8UJx4eH8/3j/qt71lDE=; b=TzWk7PDkRknULF8VSBEPI5jz1ozdAwfzVCRNUsb4ofxMrSh1/2HFNs90zQC1UZU1Af 6GekPJPNz8CxKZu9lVPIaWBlLJVIe5oAB5gjCCnl1Yg2Lf5UlJ9OXG5yRcq3GUdfo040 gWSG54xMAUWtgn3BtooWvsTbDioIYPrMx+PEM/+8AijAVYZd0s+ZsVotqVFqmH/FtF0p ZTEuJ9r2pNtDcTeZx2wuKnHnXQjrTCQBQWeGMh5kRluDLmdW4arsnWdyE0MUqS5KW0vV yuNvLo749/VPVfRJ+Q7WnRd+WvIfI/t4MZ+c9GhAIRrqq77BRlLtPnDTCxbNs9v5gRyW WAsw== X-Gm-Message-State: AOJu0YyAIXiPwf7Rg75F00+mboehDWjzIeiXcdB31Ja60saYew/Ycp8I //5AGr7xgvOzLE7jwM626CuHRgRkQfBATFVJAl17/+Ea6QGyowNrMJnpo3y0mGY= X-Google-Smtp-Source: AGHT+IH9fcX9h8dAddsirOf8W9e2zRkjaMX3KHnPUQQFi9/AO0LjE/tHFGuIEkZAFmGJ+xwruFYb9w== X-Received: by 2002:a2e:9e04:0:b0:2d8:5fe6:820d with SMTP id 38308e7fff4ca-2e95b0411c8mr97696731fa.11.1716907441945; Tue, 28 May 2024 07:44:01 -0700 (PDT) Received: from localhost.localdomain ([46.248.82.114]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a626c817ad3sm629797966b.16.2024.05.28.07.44.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 May 2024 07:44:01 -0700 (PDT) From: Uros Bizjak To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Uros Bizjak , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Dennis Zhou , Tejun Heo , Christoph Lameter Subject: [PATCH v2 2/2] mm/vmalloc: Use __this_cpu_try_cmpxchg() in preload_this_cpu_lock() Date: Tue, 28 May 2024 16:43:14 +0200 Message-ID: <20240528144345.5980-2-ubizjak@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240528144345.5980-1-ubizjak@gmail.com> References: <20240528144345.5980-1-ubizjak@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: AE5FA10002B X-Stat-Signature: s4dk75c9ik4m3p6jg76oj71aiejr3bxp X-Rspam-User: X-HE-Tag: 1716907445-599007 X-HE-Meta: U2FsdGVkX1+73f2M2aZ5jgOk7R3UXmdvViiYCD7uHMquhOCzb5rFtEcHa0V96Y6IDBvIlJv6Fa1Fja+cBjQwNT6c559a5fGYOWdIY2XcA69BKCE6knpOmXNBJBShY4bl1huQdad2nZLglQddGhE0qSP2CzItWhvHcCU/+DkP18Qriqhb1KtuQycUcsjz0kI2qyI+xjyECxfM1Y2dmfce/oncY8ABzNwVWMVvX1q1ZEhbJqhjp0kBjQAj5pwcR/Y2SIl8LOyaBhZNCofsscTgAl2+7wzk43tn9m6gt5Uyh/TWLtm94Gb9Lt14FQEqY7X4f8zt2PARcv6bkqk27DZ/jx+Oq4DX0UnfuEUruAQXrhBzeq8ADU0bf36Jzo0dLQTYch6sKgjvzRYQ6r+6h+V7Yiidgf/+uN/0ljVwvJW87QJYxYJrNQoMPe72o8z3ipJblgctRQt+HNynoE8Fs0tWrranMDXm8Fm2N1DMElV6P4HuuW0rLk1DhWMxL5XSU0YMHlLyHuDCXxJKXnLvrPIYq0ogvNEYCNbz6nIQcA4dkxkW6g0o1vU7C9zoyQq4/JdYBtgEFAGlUU48JTZiaAZ7Rk7pxvlEM2wP4FpdAsVvxyJ91h138ccP/M5QaVBt6T7F2VB8oCjYjqzKmYE1BXcKpIA1guovAzywx3jcdrOBcfBjCD4N6bdlvcWk2CPKDUM+8fJGodPeZ1WrUcDi1BkBLRH3/WkNEKfG20tIyL5cmjd0v7nztBpw1cN0ZNDmpL38+NyIsynxfrwQb8GSfigg9cuFZpmHHj2TQ9mNRhV/GsteS0RZVndKTZ5KR2xSzmmFPnkLAKKeDuFTvQnn2KQi/OBMTDqxA5vCe7yRsCelOz/RHmmFhum490xzGJ0bsFPQlr+vXCUdNkcxb1CvuchvIh40Ju4DfJ2RKBUmezeQy5k6JdASmJD/ZBC66WH2YENQls+kKQHQ8ofLL38Ib5t jW6If0KB pvEqxAWumrGl1TX95IjFdLeEN+HPKx4n12pYWuqm+TGZxAjyWU3PolwCXwpb55cO5xcAAaWnpC51VouXf1lv5bEq1pVgyfVQz1UQjeuveJ104oWaDaHBO6knggnTqwOJ3II6vQqyhb391jqpBlrGvZuN+LJ7EgSX8TudWLjg8V3St+EBhmpOS7ihQUHWjmwKCL8DVLBUtgJQbMs1UXKm9fTj5IPHG+zjAn/3hoA/VdL8dD6EkjWYVFoYac9M4JfuGCU/LLFevLSNWlX69nauloWFn1yFrsAP7MsRSWyuXA8xBTo8nfIu6bnHWHHM9W+pkBSop8mvmXSDB8lwA+8la3/0tiUZY9E9El2EBy+RfmYdBSZpIuRU9Sb21oqVO1rt3ATy4IYhwvJtpxXNkYeTyW/pC6/Vn6wVwOVke/HrYGcsWuORuPaEMKou+vpKrCdu5Pz8rznnUoWzw0lSs9lBOB8zJhtYtlQsNJqCc8nAf/YimcYxoEd3WN9UHniCkkZQLUPtOEQMmtzCSlQxTuItvPMzfuzxAAl6s/A+rEaOHnWpud33qQLObjngY4jw+OU22urHM7m2ydj3ZH7+5rfmKkmL+bHiqcimm+HA7goFyAltSBiI5osFkao6SAzL48rMWnvPrL3oqMYspAq3+ZHy3yhyQKck5KQxmZJThIyv72bDAId40AjDdp74w1odmhMNxlhdXsff2bko3366pYh2XK57tn+tkg/gBvfaL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use __this_cpu_try_cmpxchg() instead of __this_cpu_cmpxchg (*ptr, old, new) == old in preload_this_cpu_lock(). x86 CMPXCHG instruction returns success in ZF flag, so this change saves a compare after cmpxchg. The generated code improves from: 4bb6: 48 85 f6 test %rsi,%rsi 4bb9: 0f 84 10 fa ff ff je 45cf <...> 4bbf: 4c 89 e8 mov %r13,%rax 4bc2: 65 48 0f b1 35 00 00 cmpxchg %rsi,%gs:0x0(%rip) 4bc9: 00 00 4bcb: 48 85 c0 test %rax,%rax 4bce: 0f 84 fb f9 ff ff je 45cf <...> to: 4bb6: 48 85 f6 test %rsi,%rsi 4bb9: 0f 84 10 fa ff ff je 45cf <...> 4bbf: 4c 89 e8 mov %r13,%rax 4bc2: 65 48 0f b1 35 00 00 cmpxchg %rsi,%gs:0x0(%rip) 4bc9: 00 00 4bcb: 0f 84 fe f9 ff ff je 45cf <...> No functional change intended. Signed-off-by: Uros Bizjak Cc: Andrew Morton Cc: Uladzislau Rezki Cc: Christoph Hellwig Cc: Lorenzo Stoakes Cc: Dennis Zhou Cc: Tejun Heo Cc: Christoph Lameter Reviewed-by: Uladzislau Rezki (Sony) --- v2: Show generated code improvement in the commit message. --- mm/vmalloc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 5d3aa2dc88a8..4f34d935d648 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1816,7 +1816,7 @@ static void free_vmap_area(struct vmap_area *va) static inline void preload_this_cpu_lock(spinlock_t *lock, gfp_t gfp_mask, int node) { - struct vmap_area *va = NULL; + struct vmap_area *va = NULL, *tmp; /* * Preload this CPU with one extra vmap_area object. It is used @@ -1832,7 +1832,8 @@ preload_this_cpu_lock(spinlock_t *lock, gfp_t gfp_mask, int node) spin_lock(lock); - if (va && __this_cpu_cmpxchg(ne_fit_preload_node, NULL, va)) + tmp = NULL; + if (va && !__this_cpu_try_cmpxchg(ne_fit_preload_node, &tmp, va)) kmem_cache_free(vmap_area_cachep, va); }