From patchwork Thu Apr 17 16:12:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uladzislau Rezki X-Patchwork-Id: 14055835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2ED0C369B2 for ; Thu, 17 Apr 2025 16:12:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35F456B02A3; Thu, 17 Apr 2025 12:12:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30980280005; Thu, 17 Apr 2025 12:12:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1347C6B02A5; Thu, 17 Apr 2025 12:12:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E70E26B02A3 for ; Thu, 17 Apr 2025 12:12:23 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8DFB3B4199 for ; Thu, 17 Apr 2025 16:12:25 +0000 (UTC) X-FDA: 83344028250.24.C6D82E1 Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) by imf02.hostedemail.com (Postfix) with ESMTP id 874AF8000C for ; Thu, 17 Apr 2025 16:12:23 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=V3JvTYYV; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.41 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744906343; a=rsa-sha256; cv=none; b=1eIuVFxE79iwMHsUKPOsZ86XbiLLYy+fAVp6p3vkV53X42mGqDtELqYoSHZ0Yaz9rv3t8Z 6EkU4Nc/6AupNxYVqxwxYsuYVHUv9hKsiVaK99vabJhWeYjRQbpaejv2bJ9Ff6iX+H8Qbv SQrd7zZlPIZbrxY6YXzPmDWlWebpVM4= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=V3JvTYYV; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.41 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744906343; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kiIA8fIy8YmbCJnZvN/tju1TgqCKi7yKEus7AdngHaA=; b=cJKMw61YUEefz04VCySTirVI9HFPUSw86aXN7GxWCArGPhPSyRj2LmP7D1FQikaFHt2V0Y hJHhjUug6BkUpctqnKw5dj+3AZ3abBtviwOdA/IbJplMPXSH6yfFJk16yyAO/O0zOVYVWN kQPk2FhwmiLwV2WajL5s0fLPHp119Ss= Received: by mail-lf1-f41.google.com with SMTP id 2adb3069b0e04-5499c5d9691so1206255e87.2 for ; Thu, 17 Apr 2025 09:12:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1744906342; x=1745511142; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kiIA8fIy8YmbCJnZvN/tju1TgqCKi7yKEus7AdngHaA=; b=V3JvTYYVuJBKLn6i7ZdpiaeC+4bNq4tu1+80AROfwEcCF80i41XPvM3lBDkJpXCU9G HcwVL1e8KpH0mSEYRD25jDcIyEJ5Ds4BSwF9hkanxtWu8Sui5Km31k8QNbKvY+0zRLfb ckwAB/v1dwDerciRdE3wKRWalkNoQdPWMyoBoEhQlTvCcOZFcxN4MVwqBcdo1PZfLlC1 a+LF+OXvqWmf+ti1Sy3c1NSltYgsrfYtdgSc1m/2UBnrC0p4XgLE40u4XnNc5LpoDVRe I5fLWgUbUei66jzH0tZlkuXi8XULvDv7JQC49UJORUJuNTj+pispwyFCKwNcfOaK+O0k rE0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744906342; x=1745511142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kiIA8fIy8YmbCJnZvN/tju1TgqCKi7yKEus7AdngHaA=; b=Op+CToSQGtFkTun/f1tTjXMQbHV1IHkZaHaGzSY72/QWHXRzY6URT6pjr+L+taVA4Y R2zd5VagOV70tB7PHNbBM1wv+xXNuDTjoc7ndGaLyAl95AvhKesh+zvC+Bq6jXRFrC8R SQGfrxIg9+4aEZTxEcHuKhTnHm+hcjXp1M2brBgeMveS9m+u0oe8mgps8bZesr6I3t9p Tm844n9KULjzFFv9fbcQv7nIH+Pu28xb6q6wEdpNxizmUZ9UHY5kFE2Lizz5Z/LeKXyZ sICJre1VmQeIMXxIeFe84f8O3u27F2GnQzT/0j6YDFyCjEMQ/4BXHFhYW6kJ1lewSKPv ESyQ== X-Gm-Message-State: AOJu0YxpkkV1t22Eymx7GYXqtkstQnkrWkIolbAvL8L74X9pe+jt1TBp 3k/Yy8pLOUrrLrmOa45kadliJR68DDOtmLnOUW5/aPkoQ9dDS4l+ X-Gm-Gg: ASbGnctBlQId6lqSmufh2ZuxF8myPoFrW4yNEzUGJnpDZxLnPSHA45j/IJ+vi+HLqw0 6LDJ1TuvERZdoaCXV0vDOaTXlc1K7CQN7sBcrxVRTamkzwcQvEdQt8bR1ke+YUJ1vx7zONAJ4BQ oprdIsADA2JCWgqWi+1pLhfZD69ChtUyXWutxVGmyU7XIRL0Hwr0M7oicAbEKH+3c+YKnaFQRUc DUYWgJg+mU+7ccwBnFpRBo6W8PX4Az1aa0wuZe8pH655oRq12dT7g4qPl4MvtO+Olbi0m0qq0xQ 9dzpq+Ix6KbFC7HwZP5+CLolUsdhRoKUC+Dv X-Google-Smtp-Source: AGHT+IHEqcCUYv49u9TJin5pQQY9jjmHnrt6QzVErhvQ26P4aN8IZwhobOglByroR8+wQnj41vZmsQ== X-Received: by 2002:a05:6512:a87:b0:545:ee3:f3be with SMTP id 2adb3069b0e04-54d64a7b390mr2599564e87.10.1744906341619; Thu, 17 Apr 2025 09:12:21 -0700 (PDT) Received: from pc638.lan ([2001:9b1:d5a0:a500:2d8:61ff:fec9:d743]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-54d6df60cf6sm2882e87.66.2025.04.17.09.12.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Apr 2025 09:12:21 -0700 (PDT) From: "Uladzislau Rezki (Sony)" To: Andrew Morton Cc: linux-mm@kvack.org, LKML , Baoquan He , Christoph Hellwig , Uladzislau Rezki , Oleksiy Avramchenko , Mateusz Guzik Subject: [PATCH 4/4] vmalloc: Align nr_vmalloc_pages and vmap_lazy_nr Date: Thu, 17 Apr 2025 18:12:16 +0200 Message-Id: <20250417161216.88318-4-urezki@gmail.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250417161216.88318-1-urezki@gmail.com> References: <20250417161216.88318-1-urezki@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 874AF8000C X-Stat-Signature: yqetcjeyasaf8gnjmqboppp939bwmrbg X-Rspam-User: X-HE-Tag: 1744906343-267762 X-HE-Meta: U2FsdGVkX19A6yxKdFw39QkDfPQuCBzBuhE8VuudxiKD3UQBENH6tRCA98haYvZ7Ua6Y9DvPG5OW/spvm+Hu9J51pnpHNhZFUOvt5qbffDc9nKLs3EI948SL/DcZRoY8Z7dLcQJMx2/hh8HEr7nVFVRdpXEzCqhsfkA7sr1ZSxi/lDWrf4ACMTfcxrY6hPLph/z4YtEigy0X8f5ufqOl4rChnUsYcWgWUCfOvqroWfPLnsZ9Ev7iibo07SzJ+4OePaQFv5HPkzXi+BfnKu3/0EJ/gnrX52usR8Wx3TEyHha9K5bMWii0tBnFisVEB5fZS8aChTCN4viOYDt0sSvBZfKC3e+NQocOCm/FPlIwQV7Q+/Cgyoc5aeQXC3K4phrDY4hho6gRvTZgS+apHQat0xm/uaOUGw7r2RR5fBuh1JOG/yZkJtAUnAKcghPH+GSI/aV8u64PbVPnnzoIhgV0JZeFHjUGuKhprQo+K56LlwTZ/956ASsxnxq7airmtxQ1B0eiGvJpmTFBlH2OKv/u/UdUkdnaHo0me4il31FCbFXYN+lLKXV9gVhjMc8DtekXV8XkZKHbhAVs4Ba/z1GoU5+RSdTrlquIyv70rIPZDvse93zbiutijseH/CqeYku+nx7spIJf58+yOEWc/sbuaCBtrbREgb3eHoBnO6h4QmyFu87kle1UP9Z/JeTj2odaLGg8ikFyNhLsX9XuUIMjsmbq87AYJOgrewTbkzmPhgHXJDr0625rUnyHY3bNRXOTzDjLcJnsfd1f538z6bWD6OVBt6U76WBqkf7xZqk/Gl1G60gZrMjRMfJ4IL/VjZYgMPVVyVtMbyA7AkuP3AYhYxdB4fP1d+O3eT3d/sO9qA0+DnbCs7T8JXKzhwbUqWPkZbe7FBB5ZyAW4L7QIwepeuO3xYdnUN7g5riq5CWuCWXVBNtWFQ7pRBMzIccfn6MYe1PyZbPg7p+OIFPUkDC mEK22kaK yNa7I5JJjnfGte4Wj/jKqoq3Ze44CjkgR3fnueUUqJ4RCNVoFtwhODmQ4Ku651+bZyB6AggWcL0JftCEiyVZ4JNl6gRGdhAFVqkje4U+wPAccCp3UdBzL2kaO6pgco5hsStjv9JdvPI3iARhTzJvBVCiXQBa5Be+RCtdezVvCFcyhX+zDMBd4O1upE4GxlCVrSrCHFQ4SSAAZnBpkEVjQfTR55noUkVO2hzAvd/E85m6fLqrOEMICrtVyNH1+RZ/SyCazTOcjAXEPbVTc0SkaA11plapM0LDb4VvJPNhyUpxUJuwFPqMByRYYsQ3Hbs7DacS0Egco/9SdamGxuuwIW04h+V1fElLJrnhovS5ocO/TrxfuFdWnK499eun5xzyrxwmueKJNaO8ayC460UCEpXjmy05KAvKLQfBGJRxRXeJ/evU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently both atomics share one cache-line: ... ffffffff83eab400 b vmap_lazy_nr ffffffff83eab408 b nr_vmalloc_pages ... those are global variables and they are only 8 bytes apart. Since they are modified by different threads this causes a false sharing. This can lead to a performance drop due to unnecessary cache invalidations. After this patch it is aligned to a cache line boundary: ... ffffffff8260a600 d vmap_lazy_nr ffffffff8260a640 d nr_vmalloc_pages ... Cc: Mateusz Guzik Signed-off-by: Uladzislau Rezki (Sony) Reviewed-by: Baoquan He --- mm/vmalloc.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 77da4613f07ff..54f60d62051da 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1008,7 +1008,8 @@ static BLOCKING_NOTIFIER_HEAD(vmap_notify_list); static void drain_vmap_area_work(struct work_struct *work); static DECLARE_WORK(drain_vmap_work, drain_vmap_area_work); -static atomic_long_t nr_vmalloc_pages; +static __cacheline_aligned_in_smp atomic_long_t nr_vmalloc_pages; +static __cacheline_aligned_in_smp atomic_long_t vmap_lazy_nr; unsigned long vmalloc_nr_pages(void) { @@ -2117,8 +2118,6 @@ static unsigned long lazy_max_pages(void) return log * (32UL * 1024 * 1024 / PAGE_SIZE); } -static atomic_long_t vmap_lazy_nr = ATOMIC_LONG_INIT(0); - /* * Serialize vmap purging. There is no actual critical section protected * by this lock, but we want to avoid concurrent calls for performance