From patchwork Mon Nov 15 18:59:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minchan Kim X-Patchwork-Id: 12620235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62D0CC433EF for ; Mon, 15 Nov 2021 18:59:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 12CC66368D for ; Mon, 15 Nov 2021 18:59:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 12CC66368D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 2E15E6B0082; Mon, 15 Nov 2021 13:59:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 28D976B0083; Mon, 15 Nov 2021 13:59:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17D336B0085; Mon, 15 Nov 2021 13:59:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0181.hostedemail.com [216.40.44.181]) by kanga.kvack.org (Postfix) with ESMTP id 097346B0082 for ; Mon, 15 Nov 2021 13:59:25 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C1AD384761 for ; Mon, 15 Nov 2021 18:59:24 +0000 (UTC) X-FDA: 78812077848.09.58D865F Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf08.hostedemail.com (Postfix) with ESMTP id A76C4300024F for ; Mon, 15 Nov 2021 18:59:07 +0000 (UTC) Received: by mail-pj1-f49.google.com with SMTP id fv9-20020a17090b0e8900b001a6a5ab1392so637936pjb.1 for ; Mon, 15 Nov 2021 10:59:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pSLrn5y02nFYb56rbdejNlZIJ15XWP3dy4aaYUGw/MI=; b=RYLXxS412x5P9PLxG/vpcXj0PeJFVvZsbSQUYS6T0nLuS8Oh3IpnpC5stTjDBfyEbg HrB6vour5ki1ihptvQ56/i2OiyZ6Bzdz6Fr/gZkRYicQwmz1btnU/OAKDB4IIq37dOys UT+LrPgPYUcg8PWFjPZJ4DuB6HR+NU8ij0ybDL3Qs/wMi/IuEBXGHLUEoPd2Q9YU2loR ALennVfrbcQAjUsi3r8iXSnS/S8M0vxi/rDTC9avuNnqySmErzxPwth+P7o5BWR0/o3V bSq8Yw4XiJiyTHaeaHAC2jDBrzmBiMowF5kFyRI0fa6sbrGvP0UXfAFHcjOYN5XGn/0z hCLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=pSLrn5y02nFYb56rbdejNlZIJ15XWP3dy4aaYUGw/MI=; b=26UavWLpnqY59tI6aeGwup+2gddtyymg9fRsUU+h6cFL0WuzmvtIHRegEqVXdy7wSP rO4lsR9Jn8ovkD6GWZrDxURDf0RJNBI2o+THX9qbr9Fq7mo8bnf0H2Dlke4xKH37oGvp frZOF8YXUlBR7CTP5BYBwOeALk+RHUaIpuHxGBuU6TGsZ56MCGxt+bP9axWngEqZalXY eSX1ZZD1OkXekZ+nVORUq4q7TKNCcn5imXgHuAqK5mU7S3yiSxtcLadbKUyS1QZy/28/ 2Eg8WrUymVbbRtyKcCwS0SYHMqDQFobwP/5LNbuDSnXaTiEqYzy/4WUdB7yAKNUT5wLv I6bQ== X-Gm-Message-State: AOAM532qnaoTOaH9cGo6GVipuEbik4bzMO6+/eQpUTrs7ePE/QMdtCb9 mGLQHtpPVF7q2oMc+FcZrw4= X-Google-Smtp-Source: ABdhPJwrGFruznHxWpMGeZyh0kKzDStfOuXOf5UduhvpYVsBEjGiaE1wgASm37OzdreK+H5/sdFdeg== X-Received: by 2002:a17:90b:4acd:: with SMTP id mh13mr68392410pjb.230.1637002763347; Mon, 15 Nov 2021 10:59:23 -0800 (PST) Received: from bbox-1.mtv.corp.google.com ([2620:15c:211:201:7a54:8083:4365:b23d]) by smtp.gmail.com with ESMTPSA id c3sm11882941pfv.67.2021.11.15.10.59.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Nov 2021 10:59:22 -0800 (PST) From: Minchan Kim To: Andrew Morton Cc: Sergey Senozhatsky , linux-mm , LKML , Minchan Kim , Mike Galbraith , Thomas Gleixner , Sebastian Andrzej Siewior Subject: [PATCH v2 9/9] zsmalloc: replace get_cpu_var with local_lock Date: Mon, 15 Nov 2021 10:59:09 -0800 Message-Id: <20211115185909.3949505-10-minchan@kernel.org> X-Mailer: git-send-email 2.34.0.rc1.387.gb447b232ab-goog In-Reply-To: <20211115185909.3949505-1-minchan@kernel.org> References: <20211115185909.3949505-1-minchan@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A76C4300024F X-Stat-Signature: zbdkpxjpbommrtgxkmpinay5u518sq87 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=RYLXxS41; spf=pass (imf08.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-HE-Tag: 1637002747-900532 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Galbraith The usage of get_cpu_var() in zs_map_object() is problematic because it disables preemption and makes it impossible to acquire any sleeping lock on PREEMPT_RT such as a spinlock_t. Replace the get_cpu_var() usage with a local_lock_t which is embedded struct mapping_area. It ensures that the access the struct is synchronized against all users on the same CPU. Signed-off-by: Mike Galbraith Signed-off-by: Thomas Gleixner Signed-off-by: Sebastian Andrzej Siewior [minchan: remove the bit_spin_lock part and change the title] Signed-off-by: Minchan Kim --- mm/zsmalloc.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 5d4c4d254679..7e03cc9363bb 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -65,6 +65,7 @@ #include #include #include +#include #define ZSPAGE_MAGIC 0x58 @@ -276,6 +277,7 @@ struct zspage { }; struct mapping_area { + local_lock_t lock; char *vm_buf; /* copy buffer for objects that span pages */ char *vm_addr; /* address of kmap_atomic()'ed pages */ enum zs_mapmode vm_mm; /* mapping mode */ @@ -451,7 +453,9 @@ MODULE_ALIAS("zpool-zsmalloc"); #endif /* CONFIG_ZPOOL */ /* per-cpu VM mapping areas for zspage accesses that cross page boundaries */ -static DEFINE_PER_CPU(struct mapping_area, zs_map_area); +static DEFINE_PER_CPU(struct mapping_area, zs_map_area) = { + .lock = INIT_LOCAL_LOCK(lock), +}; static __maybe_unused int is_first_page(struct page *page) { @@ -1269,7 +1273,8 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, class = zspage_class(pool, zspage); off = (class->size * obj_idx) & ~PAGE_MASK; - area = &get_cpu_var(zs_map_area); + local_lock(&zs_map_area.lock); + area = this_cpu_ptr(&zs_map_area); area->vm_mm = mm; if (off + class->size <= PAGE_SIZE) { /* this object is contained entirely within a page */ @@ -1320,7 +1325,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) __zs_unmap_object(area, pages, off, class->size); } - put_cpu_var(zs_map_area); + local_unlock(&zs_map_area.lock); migrate_read_unlock(zspage); }