From patchwork Sat Jan 21 07:10:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A02B8C004D4 for ; Sat, 21 Jan 2023 07:11:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 46CC26B0075; Sat, 21 Jan 2023 02:11:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 41BD46B0078; Sat, 21 Jan 2023 02:11:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E2F06B007B; Sat, 21 Jan 2023 02:11:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1ECE46B0075 for ; Sat, 21 Jan 2023 02:11:09 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EA0E0C0A87 for ; Sat, 21 Jan 2023 07:11:08 +0000 (UTC) X-FDA: 80377934616.24.F34B1B6 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf21.hostedemail.com (Postfix) with ESMTP id 406161C000E for ; Sat, 21 Jan 2023 07:11:07 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BhY3pELB; spf=none (imf21.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285067; a=rsa-sha256; cv=none; b=OR0IT0uhXhRwVkBnVljeYb8lDI6Wp+Xu3MNzewiY4UmXlalG54MaQjbqm0P+npk+RY2R2O aYua88gFhFzIcrTzA50RFjLv796MrLaHzU029M4Jy2XULVfVL/jj87jOJVOB74AeuSRZWL o4z85y43P6BRE1YEb7ek4r77OsnDp3I= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=BhY3pELB; spf=none (imf21.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=eBVOWMkt3qqfmS1eY0rzR8noKiZ5EvmTGPqU8LqtGbE=; b=ZVhnD9zaK+5LxIoJf6xAt06u4Fay7cRNe+HwT9oEUi98Linh0bBm1R1scgloiNhcxY4xPb v7OwwZ+svIBtcCfyGCr981p5ZX0ztoBPVFQUbgZVeAM9nBLVnYlTJp+hXRyemp3CVfIXtI IAp6bwbQ3jV4E8zG98CrMK/wujycQX8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=eBVOWMkt3qqfmS1eY0rzR8noKiZ5EvmTGPqU8LqtGbE=; b=BhY3pELBYuBpORH+zQ3WgTL2AR JaRJ0lhjYYmdhki0EtCux+XcwxeANDK6l/NNLWIweICq4URG6gHCfIb6EhstJJm01jklNiXK6ICm8 LYj7GVOuI8GOhuuWel2M0FWE8LG7zB/HEdsvHIwOfAGLprWnm9e3EI/7kmLiXdCfXdyV4lt9gmXRx 6Epu60uJgHnQVQf/WB4EZg01h3RnHo4Fqyb8ZZT5xUs9H9Yqk/Aw+dPhcHKHwhDhWiycLpBD/UC7K QbCFYCh8XW/6+NU09mSESac4jMPhDLCWenfB9SRVC93wqVGPebIJKU/l84hVIloKOIm3Bd5mILMo/ SpC8YV+Q==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ81l-00DTnB-Jz; Sat, 21 Jan 2023 07:11:02 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 03/10] mm: remove __vfree_deferred Date: Sat, 21 Jan 2023 08:10:44 +0100 Message-Id: <20230121071051.1143058-4-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspam-User: X-Rspamd-Queue-Id: 406161C000E X-Rspamd-Server: rspam01 X-Stat-Signature: e4dkkxa6m4um148g9pu9hefes4p8tgeb X-HE-Tag: 1674285067-632259 X-HE-Meta: U2FsdGVkX19LqDSAxQx6dbGa8fuPMpZz4CmW+79i0ODZDIoqbO0uNqI85IbRY6HyyjkY7n5xTwvptfe6wzeOYPkflw922irxzCWZIMfhssb5pIBkDlQAnRqlbXR6pLKl+dqWL/Eo2TuTB9HXT7SPM92if+4cagDaLG3sws3sdGwVzT6LLqyWMX067Kr1t/ZxVoiej0SqWH1Q2KFHikTIDJOgUPwjH046WuCeEQ10aNns3VyUoAXo43Mnoweo03kDSQMSOZwBkCmR/ATP/n4YJGgipWvu3O+OV+MjAoE8x5O24wCbZwiqulsqwL5xQXSoibSvtqssfB1BK7LuoUnkSlBKfjlFJSAxGVOfFIUjXX8yxDPIB5Jc7re1PBwQaAj//Ia1w5LKfyiwRlN3GkpiokR7+D3RnjtkrtGRyvZs1tODL9oTEspG1CJl2qqZTITwhqxMrffGAMkd2LF7HqoE95wCvx2an4VpHZ+K7/KyWhoktcbZneGkNf5ymEYLGccld+cnEwIUcF83nxFz8bfM2M2zP4t+1BhobSdxV5fmvh7GiCYUhCRq4sSK7P0RMyJXgYrvV5uTW7GTWnBoLG/oq+eb2hq56VH5o/r832STpDur7LWkVnwJUnsGAjRMpD/JRwYESJZdbyE8QcZDDDnc/fEqToScZYQ2vrpLLdwJbSVSkDBRpOB0jd6jIPBbzN5hdHPsjCiqqpj4v2MVyKiM91GFT6skPcYXOYwY6q/HJW7ARN1viS8QwZkHyPanP/qonBINmjX+37z/aB96N+nIyWaKtBF4hIefgbzd6jw/+CsJZZLJv7u0JWIsUEA9525BHi0hhFuIgDzMEhp/I7VcEDn9Y9ljrckCp6jyqAXxFYZvFmYYa6NMAPF+9Q6nJVQsB51q0/XjnMfrB9P/x9npESRLHy4bl/kjraZ9fqbQm3/PhgAwLSv7jjovb2jpOMO8qYPiYoANmbHcV3e/Vrz Fcqfeteg oR37W8hEMuzf1O3g8KTjZbD4FUIQRYytIW06U6rKDHWZCSldICcQmnGniqbhdqPN5CbpBfLh6iBb1xheDkevQAlBtS1bqE5LcUmcpf9Ebg1OD5gxGfIebhyBg4FBohwGHDygCG+VnpTtgRu1H4b9VQXqhIiZIw9567Ex7cUlCzBNbPihccVU18cfwTGYp0dM0vJTi98NOkFeTI2x77PrJyDwB/MzZvW37e2QHq2RQkIWV0figBxIKv3c38hyoUJALgIbQFDqEEkUmL64zDW28U3F/jUHvPcXVShOS+J0KSTO7dVoONvhKD0Hugtnh6HLPIFeBZ2K++7X3jABDRzQ9f5idG5NEx+0BQrxoWhU/139eksW7EEmH+QQg85f4kfHNAsRpFTwuujJ9nSnrBVQBHPzmrJ/405JXY67h2fdFV3PrNbg/rcge4cQ/Xu9WkSS8kKyfGSyIXngSwExzw+kDL3SgbQgYl5TDqfv9Ajunei+VS9U+/ytUrnsaDjAv1vWD5U3DNmw95WPeZovyjMiOu9h3pj4F6v1rAXz0d0aYGfgM0bSDlCp2JLyi5fM0+8xWZrfG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fold __vfree_deferred into vfree_atomic, and call vfree_atomic early on from vfree if called from interrupt context so that the extra low-level helper can be avoided. Signed-off-by: Christoph Hellwig Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 43 +++++++++++++++++-------------------------- 1 file changed, 17 insertions(+), 26 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index b989828b45109a..fafb6227f4428f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2769,20 +2769,6 @@ static void __vunmap(const void *addr, int deallocate_pages) kfree(area); } -static inline void __vfree_deferred(const void *addr) -{ - /* - * Use raw_cpu_ptr() because this can be called from preemptible - * context. Preemption is absolutely fine here, because the llist_add() - * implementation is lockless, so it works even if we are adding to - * another cpu's list. schedule_work() should be fine with this too. - */ - struct vfree_deferred *p = raw_cpu_ptr(&vfree_deferred); - - if (llist_add((struct llist_node *)addr, &p->list)) - schedule_work(&p->wq); -} - /** * vfree_atomic - release memory allocated by vmalloc() * @addr: memory base address @@ -2792,13 +2778,19 @@ static inline void __vfree_deferred(const void *addr) */ void vfree_atomic(const void *addr) { - BUG_ON(in_nmi()); + struct vfree_deferred *p = raw_cpu_ptr(&vfree_deferred); + BUG_ON(in_nmi()); kmemleak_free(addr); - if (!addr) - return; - __vfree_deferred(addr); + /* + * Use raw_cpu_ptr() because this can be called from preemptible + * context. Preemption is absolutely fine here, because the llist_add() + * implementation is lockless, so it works even if we are adding to + * another cpu's list. schedule_work() should be fine with this too. + */ + if (addr && llist_add((struct llist_node *)addr, &p->list)) + schedule_work(&p->wq); } /** @@ -2820,17 +2812,16 @@ void vfree_atomic(const void *addr) */ void vfree(const void *addr) { - BUG_ON(in_nmi()); + if (unlikely(in_interrupt())) { + vfree_atomic(addr); + return; + } + BUG_ON(in_nmi()); kmemleak_free(addr); + might_sleep(); - might_sleep_if(!in_interrupt()); - - if (!addr) - return; - if (unlikely(in_interrupt())) - __vfree_deferred(addr); - else + if (addr) __vunmap(addr, 1); } EXPORT_SYMBOL(vfree);