From patchwork Sat Jan 21 07:10:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13111015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C27CBC004D4 for ; Sat, 21 Jan 2023 07:11:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5CA2E8E0003; Sat, 21 Jan 2023 02:11:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 57B848E0001; Sat, 21 Jan 2023 02:11:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 444198E0003; Sat, 21 Jan 2023 02:11:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2E9848E0001 for ; Sat, 21 Jan 2023 02:11:24 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1401540F88 for ; Sat, 21 Jan 2023 07:11:24 +0000 (UTC) X-FDA: 80377935288.15.75EFDE8 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf18.hostedemail.com (Postfix) with ESMTP id 6C03F1C000B for ; Sat, 21 Jan 2023 07:11:22 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=vk156WbC; spf=none (imf18.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674285082; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2FZpTb+3bZJMUpb6YA7LoDb6w4aUjCraFIBMJjAahss=; b=0NZxR9ndk07U8l5qPDL1RpFFyMA7j3/jFHKMCHpnh5nST6BUMMFt1jamLJDK2WVTGFV3UT K0po480plG6dhRkBoYY9+XQyCl+Qvgb+76TSmLVHwrshpBjhmrOoWc5CZNHyTydrHs6mST 1OM2K5ZKV0k45VGPgksoSf1/UnbRYdg= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=vk156WbC; spf=none (imf18.hostedemail.com: domain of BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=BATV+1651c3ebed9361b307e7+7090+infradead.org+hch@bombadil.srs.infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674285082; a=rsa-sha256; cv=none; b=qWLyCPO8uMYWqPUi6Pa/QBDuznHxZpEruW14KDI2nZjwFz5Jwa9IcXMq+cV1o+rx1/YHKl W+F+e3g2fY7LRZ5FWlfTeqVM+UbNjeSrOhPpKeP1iWu7nM09o6Ci2hno6YjJMAXEhc3bB2 qrk6FOumKq7McFGC/ohjZbNTpIceFXM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=2FZpTb+3bZJMUpb6YA7LoDb6w4aUjCraFIBMJjAahss=; b=vk156WbCNmHsZu4P/efUcpqIgq e4L1tNlJXOHKIraBOpmRmbAlKkiW6f42LHRLqxAKsLV05UPGq05Z9DvX2G4VT4KMTJ3UpsfaV2wcS nCNKXGyderjQ87xrHOZ2rPFXm7hIvnjk7GfNF9XGfSTVUcDCkdAHdr4l+5RhNCqfjnBeu9HQfW3fj CSufSOGC/oC6ozUgSD90d8zTHXpqIm6G/qvNELnzGY44tP7P9DQ8iJh3K7BwjChgKrql/CK/XdHGw 3hE1vZOdDEszNp76ffJGRv+kcYGwlMJONWpH76zXC187SEkyP7l5Iyu6pcQQVUjHjCDi6CTYyJPYn ExbvTVhw==; Received: from [2001:4bb8:19a:2039:6754:cc81:9ace:36fc] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1pJ821-00DTsL-5M; Sat, 21 Jan 2023 07:11:17 +0000 From: Christoph Hellwig To: Andrew Morton , Uladzislau Rezki Cc: Andrey Ryabinin , Alexander Potapenko , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , kasan-dev@googlegroups.com, linux-mm@kvack.org Subject: [PATCH 09/10] mm: split __vunmap Date: Sat, 21 Jan 2023 08:10:50 +0100 Message-Id: <20230121071051.1143058-10-hch@lst.de> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230121071051.1143058-1-hch@lst.de> References: <20230121071051.1143058-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6C03F1C000B X-Rspam-User: X-Stat-Signature: 75n17tdeeze9tccagq5dpc4396rqak8a X-HE-Tag: 1674285082-184560 X-HE-Meta: U2FsdGVkX1+Ud8UfHqY9WHqrPmPAcuWr5E0dEE6/dLKgwNgWKSzCBBi0db1jXQlcVLzcpQUwyAUvAtBZCOjD9ZFSFIw8SrX491MyFfF5df6WCXxFLku6WpDcKl0iHxaQ9TOQqwyzkFrYJhWSGiglR5jJxiPPJoKlcz8Obg2XiN9OyimrjlzGxgRUKEUXfCnGnAwZK15yMJfYBjX5DQ0ZdtQ+QPnNczEtVNGNnxOmseTkmhYbf++PoaelY0l+pjdZzVuXnTdpufZqlOkTFAZ/5eaarPb12P+I+N3TibqWQQAPb5WatI9vPCzzFGOGTqAy1kZFeydgxmas165YOSWtW5IxX6QpQoD9JTvAJuU6ivS1Yyax3zjaAdj/EgmBK24muc8KvMO3TPmRr2Vr8UMN5/dlXQIRaBvrjWGkagL/pDDOK7d93T/YZYN784KERJc+LdBK/Fm9w+d6/qodTEXqg25sx6eeaqMtsUwR5VNKRDW5rxgl2Cg3nI10CKbHyBCokyDr5qE1NxdEPVXXijoOdbddBMIBTbcGscSg9Mqojuj+WBHMal8M+oEhQ/mGwfY0DzghpBI9N3Zp5bRi2L/H6VE9vm9JYN64Lu8YRfwGNUyDIa+E8yCjUM+Dzxk4fBg8GHPx4I6CBrPugOwwQFLFZ0Y7DrhCfzhdZQu9D/E0AXJGwpkJLYnxxxjJfs5oHT5n4Rg46Z+Fs+guiiwXTxweLGa15k6c/Dh/LFuJOy9eRBOJLwhqOZRfwspUXP+opKjRHcXy5GjEQfAC80XyghpCmVwN6Ola4OC35W0c8M8XLdunZuX/d9R75H9a4xAMFewRYkxI2+NsOFRKA8q21DWinc/qzmnNFdehiAu8/14RoMrKbyOF2FYg/ExYfCCwm2HJKZWdkoMkT8cV6AmBL/uh56ZoJbEw6scBQWlHMeTIs3VYZDOKpWEZt7O7V/JJjkghIPTA/TjKi/+r5VpVBMI V1kZmK2/ Gvfg341oyhBxebx8G7O8iTP2tBQ8dYO+lLgr0CoRXYGD6qwz+6onu6JnbastYi4TCJlMaEvVRc5GpCYsE5AHKvpm3FE1NRN/h6VdfneKXfzlFe6TMWxbiRgYtkXyKAIasq6NpaTG4d1kjSaU8n3txZs3iZlkDBfwuSHIMex7J1ts7zNxdmySjQOLiQj0U/yEtccUlQB0zQrK5IL2aybbx2OXUwKBvzSoBqEEb4I2mAesMOOtEkFXICHkRrkMCK9tSAuofGrHNo7W0HAyhpt2Tn40xVVMCFopYx4z2lBIS5wkvcAfOLEPienmrZSohaTvoamdiOC7Jq7ILF4TIdj59R3OBSClaXGKv3xQFiNg9/mnDc5d56QnVrjFIM3S1oFXVj45VfdQQnMupxhnMrqeGy88Qkdv4yiGJpdzi5q+f9w6PLOkmsz98/hOwJSCe6HZGlTKoy0ygqy9twSC9dp2SBRrnDwnh3VvfDXEur8X7rYUXPoi+MBFHegjOyVwC67Ou5ZgftAR4YlUmisNLdVerUbwureCxuhHEPT6sSFHB9lAa7kA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: vunmap only needs to find and free the vmap_area and vm_strut, so open code that there and merge the rest of the code into vfree. Signed-off-by: Christoph Hellwig Reviewed-by: David Hildenbrand --- mm/vmalloc.c | 84 +++++++++++++++++++++++++--------------------------- 1 file changed, 41 insertions(+), 43 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 5b432508319a4f..6bd811e4b7561d 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2666,45 +2666,6 @@ static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) set_area_direct_map(area, set_direct_map_default_noflush); } -static void __vunmap(const void *addr, int deallocate_pages) -{ - struct vm_struct *area; - - if (!addr) - return; - - area = remove_vm_area(addr); - if (unlikely(!area)) { - WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", - addr); - return; - } - - vm_remove_mappings(area, deallocate_pages); - - if (deallocate_pages) { - int i; - - for (i = 0; i < area->nr_pages; i++) { - struct page *page = area->pages[i]; - - BUG_ON(!page); - mod_memcg_page_state(page, MEMCG_VMALLOC, -1); - /* - * High-order allocs for huge vmallocs are split, so - * can be freed as an array of order-0 allocations - */ - __free_pages(page, 0); - cond_resched(); - } - atomic_long_sub(area->nr_pages, &nr_vmalloc_pages); - - kvfree(area->pages); - } - - kfree(area); -} - static void delayed_vfree_work(struct work_struct *w) { struct vfree_deferred *p = container_of(w, struct vfree_deferred, wq); @@ -2757,6 +2718,9 @@ void vfree_atomic(const void *addr) */ void vfree(const void *addr) { + struct vm_struct *vm; + int i; + if (unlikely(in_interrupt())) { vfree_atomic(addr); return; @@ -2766,8 +2730,32 @@ void vfree(const void *addr) kmemleak_free(addr); might_sleep(); - if (addr) - __vunmap(addr, 1); + if (!addr) + return; + + vm = remove_vm_area(addr); + if (unlikely(!vm)) { + WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", + addr); + return; + } + + vm_remove_mappings(vm, true); + for (i = 0; i < vm->nr_pages; i++) { + struct page *page = vm->pages[i]; + + BUG_ON(!page); + mod_memcg_page_state(page, MEMCG_VMALLOC, -1); + /* + * High-order allocs for huge vmallocs are split, so + * can be freed as an array of order-0 allocations + */ + __free_pages(page, 0); + cond_resched(); + } + atomic_long_sub(vm->nr_pages, &nr_vmalloc_pages); + kvfree(vm->pages); + kfree(vm); } EXPORT_SYMBOL(vfree); @@ -2782,10 +2770,20 @@ EXPORT_SYMBOL(vfree); */ void vunmap(const void *addr) { + struct vm_struct *vm; + BUG_ON(in_interrupt()); might_sleep(); - if (addr) - __vunmap(addr, 0); + + if (!addr) + return; + vm = remove_vm_area(addr); + if (unlikely(!vm)) { + WARN(1, KERN_ERR "Trying to vunmap() nonexistent vm area (%p)\n", + addr); + return; + } + kfree(vm); } EXPORT_SYMBOL(vunmap);