From patchwork Sun Apr 7 13:08:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yunsheng Lin X-Patchwork-Id: 13620168 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02BD8CD128A for ; Sun, 7 Apr 2024 13:11:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 43D826B0096; Sun, 7 Apr 2024 09:11:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C7E86B0098; Sun, 7 Apr 2024 09:11:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CBF16B0099; Sun, 7 Apr 2024 09:11:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EB5056B0096 for ; Sun, 7 Apr 2024 09:11:18 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A718D40B10 for ; Sun, 7 Apr 2024 13:11:18 +0000 (UTC) X-FDA: 81982771836.12.13F7044 Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by imf23.hostedemail.com (Postfix) with ESMTP id 04622140009 for ; Sun, 7 Apr 2024 13:11:15 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712495476; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qN8xvUp/Pz6qhwsNImm1ke4lu/qTGOMKjtF+7ZY9FQE=; b=VwfPKNi2JSxAu3Hr4Tr4Tvs+UL9FkXYyynExTw4csq9s3eQMo7VPiJxUl3tjNrE6xrrG2a n6uNDSpYG3Q9lqhYTA/5AwLZPiBmBwFtm9i3Y3ZJizLJ4imz32P6UaHiJP7ltpR+97vxau gu4uAflvyEcRumxKpl3ukGz5WmM4gV4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of linyunsheng@huawei.com designates 45.249.212.189 as permitted sender) smtp.mailfrom=linyunsheng@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712495476; a=rsa-sha256; cv=none; b=fKRAvumrlXU21J987NEk+5GtFY9Zk9AJX6J4lmY1oG6i87ZKrX9tSUr9xCT5RnVgxl+Xd1 wPhNDKmATqRUkBgufkbq52x+CB0LhEvg2EiA3OkYBeYgw1CSAvLviElEOX4sUE5/JTNPeC e4acELnOla59yptKq+OqfnihDq7x6Qg= Received: from mail.maildlp.com (unknown [172.19.163.174]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4VCCGg1PZQzNn76; Sun, 7 Apr 2024 21:08:59 +0800 (CST) Received: from dggpemm500005.china.huawei.com (unknown [7.185.36.74]) by mail.maildlp.com (Postfix) with ESMTPS id 2B7ED1404F7; Sun, 7 Apr 2024 21:11:12 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by dggpemm500005.china.huawei.com (7.185.36.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Sun, 7 Apr 2024 21:11:11 +0800 From: Yunsheng Lin To: , , CC: , , Yunsheng Lin , Andrew Morton , Subject: [PATCH net-next v1 06/12] mm: page_frag: reuse MSB of 'size' field for pfmemalloc Date: Sun, 7 Apr 2024 21:08:43 +0800 Message-ID: <20240407130850.19625-7-linyunsheng@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20240407130850.19625-1-linyunsheng@huawei.com> References: <20240407130850.19625-1-linyunsheng@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm500005.china.huawei.com (7.185.36.74) X-Rspamd-Queue-Id: 04622140009 X-Rspam-User: X-Stat-Signature: 4oy85abjt736jj74cg8mkaq11q5cknma X-Rspamd-Server: rspam01 X-HE-Tag: 1712495475-530509 X-HE-Meta: U2FsdGVkX18md98p+AbgwqP/FM4yDjfYfFc3apoDwRwEQeFOw0EvYmSDNbEeQvCtvccqWC21ae7qazDW53CKyRAa58vHZ1hatNLUyQ+jDoZIw/bZvA/HU4i15QN/zCKJwc8DEWckNCRmVNbWxjvtA/ukDgbMiFv5YwDsyfu8sJyTELzoJg6dNZqlTHkMZSQ5fup2RrneYXQTBAO+OfjDLI4tnxip1Orl9WKNajBxWFZNFXJdyNS/mSRGHNWjmH4f2GYUpZ1WBxA9q+JPJ+kBKgWe6mqUSuQCPqUvqd0RLct/lsoeLqn+28stOT3c54LKpPHPNCvHjl2gtJSiOOTwz7wLAGG9r+Tolt3FSajbNEtw+KEd6kxMlvFsKIG9fNCOkx4qkhSWvlVKMhzgoLIqpeTsZ6uHPcUfWr3N2EkBVwFaXj17N1wx2Hjo6c0VBDYTd4TJSO9CnEoTh/MKWB6pc2f0dWbFyN+kDDOf8jSLjHIZrVjeEj5k7MVRNRdaCg97O1CAK7XgztPG3phgmgUXbYZZkVgoJQ9wtH+QJI48soeLjVXZZu0KnjskL4ecJ6xoWZeDcdGwbHgvlzV5hVXBETKSxf/2fBq73z6XZ33kWFYICitwq40UbuJJHFsXccupXw17mN7TkrbIn0CJSPsMjbfx6+22LDp+PWzLlf1BDiHSnSgevTsJXBG8SLlO4E/u8gKoyQ+JCRlEoI2/WnrCsEn0LcGijAVViqijaWoHZJKsYhVhBpxeoGh6USaW9Abz1Ggy2ZFH6NvRh4b7AyxC05WjFtBbkQ+3vvAHg+XgkPB4uD/1wxXhb1WEfeHhKwbKFpy4dYHhzjEK9ce4SefLHEH6CRPuKh3cfwAUm9RCHsKcw6FiezFHT+KZtiYulF9Kr5nAXtr1qzitRIN/uqYpWU2/Vtm1MCmu6Qhn1xJ4Nd8U0SH15czSzDOYzksTMa4oirdOsjemuzxreKSx3ny AtvX1IZI BbDzYXf00Y4GgtUObIfyS9HcZmYmliRQ2fOn7eKH4TLSmbAOjA79ZW0hlQFwp/mGLEiQV2rxZkuKA5Ryp50uibXdb0ivatahu/5ew1Mm6z/KA/Gvb38cGVIoarALs1MKTCqTvRGgJZs4B6htpmNotAU2KoG2MYHJBAlJIVw/ScLG8RaHORx0Ze680BgM5rUM95nbZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The '(PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE)' case is for the system with page size less than 32KB, which is 0x8000 bytes requiring 16 bits space, change 'size' to 'size_mask' to avoid using the MSB, and change 'pfmemalloc' field to reuse the that MSB, so that we remove the orginal space needed by 'pfmemalloc'. For another case, the MSB of 'offset' is reused for 'pfmemalloc'. Signed-off-by: Yunsheng Lin --- include/linux/page_frag_cache.h | 13 ++++++++----- mm/page_frag_cache.c | 5 +++-- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/include/linux/page_frag_cache.h b/include/linux/page_frag_cache.h index fe5faa80b6c3..40a7d6da9ef0 100644 --- a/include/linux/page_frag_cache.h +++ b/include/linux/page_frag_cache.h @@ -12,15 +12,16 @@ struct page_frag_cache { void *va; #if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) __u16 offset; - __u16 size; + __u16 size_mask:15; + __u16 pfmemalloc:1; #else - __u32 offset; + __u32 offset:31; + __u32 pfmemalloc:1; #endif /* we maintain a pagecount bias, so that we dont dirty cache line * containing page->_refcount every time we allocate a fragment. */ unsigned int pagecnt_bias; - bool pfmemalloc; }; static inline void page_frag_cache_init(struct page_frag_cache *nc) @@ -43,7 +44,9 @@ static inline void *__page_frag_alloc_va_align(struct page_frag_cache *nc, gfp_t gfp_mask, unsigned int align) { - nc->offset = ALIGN(nc->offset, align); + unsigned int offset = nc->offset; + + nc->offset = ALIGN(offset, align); return page_frag_alloc_va(nc, fragsz, gfp_mask); } @@ -53,7 +56,7 @@ static inline void *page_frag_alloc_va_align(struct page_frag_cache *nc, gfp_t gfp_mask, unsigned int align) { - WARN_ON_ONCE(!is_power_of_2(align)); + WARN_ON_ONCE(!is_power_of_2(align) || align >= PAGE_SIZE); return __page_frag_alloc_va_align(nc, fragsz, gfp_mask, align); } diff --git a/mm/page_frag_cache.c b/mm/page_frag_cache.c index 7f639af4e518..a02e57a439f0 100644 --- a/mm/page_frag_cache.c +++ b/mm/page_frag_cache.c @@ -29,7 +29,8 @@ static struct page *__page_frag_cache_refill(struct page_frag_cache *nc, __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC; page = alloc_pages_node(NUMA_NO_NODE, gfp_mask, PAGE_FRAG_CACHE_MAX_ORDER); - nc->size = page ? PAGE_FRAG_CACHE_MAX_SIZE : PAGE_SIZE; + nc->size_mask = page ? PAGE_FRAG_CACHE_MAX_SIZE - 1 : PAGE_SIZE - 1; + VM_BUG_ON(page && nc->size_mask != PAGE_FRAG_CACHE_MAX_SIZE - 1); #endif if (unlikely(!page)) page = alloc_pages_node(NUMA_NO_NODE, gfp, 0); @@ -88,7 +89,7 @@ void *page_frag_alloc_va(struct page_frag_cache *nc, unsigned int fragsz, #if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) /* if size can vary use size else just use PAGE_SIZE */ - size = nc->size; + size = nc->size_mask + 1; #else size = PAGE_SIZE; #endif