From patchwork Thu May 13 16:58:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matteo Croce X-Patchwork-Id: 12256015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F2E8C433ED for ; Thu, 13 May 2021 16:59:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B43A96141B for ; Thu, 13 May 2021 16:59:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B43A96141B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3CFE28D0001; Thu, 13 May 2021 12:59:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A6F96B0070; Thu, 13 May 2021 12:59:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2213C8D0001; Thu, 13 May 2021 12:59:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id E7B456B006E for ; Thu, 13 May 2021 12:59:33 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 7664DBBE8 for ; Thu, 13 May 2021 16:59:33 +0000 (UTC) X-FDA: 78136819026.33.0A41E6F Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf26.hostedemail.com (Postfix) with ESMTP id 69CB5407F8FA for ; Thu, 13 May 2021 16:59:33 +0000 (UTC) Received: by mail-ej1-f51.google.com with SMTP id gx5so40796535ejb.11 for ; Thu, 13 May 2021 09:59:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rxLHJd8XD9Hb4Rf+opd9ZLjsRH1xJ79b5KXjgff+Bjc=; b=Yi4va5xenXMW76Wqk5elxXM7nmdq0X5qrr09ZPfkqYTr5RRyJt0ExUIy3W0lPhLbgf wocnlp3y4bCOeB6E98ENKisB5YrfGLJNzvWbaDBEEFTpLtgCKLEVnnjTR946mWcvmqMk rCS5t0iTCL6X4w1TzYphGpdQPeNiFRHJGuB0zfesDilvBNEM/QvJTw3s3nBkwxEs3q3W za/bBWkVZCNLHgPXan7nelL7sa5Ji13BjMyIDrw0OydmSglyuWcbbdtZ3UXjJh5yaVnY wh5PBoshGN+Fu9cWYBL9y7CswfJaQDJL6hmbRMkkr2HtBQZCeHWQqxH2ksPDlSG+lrJI w9tA== X-Gm-Message-State: AOAM5330kBg88ytxTB26cg/Iakd8BPTk0NAbJ6844hc7bXwHnZ21KnQy Xzud/l4UzQnk5D+yCA7b60M= X-Google-Smtp-Source: ABdhPJzNfQFARn/zPNO4hoyHVkKONaOFliYJKvo89sBhiZsOEvjF+mAbgI9X4R5U6JV8zjP2yJmHUQ== X-Received: by 2002:a17:906:b6c5:: with SMTP id ec5mr44910742ejb.290.1620925171875; Thu, 13 May 2021 09:59:31 -0700 (PDT) Received: from msft-t490s.teknoraver.net (net-5-94-253-60.cust.vodafonedsl.it. [5.94.253.60]) by smtp.gmail.com with ESMTPSA id w11sm2959431ede.54.2021.05.13.09.59.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 May 2021 09:59:31 -0700 (PDT) From: Matteo Croce To: netdev@vger.kernel.org, linux-mm@kvack.org Cc: Ayush Sawal , Vinay Kumar Yadav , Rohit Maheshwari , "David S. Miller" , Jakub Kicinski , Thomas Petazzoni , Marcin Wojtas , Russell King , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Jesper Dangaard Brouer , Ilias Apalodimas , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Boris Pismenny , Arnd Bergmann , Andrew Morton , "Peter Zijlstra (Intel)" , Vlastimil Babka , Yu Zhao , Will Deacon , Fenghua Yu , Roman Gushchin , Hugh Dickins , Peter Xu , Jason Gunthorpe , Jonathan Lemon , Alexander Lobakin , Cong Wang , wenxu , Kevin Hao , Jakub Sitnicki , Marco Elver , Willem de Bruijn , Miaohe Lin , Yunsheng Lin , Guillaume Nault , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, Matthew Wilcox , Eric Dumazet , David Ahern , Lorenzo Bianconi , Saeed Mahameed , Andrew Lunn , Paolo Abeni , Sven Auhagen Subject: [PATCH net-next v5 1/5] mm: add a signature in struct page Date: Thu, 13 May 2021 18:58:42 +0200 Message-Id: <20210513165846.23722-2-mcroce@linux.microsoft.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210513165846.23722-1-mcroce@linux.microsoft.com> References: <20210513165846.23722-1-mcroce@linux.microsoft.com> MIME-Version: 1.0 Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=linux.microsoft.com (policy=none); spf=pass (imf26.hostedemail.com: domain of technoboy85@gmail.com designates 209.85.218.51 as permitted sender) smtp.mailfrom=technoboy85@gmail.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 69CB5407F8FA X-Stat-Signature: 4qmbp9yf9f7baxo4biy8ydjbqtw7c33x X-HE-Tag: 1620925173-353963 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Matteo Croce This is needed by the page_pool to avoid recycling a page not allocated via page_pool. The page->signature field is aliased to page->lru.next and page->compound_head, but it can't be set by mistake because the signature value is a bad pointer, and can't trigger a false positive in PageTail() because the last bit is 0. Co-developed-by: Matthew Wilcox (Oracle) Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Matteo Croce --- include/linux/mm.h | 12 +++++++----- include/linux/mm_types.h | 12 ++++++++++++ include/net/page_pool.h | 2 ++ net/core/page_pool.c | 4 ++++ 4 files changed, 25 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 322ec61d0da7..48268d2d0282 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1668,10 +1668,12 @@ struct address_space *page_mapping(struct page *page); static inline bool page_is_pfmemalloc(const struct page *page) { /* - * Page index cannot be this large so this must be - * a pfmemalloc page. + * This is not a tail page; compound_head of a head page is unused + * at return from the page allocator, and will be overwritten + * by callers who do not care whether the page came from the + * reserves. */ - return page->index == -1UL; + return page->compound_head & 2; } /* @@ -1680,12 +1682,12 @@ static inline bool page_is_pfmemalloc(const struct page *page) */ static inline void set_page_pfmemalloc(struct page *page) { - page->index = -1UL; + page->compound_head = 2; } static inline void clear_page_pfmemalloc(struct page *page) { - page->index = 0; + page->compound_head = 0; } /* diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5aacc1c10a45..44cf328e94e2 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -96,6 +96,18 @@ struct page { unsigned long private; }; struct { /* page_pool used by netstack */ + /** + * @pp_magic: magic value to avoid recycling non + * page_pool allocated pages. + * It aliases with page->lru.next + */ + unsigned long pp_magic; + /** + * @pp: pointer to page_pool. + * It aliases with page->lru.prev + */ + struct page_pool *pp; + unsigned long _pp_mapping_pad; /** * @dma_addr: might require a 64-bit value on * 32-bit architectures. diff --git a/include/net/page_pool.h b/include/net/page_pool.h index b4b6de909c93..24b3d42c62c0 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -63,6 +63,8 @@ */ #define PP_ALLOC_CACHE_SIZE 128 #define PP_ALLOC_CACHE_REFILL 64 +#define PP_SIGNATURE (POISON_POINTER_DELTA + 0x40) + struct pp_alloc_cache { u32 count; struct page *cache[PP_ALLOC_CACHE_SIZE]; diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 3c4c4c7a0402..9de5d8c08c17 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -221,6 +221,8 @@ static struct page *__page_pool_alloc_page_order(struct page_pool *pool, return NULL; } + page->pp_magic = PP_SIGNATURE; + /* Track how many pages are held 'in-flight' */ pool->pages_state_hold_cnt++; trace_page_pool_state_hold(pool, page, pool->pages_state_hold_cnt); @@ -341,6 +343,8 @@ void page_pool_release_page(struct page_pool *pool, struct page *page) DMA_ATTR_SKIP_CPU_SYNC); page_pool_set_dma_addr(page, 0); skip_dma_unmap: + page->pp_magic = 0; + /* This may be the last page returned, releasing the pool, so * it is not safe to reference pool afterwards. */