From patchwork Wed Aug 19 18:48:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724877 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F094739 for ; Wed, 19 Aug 2020 18:49:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6C678207DE for ; Wed, 19 Aug 2020 18:49:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="JPo0Ggqo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C678207DE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D75F96E5AB; Wed, 19 Aug 2020 18:49:05 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id C0D966E5B0 for ; Wed, 19 Aug 2020 18:49:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ND98uMZYrcjfbVhxga4enxbqENG9QME9SJj8HZvY1Rw=; b=JPo0GgqofTk7M5A0cGUAOy99T2 5nHqBXZ+m83jatnjFvA0RKJJ/SDhAoR8rXyzbhaBPVI1ig31d/FaB9LjNYGIPTUJOsk5E3HSrLbS4 NXj00j+SXuSiu6Xl3KXQhcIGyvr/vPp7cvZIRk0alqg2evX3lW37FqP56ZOrlvjeCYAc5TJyTF7kG tKvYFXrnqSLamGVH1LjMrnspOcwR97NVZkxKZ/b+gcDGQe/WPt0hHvruO4I8QTyReFX1IezLYtTLS wuHER1m2boO71ITzeQmzJBMJmi9jH9hs3Rw1QFCTk6q7j9BDEQ9Lj+120aiy7ZnDSlOwyMloN7scs JYe52Trw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8m-0006Ss-Tl; Wed, 19 Aug 2020 18:48:52 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:43 +0100 Message-Id: <20200819184850.24779-2-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 1/8] mm: Factor find_get_swap_page out of mincore_page X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Provide this functionality from the swap cache. It's useful for more than just mincore(). Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/swap.h | 7 +++++++ mm/mincore.c | 28 ++-------------------------- mm/swap_state.c | 31 +++++++++++++++++++++++++++++++ 3 files changed, 40 insertions(+), 26 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 661046994db4..247527ea3e66 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -427,6 +427,7 @@ extern void free_pages_and_swap_cache(struct page **, int); extern struct page *lookup_swap_cache(swp_entry_t entry, struct vm_area_struct *vma, unsigned long addr); +struct page *find_get_swap_page(struct address_space *mapping, pgoff_t index); extern struct page *read_swap_cache_async(swp_entry_t, gfp_t, struct vm_area_struct *vma, unsigned long addr, bool do_poll); @@ -569,6 +570,12 @@ static inline struct page *lookup_swap_cache(swp_entry_t swp, return NULL; } +static inline +struct page *find_get_swap_page(struct address_space *mapping, pgoff_t index) +{ + return find_get_page(mapping, index); +} + static inline int add_to_swap(struct page *page) { return 0; diff --git a/mm/mincore.c b/mm/mincore.c index 453ff112470f..2806258f99c6 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -48,7 +48,7 @@ static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long addr, * and is up to date; i.e. that no page-in operation would be required * at this time if an application were to map and access this page. */ -static unsigned char mincore_page(struct address_space *mapping, pgoff_t pgoff) +static unsigned char mincore_page(struct address_space *mapping, pgoff_t index) { unsigned char present = 0; struct page *page; @@ -59,31 +59,7 @@ static unsigned char mincore_page(struct address_space *mapping, pgoff_t pgoff) * any other file mapping (ie. marked !present and faulted in with * tmpfs's .fault). So swapped out tmpfs mappings are tested here. */ -#ifdef CONFIG_SWAP - if (shmem_mapping(mapping)) { - page = find_get_entry(mapping, pgoff); - /* - * shmem/tmpfs may return swap: account for swapcache - * page too. - */ - if (xa_is_value(page)) { - swp_entry_t swp = radix_to_swp_entry(page); - struct swap_info_struct *si; - - /* Prevent swap device to being swapoff under us */ - si = get_swap_device(swp); - if (si) { - page = find_get_page(swap_address_space(swp), - swp_offset(swp)); - put_swap_device(si); - } else - page = NULL; - } - } else - page = find_get_page(mapping, pgoff); -#else - page = find_get_page(mapping, pgoff); -#endif + page = find_get_swap_page(mapping, index); if (page) { present = PageUptodate(page); put_page(page); diff --git a/mm/swap_state.c b/mm/swap_state.c index c16eebb81d8b..986b4e3d3bad 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -414,6 +414,37 @@ struct page *lookup_swap_cache(swp_entry_t entry, struct vm_area_struct *vma, return page; } +/** + * find_get_swap_page - Find and get a page from the page or swap caches. + * @mapping: The address_space to search. + * @index: The page cache index. + * + * This is like find_get_page(), but if we find a swap entry for + * shmem/tmpfs, we also look in the swap cache for the page. + * + * Return: The found page or %NULL. + */ +struct page *find_get_swap_page(struct address_space *mapping, pgoff_t index) +{ + swp_entry_t swp; + struct swap_info_struct *si; + struct page *page = find_get_entry(mapping, index); + + if (!xa_is_value(page)) + return page; + if (!shmem_mapping(mapping)) + return NULL; + + swp = radix_to_swp_entry(page); + /* Prevent swapoff from happening to us */ + si = get_swap_device(swp); + if (!si) + return NULL; + page = find_get_page(swap_address_space(swp), swp_offset(swp)); + put_swap_device(si); + return page; +} + struct page *__read_swap_cache_async(swp_entry_t entry, gfp_t gfp_mask, struct vm_area_struct *vma, unsigned long addr, bool *new_page_allocated) From patchwork Wed Aug 19 18:48:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724893 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC54E618 for ; Wed, 19 Aug 2020 18:49:13 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CABFE207DE for ; Wed, 19 Aug 2020 18:49:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="b4pHEvLP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CABFE207DE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 273CC6E5B6; Wed, 19 Aug 2020 18:49:13 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 11FE86E5B6 for ; Wed, 19 Aug 2020 18:49:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=66DP4Pf9efccaQIqwctMuv9XwKLU5wWgwMF4qWqr0Vk=; b=b4pHEvLPG1hkKsnmbfQrrqTHYi VWkIBNDtjyN9c8Z4LmNDcpjHj7yq6xtVSvSuiJUBrLhJ6I96lrnl+ChJ/s76uL2F4feghrlNFAQ1N MsxinyVoKVilyk25tpijsshqsL4+yiO4m5XrhCgIu6pXy0OOAd7tzIkD89C5t7Qz1KUbuj5qz01fd MSmojCrJigJEHzAYVnM3DkoJTYfyYNJlXIZjcizjLjuA9IesPN3cy97gRx00+1QRiJdbmQw8GmnLe IJOGDVShbQY8CWQ6aDu09E/pboQS5RIZ4Q1K2ryIsClNEi9KQZN5QbfFYiHPrEQ9S1eZ6yfVivX38 rEAPV0mw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8n-0006Sx-83; Wed, 19 Aug 2020 18:48:53 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:44 +0100 Message-Id: <20200819184850.24779-3-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 2/8] mm: Use find_get_swap_page in memcontrol X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" The current code does not protect against swapoff of the underlying swap device, so this is a bug fix as well as a worthwhile reduction in code complexity. Signed-off-by: Matthew Wilcox (Oracle) --- mm/memcontrol.c | 25 ++----------------------- 1 file changed, 2 insertions(+), 23 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b807952b4d43..4075f214a841 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5539,35 +5539,14 @@ static struct page *mc_handle_swap_pte(struct vm_area_struct *vma, static struct page *mc_handle_file_pte(struct vm_area_struct *vma, unsigned long addr, pte_t ptent, swp_entry_t *entry) { - struct page *page = NULL; - struct address_space *mapping; - pgoff_t pgoff; - if (!vma->vm_file) /* anonymous vma */ return NULL; if (!(mc.flags & MOVE_FILE)) return NULL; - mapping = vma->vm_file->f_mapping; - pgoff = linear_page_index(vma, addr); - /* page is moved even if it's not RSS of this task(page-faulted). */ -#ifdef CONFIG_SWAP - /* shmem/tmpfs may report page out on swap: account for that too. */ - if (shmem_mapping(mapping)) { - page = find_get_entry(mapping, pgoff); - if (xa_is_value(page)) { - swp_entry_t swp = radix_to_swp_entry(page); - *entry = swp; - page = find_get_page(swap_address_space(swp), - swp_offset(swp)); - } - } else - page = find_get_page(mapping, pgoff); -#else - page = find_get_page(mapping, pgoff); -#endif - return page; + return find_get_swap_page(vma->vm_file->f_mapping, + linear_page_index(vma, addr)); } /** From patchwork Wed Aug 19 18:48:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724881 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F87C618 for ; Wed, 19 Aug 2020 18:49:08 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E6A4207DE for ; Wed, 19 Aug 2020 18:49:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mtqG2R5K" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E6A4207DE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3C8C36E5B0; Wed, 19 Aug 2020 18:49:07 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id C3C2E6E5B2 for ; Wed, 19 Aug 2020 18:49:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=yHlibx9T0xrkfaCUzHO6K9mta5VlhggyQEWPghnljAs=; b=mtqG2R5K+Yx4dE8Kn6VbN3423G kIyD7iHdEs9//9zn4d+MqCrVYAzK+wcoAiner0u6ns9sOmIc5tfRAgo27IXLWgMP6nwGY0e6y6pmA 3HyU19g7ZHaw31ZJ8zxxv6IAybbAuxOaFrkQK2spsvcHh6QuyTpP2yNUbuYLJtWlcWRvqXSIlF0jI 9Vy40hmXVmrp6grQd7r+oJ7KtTvTI521d9fNWtfKyyqTvGJ4P+MhRWJZc+95qtI0mQz6V/ZnZbfe4 Kgct1GcrVAvfxOQF0Ag9wTk4R925hXyZsT3fcWZRlbRDSahohQn09B2qh4/GBAkF9CnxwnjMmjRTs EKPHJUnA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8n-0006T8-TC; Wed, 19 Aug 2020 18:48:53 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:45 +0100 Message-Id: <20200819184850.24779-4-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 3/8] mm: Optimise madvise WILLNEED X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Instead of calling find_get_entry() for every page index, use an XArray iterator to skip over NULL entries, and avoid calling get_page(), because we only want the swap entries. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner --- mm/madvise.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index dd1d43cf026d..96189acd6969 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -224,25 +224,28 @@ static void force_shm_swapin_readahead(struct vm_area_struct *vma, unsigned long start, unsigned long end, struct address_space *mapping) { - pgoff_t index; + XA_STATE(xas, &mapping->i_pages, linear_page_index(vma, start)); + pgoff_t end_index = end / PAGE_SIZE; struct page *page; - swp_entry_t swap; - for (; start < end; start += PAGE_SIZE) { - index = ((start - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff; + rcu_read_lock(); + xas_for_each(&xas, page, end_index) { + swp_entry_t swap; - page = find_get_entry(mapping, index); - if (!xa_is_value(page)) { - if (page) - put_page(page); + if (!xa_is_value(page)) continue; - } + rcu_read_unlock(); + swap = radix_to_swp_entry(page); page = read_swap_cache_async(swap, GFP_HIGHUSER_MOVABLE, NULL, 0, false); if (page) put_page(page); + + rcu_read_lock(); + xas_reset(&xas); } + rcu_read_unlock(); lru_add_drain(); /* Push any new pages onto the LRU now */ } From patchwork Wed Aug 19 18:48:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724891 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B987739 for ; Wed, 19 Aug 2020 18:49:12 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3985D20758 for ; Wed, 19 Aug 2020 18:49:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mpxwSwSB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3985D20758 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 525826E5BB; Wed, 19 Aug 2020 18:49:09 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id BFF8A6E5AB for ; Wed, 19 Aug 2020 18:49:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=6FEH2RzEKpeGkw1ry8PIMFPV7YWAJWwmC36GZdEh8I8=; b=mpxwSwSBY6UqYCwG+sE65V35BB J8yzFZhDe0QRMl1ZP/nlpb2+WKDQLYNkDnz76pIkiUTqTr189msFIwBJ8Mr8+Fb5eKxFjMBlC264q MFMM2mcRXQAZBJw4EqOpTiDhiJTFSkQjlg6ucjHSbR1dPWYPrVqiLTTEa+IgaEziprwqMjqE/2vJN 8HJKcVy+CFy9D2tDrJzoPrvYKkIeAngzZupTTfRif0cbwdr6/FFFFOSjbC7q0zK/5WOG5hKulHIBH VFCsPpPgHZOLoqSynTDkWtpsghEfDw6rADRHNjdQRwN5rhOkDtE0OXzWVqUtA/fsw+CJSKW2WX9+8 OaTNPKkg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8o-0006TI-Gl; Wed, 19 Aug 2020 18:48:54 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:46 +0100 Message-Id: <20200819184850.24779-5-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 4/8] proc: Optimise smaps for shmem entries X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Avoid bumping the refcount on pages when we're only interested in the swap entries. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner --- fs/proc/task_mmu.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5066b0251ed8..e42d9e5e9a3c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -520,16 +520,10 @@ static void smaps_pte_entry(pte_t *pte, unsigned long addr, page = device_private_entry_to_page(swpent); } else if (unlikely(IS_ENABLED(CONFIG_SHMEM) && mss->check_shmem_swap && pte_none(*pte))) { - page = find_get_entry(vma->vm_file->f_mapping, + page = xa_load(&vma->vm_file->f_mapping->i_pages, linear_page_index(vma, addr)); - if (!page) - return; - if (xa_is_value(page)) mss->swap += PAGE_SIZE; - else - put_page(page); - return; } From patchwork Wed Aug 19 18:48:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724895 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D404A618 for ; Wed, 19 Aug 2020 18:49:14 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B26F020758 for ; Wed, 19 Aug 2020 18:49:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="bV5mILzX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B26F020758 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 631F56E5BE; Wed, 19 Aug 2020 18:49:13 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id B8FAB6E5B0 for ; Wed, 19 Aug 2020 18:49:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=lHH3FI1tdZZ9OqCoI613Ez6EcSFNmDwjhfuwAt20rC0=; b=bV5mILzXSPzh12r4bbJtUTmcyK mmNzxSnEJpTqrEIi28olHtDus3qRFErxyqk3CfVa6Nd4m1tQJRl4IgaCasF0ja9tQDItvvrax3GnS 8UkqbyOfosJ5Q1ytEse0bM/qsh4yxOC1BjxdwFk5+1XwRJS5Mzl9pD0gRQiZFy1b0n+04o76H48LY t3U9M07QP7XUBrtP90Ne92iZXCDHymovYeKar5RSM0+3U/Kkv4PkKew1BHT59pZ55s+bYN1mtFgId s+FXFgC5nX3xOrYqcNbRh6at1KntZUZAJORErYWO3YvaY8XYRm8CaujUuJjZ+cHzX44rlsGU7au8p ZnGIytaQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8p-0006Te-2y; Wed, 19 Aug 2020 18:48:55 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:47 +0100 Message-Id: <20200819184850.24779-6-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 5/8] i915: Use find_lock_page instead of find_lock_entry X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" i915 does not want to see value entries. Switch it to use find_lock_page() instead, and remove the export of find_lock_entry(). Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Johannes Weiner --- drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 4 ++-- mm/filemap.c | 1 - 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index 38113d3c0138..75e8b71c18b9 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -258,8 +258,8 @@ shmem_writeback(struct drm_i915_gem_object *obj) for (i = 0; i < obj->base.size >> PAGE_SHIFT; i++) { struct page *page; - page = find_lock_entry(mapping, i); - if (!page || xa_is_value(page)) + page = find_lock_page(mapping, i); + if (!page) continue; if (!page_mapped(page) && clear_page_dirty_for_io(page)) { diff --git a/mm/filemap.c b/mm/filemap.c index 1aaea26556cc..36067cf7f8c5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1628,7 +1628,6 @@ struct page *find_lock_entry(struct address_space *mapping, pgoff_t offset) } return page; } -EXPORT_SYMBOL(find_lock_entry); /** * pagecache_get_page - Find and get a reference to a page. From patchwork Wed Aug 19 18:48:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724897 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDC67618 for ; Wed, 19 Aug 2020 18:49:15 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C50120758 for ; Wed, 19 Aug 2020 18:49:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="fHiweH4x" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C50120758 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8DECD6E5C0; Wed, 19 Aug 2020 18:49:13 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 025C56E5B0 for ; Wed, 19 Aug 2020 18:49:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=SFuii2T6WPhgcSR/n6o4MUthReftxuKz6bFegDk7V6Y=; b=fHiweH4xvcHxBZBo/Hs4Zh/x8q IOR/lnZgsQ40NMoRAkmTBSFUeWvZ+BhxiVYwvmJ6/Pq6aCH0ZcHkN4WQpVwHrbcFgnTPrMOR5ZG+2 sb+yfADxrEVowOqFCkiUp4DXUwwrI2Aoz5w+0Ps1J3NhxsPfFcad6NmL7IL38er0SZfwH6JR7ji29 hGF/hfcvukAn8t3r6FgFgVC4LRBRJHcWoDjO9oczio+FTlp8gZq5Lyn6eyhG8b1Bik8v4IiSoW96m g4sKNy+aJq94mEab3qR1Xgw19wyvZ83QaPb2dmDmI8OXMdf9Fl7fauESRq9u9FzhI8WObi5xkFtEh ACKe314Q==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8p-0006Tz-Hr; Wed, 19 Aug 2020 18:48:55 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:48 +0100 Message-Id: <20200819184850.24779-7-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 6/8] mm: Convert find_get_entry to return the head page X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" There are only three callers remaining of find_get_entry(). find_get_swap_page() is happy to get the head page instead of the subpage. Add find_subpage() calls to find_lock_entry() and pagecache_get_page() to avoid auditing all their callers. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/pagemap.h | 4 ++-- mm/filemap.c | 13 +++++++------ 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 7de11dcd534d..8261c64f592d 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -384,8 +384,8 @@ static inline struct page *find_subpage(struct page *head, pgoff_t index) return head + (index & (thp_nr_pages(head) - 1)); } -struct page *find_get_entry(struct address_space *mapping, pgoff_t offset); -struct page *find_lock_entry(struct address_space *mapping, pgoff_t offset); +struct page *find_get_entry(struct address_space *mapping, pgoff_t index); +struct page *find_lock_entry(struct address_space *mapping, pgoff_t index); unsigned find_get_entries(struct address_space *mapping, pgoff_t start, unsigned int nr_entries, struct page **entries, pgoff_t *indices); diff --git a/mm/filemap.c b/mm/filemap.c index 36067cf7f8c5..5a87e1bdddd6 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1547,19 +1547,19 @@ EXPORT_SYMBOL(page_cache_prev_miss); /** * find_get_entry - find and get a page cache entry * @mapping: the address_space to search - * @offset: the page cache index + * @index: The page cache index. * * Looks up the page cache slot at @mapping & @offset. If there is a - * page cache page, it is returned with an increased refcount. + * page cache page, the head page is returned with an increased refcount. * * If the slot holds a shadow entry of a previously evicted page, or a * swap entry from shmem/tmpfs, it is returned. * - * Return: the found page or shadow entry, %NULL if nothing is found. + * Return: The head page or shadow entry, %NULL if nothing is found. */ -struct page *find_get_entry(struct address_space *mapping, pgoff_t offset) +struct page *find_get_entry(struct address_space *mapping, pgoff_t index) { - XA_STATE(xas, &mapping->i_pages, offset); + XA_STATE(xas, &mapping->i_pages, index); struct page *page; rcu_read_lock(); @@ -1587,7 +1587,6 @@ struct page *find_get_entry(struct address_space *mapping, pgoff_t offset) put_page(page); goto repeat; } - page = find_subpage(page, offset); out: rcu_read_unlock(); @@ -1624,6 +1623,7 @@ struct page *find_lock_entry(struct address_space *mapping, pgoff_t offset) put_page(page); goto repeat; } + page = find_subpage(page, offset); VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); } return page; @@ -1670,6 +1670,7 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, page = NULL; if (!page) goto no_page; + page = find_subpage(page, index); if (fgp_flags & FGP_LOCK) { if (fgp_flags & FGP_NOWAIT) { From patchwork Wed Aug 19 18:48:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724887 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3FF8F618 for ; Wed, 19 Aug 2020 18:49:11 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E519207DE for ; Wed, 19 Aug 2020 18:49:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="s/Fj5e9N" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E519207DE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EDF096E5B4; Wed, 19 Aug 2020 18:49:08 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id 089946E5B0 for ; Wed, 19 Aug 2020 18:49:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=p9w3DQmYzQVsLNeG7R7MdPybBJO6jXoNftILGTf+IE0=; b=s/Fj5e9N4i8MPBUyLdHzmYltJH W95qh4IqdO+bgeV/dKBXO34yVAYFsjEf92o4CweV2SCp098vES4xu2VCaPtffdljiQgVFusO0lPtR mVxJiPVCa8G/0SULJ8xl53dhiTptsPOCo1sLkBvQH4SedICuiVc6SD74X35sX59AOfjyMClCYGUzr 978Ku/CwTM5ej8GuG7J5Hwsz0uakDJFFzqaSPsaVyedNaFZ8G9IsmSt4Xk4MOwfTnNBgqvB1gV7u9 lDbXNqBWefAsnr4EO2lUpq8B6IotkPmE/O6f+kHf5745L3YB4RJNSt3UO0+ggGjOLpewsg3FA6VVz e6DXDilg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8p-0006UC-TJ; Wed, 19 Aug 2020 18:48:56 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:49 +0100 Message-Id: <20200819184850.24779-8-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 7/8] mm: Return head page from find_lock_entry X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" Convert the one caller of find_lock_entry() to cope with a head page being returned instead of the subpage for the index. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/pagemap.h | 12 ++++++++++++ mm/filemap.c | 25 +++++++++++-------------- mm/shmem.c | 15 ++++++++------- 3 files changed, 31 insertions(+), 21 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 8261c64f592d..de8f3758ceaa 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -371,6 +371,18 @@ static inline struct page *grab_cache_page_nowait(struct address_space *mapping, mapping_gfp_mask(mapping)); } +/* + * Is this index for one of the subpages of this page? + * This should always be called for a locked, non-hugetlbfs page. + */ +static inline bool thp_valid_index(struct page *head, pgoff_t index) +{ + VM_BUG_ON_PAGE(PageHuge(head), head); + VM_BUG_ON_PAGE(!PageLocked(head), head); + + return head->index == (index & ~(thp_nr_pages(head) - 1)); +} + /* * Given the page we found in the page cache, return the page corresponding * to this index in the file diff --git a/mm/filemap.c b/mm/filemap.c index 5a87e1bdddd6..6594baae7cd2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1594,37 +1594,34 @@ struct page *find_get_entry(struct address_space *mapping, pgoff_t index) } /** - * find_lock_entry - locate, pin and lock a page cache entry - * @mapping: the address_space to search - * @offset: the page cache index + * find_lock_entry - Locate and lock a page cache entry. + * @mapping: The address_space to search. + * @index: The page cache index. * - * Looks up the page cache slot at @mapping & @offset. If there is a - * page cache page, it is returned locked and with an increased - * refcount. + * Looks up the page at @mapping & @index. If there is a page in the + * cache, the head page is returned locked and with an increased refcount. * * If the slot holds a shadow entry of a previously evicted page, or a * swap entry from shmem/tmpfs, it is returned. * - * find_lock_entry() may sleep. - * - * Return: the found page or shadow entry, %NULL if nothing is found. + * Context: May sleep. + * Return: The head page or shadow entry, %NULL if nothing is found. */ -struct page *find_lock_entry(struct address_space *mapping, pgoff_t offset) +struct page *find_lock_entry(struct address_space *mapping, pgoff_t index) { struct page *page; repeat: - page = find_get_entry(mapping, offset); + page = find_get_entry(mapping, index); if (page && !xa_is_value(page)) { lock_page(page); /* Has the page been truncated? */ - if (unlikely(page_mapping(page) != mapping)) { + if (unlikely(page->mapping != mapping)) { unlock_page(page); put_page(page); goto repeat; } - page = find_subpage(page, offset); - VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); + VM_BUG_ON_PAGE(!thp_valid_index(page, index), page); } return page; } diff --git a/mm/shmem.c b/mm/shmem.c index 271548ca20f3..bbb43e9d35df 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1834,8 +1834,9 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, page = NULL; } if (page || sgp == SGP_READ) { - *pagep = page; - return 0; + if (page && PageTransHuge(page)) + hindex = round_down(index, HPAGE_PMD_NR); + goto out; } /* @@ -1961,14 +1962,13 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, * it now, lest undo on failure cancel our earlier guarantee. */ if (sgp != SGP_WRITE && !PageUptodate(page)) { - struct page *head = compound_head(page); int i; - for (i = 0; i < compound_nr(head); i++) { - clear_highpage(head + i); - flush_dcache_page(head + i); + for (i = 0; i < compound_nr(page); i++) { + clear_highpage(page + i); + flush_dcache_page(page + i); } - SetPageUptodate(head); + SetPageUptodate(page); } /* Perhaps the file has been truncated since we checked */ @@ -1984,6 +1984,7 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, error = -EINVAL; goto unlock; } +out: *pagep = page + index - hindex; return 0; From patchwork Wed Aug 19 18:48:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 11724885 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F48814F6 for ; Wed, 19 Aug 2020 18:49:10 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E2086207DE for ; Wed, 19 Aug 2020 18:49:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="QBZEY6GN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E2086207DE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DCD8F6E5B2; Wed, 19 Aug 2020 18:49:08 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by gabe.freedesktop.org (Postfix) with ESMTPS id EBD416E5AB for ; Wed, 19 Aug 2020 18:49:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=7+YSI5YaFA7nV1yaE7i/RW3xZbJtF/8x3LDr4io57Q8=; b=QBZEY6GNdSOmnF4ujehB7tIL/+ 8WlFkWpSxuLPFzkGOnnhlBlJavyO4Z+T0yXOfXgJgAGvoDk+Cf1m8ak9wPM8AkhSIMWn8nFA51xkD WHXh1JSEJrwxa10n1dsaReCF4jaEG4ACDpvQROgOqopHoed3BHKh9cV153/2lHtDBdHzNGvzALDwt xhCJjd4YgCOtHb28bEt8UYDUq80/QJIuQXJRj9csQ+VHnOkdnIJ+SSvdZLDIr692dGd7QtGw5TKT0 kT9IZJattIKOxBcoS1wr/w22x9PoXKBXAMBPZyBB/EKkYK4S922ZQv2l+P8miuoSBxQIRrrF5OJXs 10W3+YmQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1k8T8q-0006UR-DL; Wed, 19 Aug 2020 18:48:56 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Date: Wed, 19 Aug 2020 19:48:50 +0100 Message-Id: <20200819184850.24779-9-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20200819184850.24779-1-willy@infradead.org> References: <20200819184850.24779-1-willy@infradead.org> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH 8/8] mm: Hoist find_subpage call further up in pagecache_get_page X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-gfx@lists.freedesktop.org, Huang Ying , Hugh Dickins , linux-kernel@vger.kernel.org, Chris Wilson , William Kucharski , "Matthew Wilcox \(Oracle\)" , Johannes Weiner , cgroups@vger.kernel.org, Andrew Morton , Alexey Dobriyan , Matthew Auld Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" This avoids a call to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 6594baae7cd2..8c354277108d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1667,7 +1667,6 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, page = NULL; if (!page) goto no_page; - page = find_subpage(page, index); if (fgp_flags & FGP_LOCK) { if (fgp_flags & FGP_NOWAIT) { @@ -1680,12 +1679,12 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, } /* Has the page been truncated? */ - if (unlikely(compound_head(page)->mapping != mapping)) { + if (unlikely(page->mapping != mapping)) { unlock_page(page); put_page(page); goto repeat; } - VM_BUG_ON_PAGE(page->index != index, page); + VM_BUG_ON_PAGE(!thp_valid_index(page, index), page); } if (fgp_flags & FGP_ACCESSED) @@ -1695,6 +1694,7 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, if (page_is_idle(page)) clear_page_idle(page); } + page = find_subpage(page, index); no_page: if (!page && (fgp_flags & FGP_CREAT)) {