From patchwork Fri Jun 9 01:36:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13273087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F7E8C7EE25 for ; Fri, 9 Jun 2023 01:36:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98E168E0002; Thu, 8 Jun 2023 21:36:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9175F8E0001; Thu, 8 Jun 2023 21:36:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76ACB8E0002; Thu, 8 Jun 2023 21:36:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3BC5E8E0001 for ; Thu, 8 Jun 2023 21:36:18 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 07EED1A039B for ; Fri, 9 Jun 2023 01:36:18 +0000 (UTC) X-FDA: 80881494036.12.38AA773 Received: from mail-yw1-f175.google.com (mail-yw1-f175.google.com [209.85.128.175]) by imf07.hostedemail.com (Postfix) with ESMTP id 431D24000F for ; Fri, 9 Jun 2023 01:36:16 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=a5ewmipk; spf=pass (imf07.hostedemail.com: domain of hughd@google.com designates 209.85.128.175 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686274576; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7TjV4RyPitx7LIAI3Pj7MSABK+uluBQcp84mqUzVC6k=; b=RJ9/F+PDxUmSFmWzJzUNLxnWdLb0ZzHqMAGCJWOu3j9e4+4f+xKIVVrZhvqfar0V0Jh3jQ u/BpMpQN4WbDXqyiOc8wzapeJbgyd5xlXKWAeS3pA5+68wMyXdX2cm1aFxEoyKkkufUxM1 cp2UV+6fck9zZwgdpomsQZuXf1+VuYc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686274576; a=rsa-sha256; cv=none; b=DotJIhhMBso+MFg+VIqK1icrrX/APjCV9wB8+nGtYYoH6CABuKAYZFG3zIQbeGy4lC7Q+b 9yiM68TmrQT6hO7ga88+hj8JXREnXQ5YX65Kz6cvX0CUZPYMzgCyJ0/hvi+vOhe5MvpMGT ZpsP9PcOSoLw+osd199LW8t9yrT5k28= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=a5ewmipk; spf=pass (imf07.hostedemail.com: domain of hughd@google.com designates 209.85.128.175 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f175.google.com with SMTP id 00721157ae682-565cfe4ece7so11390117b3.2 for ; Thu, 08 Jun 2023 18:36:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686274575; x=1688866575; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=7TjV4RyPitx7LIAI3Pj7MSABK+uluBQcp84mqUzVC6k=; b=a5ewmipktwitFlsPba8meBOEdOaTniMgfwOwR66LxzGfjmzDH4IFIjppt+zCAFlN0g 4qVFbiljADzlv99p9uKv0I6LLrtlgpWP5C7ZN5jYvFhWguH9wpOq9f3yGPT9NIjCbt8f ccDNMfVoLAT69ZaBFF/iOjiPqCtqRwYPT0P1Yb6UZGbQci3A4kQCZ/DTEhdlS+lQWdYY KzmqHWulcwNNrAp2wlegE+sh4YyxODsI8618mTIxeZ3NAKuS71wgysjyvDWVHPesaDpk PafA85IWdjWaVeXqTpvoM+ur2RY2SxdVNDgFZ9WN6yrvvp/2wYtNHHpsHPpA7aoAQsWI /+rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686274575; x=1688866575; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7TjV4RyPitx7LIAI3Pj7MSABK+uluBQcp84mqUzVC6k=; b=GumVFBAP5enzKWsn8DegwVBm4OQeq2IUNz50+ODhtlerLCaYCrduNbI45Y2EAIv54s yCuVhHKg5eQ65/U1HFKbShIJavObEWXaN58XKOmpy+FYjOOQr6C2bphCMDqRpq88vt2b 13fZ4+l6mzA8mBNumJYr+qBelrCUmm6pgW/ogc+9SgQzlwye54tsVEep4/w8lqmHJ7p0 fYNf/ghtUU3Btbl5cErB7VANytXIEJGVT4grJFtoTHRzToP9BzUNZDVrsO/TfDt0ijr+ J/1OYGfp5on9zKIuWARHI6vmtjj1RPgk1edx91YU8XE97lSYhymRUPeyBvVV3h6i8BdC y57w== X-Gm-Message-State: AC+VfDxEfZdeQtCxTWJwXGWqOOMDN/BMhkmaF3GSIDXTh0s0B68prrcE zWnzXE0ygceja2NypSVCsPtJVg== X-Google-Smtp-Source: ACHHUZ41tpEmUAG3j7GY9w97TvrJQ50NkLe4OKcO0eJ30STZxTamEDNBY6XFXEyrE/Y12cF7SVrDgA== X-Received: by 2002:a81:a0c9:0:b0:568:a870:314f with SMTP id x192-20020a81a0c9000000b00568a870314fmr1280229ywg.30.1686274575219; Thu, 08 Jun 2023 18:36:15 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id d191-20020a0ddbc8000000b00569fdf7f58bsm293760ywe.66.2023.06.08.18.36.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jun 2023 18:36:14 -0700 (PDT) Date: Thu, 8 Jun 2023 18:36:11 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Lorenzo Stoakes , Huang Ying , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 22/32] mm/swapoff: allow pte_offset_map[_lock]() to fail In-Reply-To: Message-ID: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 431D24000F X-Rspam-User: X-Stat-Signature: jsob9jamqkzepwrufi7p1xrqiskehow1 X-Rspamd-Server: rspam03 X-HE-Tag: 1686274576-652042 X-HE-Meta: U2FsdGVkX1+MYPd41IHoliuaQ/iKz7F+zYFzLdj/SFPCRB3tIiQSkhgcb7RA0QMf1r9znZIEsoNv2G7c+K5b+yVksla/H2vUWawFpX8NsvKI17WgbElTcaXecVEnZRGAVDuAM7u6+8LwqSCQ3rG9goJVFOvVlZXml1Ew+qYINTuYf0i6VHxGmfjWpjdiOD8JPlLtAtFS8tovDyiuNi5kUE1HlO2ynanfIF6z7+Cd0Q45d7kgtZEp4k1oBIYmSrXisCjPu02rZRi1udVMLblaV3yDpoXUTcQv/9M6m2Cb7FrKgm1c4QOQWm3V+67ZyqpVKDx4aLGd07RAKc1pU7bHlnXXL+8haH5pQ49+XbcFKoSgxpD4C2LHvPz50X1yTv2qvsyJhk/uUV4heQwUMkIbY04NYF6WmfEHmwh6bblIiYSMGL6c3yYh2BGJ/aDJMtf8NFV/kIbYPUcZTCOCEAxC38lnM9T5bSuI589/Trn+yLMCdsERTc9Oq60utdFhlOBOSXFHRER7cALpII9cIn3SbuVk8A6D+I+bxD3uYkrTCniTZp3jkhOuwigFfMdhOwVaWDGX50vXSrUphUQf+CZ9apnacg28v4dLl/tPq4ui8Ua0Any/vapynqft/e1ShzSRnfut0GIox8xZjWEg0AtjZgvU/qUlqgzkaAb1DasNxBqZqQODyIdGtkcxm+2Mk3qewucJAGEwGHJ36NXBDBL9wrjlv99/QIRCuHxUcMMu3swJT0ZmqfivN+NrStHJTtj8GE42sEpp031SlF5xJG/E7q2SyPwNqOqZsb9QTZI2jvDHuNQhMJSodjZ+RR/hBkd4vAk+sH9stkkUR0ICensyOlUxi3JnaqrMj/wpXvnAuKabgL1WTTW/xIb7l8p8zCV55vpUh7Q79E+Kdsorq22FGZFXsOb6lTvDvB1EmWbpwg8hvFN7koRb9gGnjbjKzkXlO6qxaYmArxdSqTZ+DiO 9UDuXD99 BguknwRFEiQSlTIGC5zNbf3wJYpz21omKa1tfhkAvk11ePtemFTX7DAb8S4Mh+0khVuBvgyOG5Bjh6SIPl4SIvjtL5VCjYeJcqafaD0slJ5fl/wc+O9Mc6jxPmeeSp6MWITdVM++zLBiiRC/zg5i6PQItrfV1xxBMvblgf1WRXFdOUOHih0ZWtaMaA4zZBTFkGo6JDqKr/m0S4J/0WMoDCou+ppPwF3MAdoGsHbTYol/mAcUcDnlrltmUuSzPVFjvYaR4/vM30G6lvxBuSOth9X9ICFnEa+6863oGlhS8iAtftPtsY3YxQYakGz+Yy57C+1jPzcpoqMhqNrGyWhO8x5QU2lpsL8ho5e2eGYLoYkSptP9G/bzzSZaWWPzfjHwpDhgZlVVUrJyx0vnFPs9tiXYA7s2Bkl+Rvjy5daJzlU8F0jNMvt+1RQkwmQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Adjust unuse_pte() and unuse_pte_range() to allow pte_offset_map_lock() and pte_offset_map() failure; remove pmd_none_or_trans_huge_or_clear_bad() from unuse_pmd_range() now that pte_offset_map() does all that itself. Signed-off-by: Hugh Dickins --- mm/swapfile.c | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 274bbf797480..12d204e6dae2 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1774,7 +1774,7 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd, hwposioned = true; pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl); - if (unlikely(!pte_same_as_swp(*pte, swp_entry_to_pte(entry)))) { + if (unlikely(!pte || !pte_same_as_swp(*pte, swp_entry_to_pte(entry)))) { ret = 0; goto out; } @@ -1827,7 +1827,8 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd, set_pte_at(vma->vm_mm, addr, pte, new_pte); swap_free(entry); out: - pte_unmap_unlock(pte, ptl); + if (pte) + pte_unmap_unlock(pte, ptl); if (page != swapcache) { unlock_page(page); put_page(page); @@ -1839,17 +1840,22 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, unsigned int type) { - swp_entry_t entry; - pte_t *pte; + pte_t *pte = NULL; struct swap_info_struct *si; - int ret = 0; si = swap_info[type]; - pte = pte_offset_map(pmd, addr); do { struct folio *folio; unsigned long offset; unsigned char swp_count; + swp_entry_t entry; + int ret; + + if (!pte++) { + pte = pte_offset_map(pmd, addr); + if (!pte) + break; + } if (!is_swap_pte(*pte)) continue; @@ -1860,6 +1866,8 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, offset = swp_offset(entry); pte_unmap(pte); + pte = NULL; + folio = swap_cache_get_folio(entry, vma, addr); if (!folio) { struct page *page; @@ -1878,8 +1886,7 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, if (!folio) { swp_count = READ_ONCE(si->swap_map[offset]); if (swp_count == 0 || swp_count == SWAP_MAP_BAD) - goto try_next; - + continue; return -ENOMEM; } @@ -1889,20 +1896,17 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd, if (ret < 0) { folio_unlock(folio); folio_put(folio); - goto out; + return ret; } folio_free_swap(folio); folio_unlock(folio); folio_put(folio); -try_next: - pte = pte_offset_map(pmd, addr); - } while (pte++, addr += PAGE_SIZE, addr != end); - pte_unmap(pte - 1); + } while (addr += PAGE_SIZE, addr != end); - ret = 0; -out: - return ret; + if (pte) + pte_unmap(pte); + return 0; } static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud, @@ -1917,8 +1921,6 @@ static inline int unuse_pmd_range(struct vm_area_struct *vma, pud_t *pud, do { cond_resched(); next = pmd_addr_end(addr, end); - if (pmd_none_or_trans_huge_or_clear_bad(pmd)) - continue; ret = unuse_pte_range(vma, pmd, addr, next, type); if (ret) return ret;