From patchwork Fri Jun 9 01:18:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13273073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1FCBC7EE29 for ; Fri, 9 Jun 2023 01:18:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 662B28E0002; Thu, 8 Jun 2023 21:18:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E9DF8E0001; Thu, 8 Jun 2023 21:18:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4641A8E0002; Thu, 8 Jun 2023 21:18:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 36D7C8E0001 for ; Thu, 8 Jun 2023 21:18:58 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 00561A03A1 for ; Fri, 9 Jun 2023 01:18:56 +0000 (UTC) X-FDA: 80881450272.18.BA5857F Received: from mail-yb1-f177.google.com (mail-yb1-f177.google.com [209.85.219.177]) by imf06.hostedemail.com (Postfix) with ESMTP id 33E2F18000B for ; Fri, 9 Jun 2023 01:18:55 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=ngXLVdTp; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of hughd@google.com designates 209.85.219.177 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686273535; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=s/wneZk+WXTrjMA3fZuua9PXiv3fUuHnEiWoD80tgK8=; b=6eXUz20BV4BC3to9kLOyiNzLOEsw5FrffPaUywGt9JJcMru86qxjkL2SIU0qbxTIlxv0jb kJmfanHFPkij/CLpBBS8K9VOdr20HLvVNQSAx7cpTsNeEL9KcP07admtToImkxgncFzgdx aHL925n9JR7fTQqbIVECFZUM1UjR9+k= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=ngXLVdTp; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of hughd@google.com designates 209.85.219.177 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686273535; a=rsa-sha256; cv=none; b=fpuiH7ttDwU9Wa+gCuJiFjZ3KjHPxM1BSzPURi99pmBLb5PHoD98HiiY6la3secOtwIQLp UN/LNmZiQMngR5mAtN8j17+BSs3kAsu72HGIl7bdxPOZZu3y4ugwn4acrgJAQ2jEEq1EXJ RnX0AFSCNaCbYp9eiO6wqCNpx0uuyl0= Received: by mail-yb1-f177.google.com with SMTP id 3f1490d57ef6-bad1ae90c2eso1274939276.2 for ; Thu, 08 Jun 2023 18:18:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686273534; x=1688865534; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=s/wneZk+WXTrjMA3fZuua9PXiv3fUuHnEiWoD80tgK8=; b=ngXLVdTpVyrnOmcx9sqd11fDN/DpMfDjRFc20sS67KuKXUBPfJVN7b54WivfwCCqcm 2w8p0+WuKxMdG7YLSaNER90dnOHjPSlvPspYoe5kIeMhFky06+Enxr+OKG3u1lmvAOqa C+PVbym1M2SnYvLxMqSZg33YxKhOPm3FgR9jzQ+MnLfZGFUC2OJbbjuScpkVkepg/TEp 276YbysaG6RpeWzjjsraxwjBqi8YR2LoYFLfcozNIrD2bqcAOk2V+q1v/295yDeTTnP9 TfyYNMuGFwQjegWuOVfz1GKj11hEcBHgzfIvjlO8LY2ikBAtYqRoDenRCee7HmvBFb1A njIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686273534; x=1688865534; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=s/wneZk+WXTrjMA3fZuua9PXiv3fUuHnEiWoD80tgK8=; b=D1Op+U+gFI9vbgurzG1VKtoJNI74Cs3Mjj5ZmB6fjMx2C1ZM4Pb3/De7wnEudKU+0r pIY+6fSnzcy7BtIzVsiCPzf9MLuh3mx3MNtQRT+i9pRNZup2fi3dERUz1NdLAXtfY3oP 6AHgu0W5/yo9tVSw6L7/27FLSadUaXgRoSmiArFQMAXwi1xf7Dv/YjEIdvcv30/NiSTh 1+GeXhdr7fwwoqX/7KVASUt95MTzg29uo4Xz75SlQuoHXEDh0Oz+zD5s3igyfBvYR2LE rVXzfMhzcMn1YeFXbx+95Q4bu+l+FA2JSL0DFiDljEBv3UBlaFkB7gpiHp2hH4I0xJSu pWeg== X-Gm-Message-State: AC+VfDxhFtmUtFBeGaassy3Q5SGo+LHm+pYaodjNyc8s5zrfnPKnblEU dWCsYRCA/DfsBlSajIeIUQIGAA== X-Google-Smtp-Source: ACHHUZ5bBUa+IZ+zQxwNmCb4pbZbizWCezt9RItrFgkqOHN09PZD1rbRUEyPvItVL/IZu/e+lOkb4A== X-Received: by 2002:a25:d757:0:b0:ba7:ff37:4603 with SMTP id o84-20020a25d757000000b00ba7ff374603mr1178588ybg.45.1686273534119; Thu, 08 Jun 2023 18:18:54 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id v38-20020a25aba9000000b00b923b2935d9sm603286ybi.20.2023.06.08.18.18.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jun 2023 18:18:53 -0700 (PDT) Date: Thu, 8 Jun 2023 18:18:49 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Lorenzo Stoakes , Huang Ying , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 10/32] mm/pagewalk: walk_pte_range() allow for pte_offset_map() In-Reply-To: Message-ID: <3eba6f0-2b-fb66-6bb6-2ee8533e221@google.com> References: MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 33E2F18000B X-Stat-Signature: 5jnmm9iduxkb6c51xpm6rfiq3zqehq5t X-Rspam-User: X-HE-Tag: 1686273535-494487 X-HE-Meta: U2FsdGVkX19p5lz95ktj7lNgX0ayDZ7Rip2Q1XJxBniyT6UAZCKTpMBcRlj7UfLAdIFpEwSc2iVClhdFeH0asZvGWTP5LcE8jdqExtqmycvL+uvGjaPiE4YCQ7xpNlKId0zWK7uZqxOrrCHxU+xidMD2AVdbTCuvYf7ewCCoKVoKMKp+bnQ5KSnGPW7nL04raj2ahxZtsuK+1sbbn7TL9mlr0XFRXaI8sN/dEOm1+6OMeSpVLUmeBO7N6/zY6fasctKc5ziopg65PeD8XBRKs9+N0tgWDtE0wqkdQQ35t1uW5lsNg/Ir85FlPEtm4r2aT8iZSLW7Ub7ElV7lYUQA0h10D9OpqPE27ZrWyLbk9OvVjd/64kxYHD9CpCEH6wupUkSNdjR8zUgMlwei5CxMOfLdLGogbKCvmGuf0nuyQ6u358p/V6TFJT+/o7hEEjuf5hpYRpF6ML+lesRE4xaPS/i/plSfb65tKEWfmAgR9xXsr+t7mJaVsrnG7vSfwHxyyv+5xooczBopkoI8W177c5Qxz9Z3O5Af9JWx8XI9EGwpNCQZ4x8YElSSzNfOEQFGJ3r6Pva2eJYK45qsCAFwTehm8RGq4qjJ0L92Yz88S5y4ElLMu4Vs0Hiw5gMBl92IJijK2LVeS2gBnn//TLxfbZcPLW9QDkiTvDnzEcNsgrOrvEdlWSehaLJkElWJ+v+vQ3jyzml59pbmMUSDXrzM90A8J3z3lPrdV5ROoLME8X1x+T43IoWJTLfSJQmPPODLU2mwshhIHjno5tK7RBsC+5hBfR7ce1nMm3MKTxwSE3nVWLEoxbOt9TS9IG+qedplLAqIAU0jTj4NUdPYFpmICJc6Hk9SdHBBV7uuw7cHWLiKGyGJc/qerKaOZvgoBt6pf4BD+BUgier9SpCfrzrYBCi9Vy7OaSEw6UgRsuce/lmH08HxwqeFKYl/5N5jOJNQFpgatldpQmo3dbLYTJK wkpIYw14 s6SsEX8ugdPvzRDMappQRqKjFTZYFQE3m4Zww/dFWYxXGZicbgBddV+3y2W6F8PCqFci33A3g+1hpVvzeHAf8RnNX7QPM/KsNGwDotRm/sMiIXJ4pEkBNen+66+/cAlZdkMscjy9ldVzufLeRTWIXllsIDbaNcRrWeMnyxrM0YrAkgR2LHerbPcWy9FTkn2L8o7pPjiTE0naQZWFvpmA3N/SlHgjkAyGOsd7YoEk25yJxmwawlNNJM7FbynvUS8Le60UKc6/14ItTMnbJh+dV4B37Wl9hM9iTW3CQ2++6qEdJsXrMcwMhlNAhOVaK4JFCVrVDN5Tdtndf4jHmTIC5ApGsO5292SGqFUwFfmsFmwrX+Pupe/qprVkLP18I+/DkkgHloPKq8XHxkCSdno0wxStHxtAOughCJmerApBPtv7keHlxMDHXK5jodQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: walk_pte_range() has a no_vma option to serve walk_page_range_novma(). I don't know of any problem, but it looks safer to check for init_mm, and use pte_offset_kernel() rather than pte_offset_map() in that case: pte_offset_map()'s pmdval validation is intended for userspace. Allow for its pte_offset_map() or pte_offset_map_lock() to fail, and retry with ACTION_AGAIN if so. Add a second check for ACTION_AGAIN in walk_pmd_range(), to catch it after return from walk_pte_range(). Remove the pmd_trans_unstable() check after split_huge_pmd() in walk_pmd_range(): walk_pte_range() now handles those cases safely (and they must fail powerpc's is_hugepd() check). Signed-off-by: Hugh Dickins --- mm/pagewalk.c | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index cb23f8a15c13..64437105fe0d 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -46,15 +46,27 @@ static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, spinlock_t *ptl; if (walk->no_vma) { - pte = pte_offset_map(pmd, addr); - err = walk_pte_range_inner(pte, addr, end, walk); - pte_unmap(pte); + /* + * pte_offset_map() might apply user-specific validation. + */ + if (walk->mm == &init_mm) + pte = pte_offset_kernel(pmd, addr); + else + pte = pte_offset_map(pmd, addr); + if (pte) { + err = walk_pte_range_inner(pte, addr, end, walk); + if (walk->mm != &init_mm) + pte_unmap(pte); + } } else { pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl); - err = walk_pte_range_inner(pte, addr, end, walk); - pte_unmap_unlock(pte, ptl); + if (pte) { + err = walk_pte_range_inner(pte, addr, end, walk); + pte_unmap_unlock(pte, ptl); + } } - + if (!pte) + walk->action = ACTION_AGAIN; return err; } @@ -141,11 +153,8 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, !(ops->pte_entry)) continue; - if (walk->vma) { + if (walk->vma) split_huge_pmd(walk->vma, pmd, addr); - if (pmd_trans_unstable(pmd)) - goto again; - } if (is_hugepd(__hugepd(pmd_val(*pmd)))) err = walk_hugepd_range((hugepd_t *)pmd, addr, next, walk, PMD_SHIFT); @@ -153,6 +162,10 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, err = walk_pte_range(pmd, addr, next, walk); if (err) break; + + if (walk->action == ACTION_AGAIN) + goto again; + } while (pmd++, addr = next, addr != end); return err;