From patchwork Fri Sep 2 00:35:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 12963467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE709ECAAD3 for ; Fri, 2 Sep 2022 00:36:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 539F78D0007; Thu, 1 Sep 2022 20:36:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C1ED8D0001; Thu, 1 Sep 2022 20:36:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3141B8D0007; Thu, 1 Sep 2022 20:36:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1B6CD8D0001 for ; Thu, 1 Sep 2022 20:36:28 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E18771403AB for ; Fri, 2 Sep 2022 00:36:27 +0000 (UTC) X-FDA: 79865279214.22.BABE01D Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2059.outbound.protection.outlook.com [40.107.92.59]) by imf23.hostedemail.com (Postfix) with ESMTP id 591E014005F for ; Fri, 2 Sep 2022 00:36:27 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BwGIx7uOkT4bj9sFaQJRp8+TJF50AHCi5/3kMKOiUFiRY+H2jpRwQyOdM361qYCHZAyoeHaorIEA8h/8gtGNVpMh1ed88IHzx4OYkEppJfZm7MbMVYGag7qy9p+HztCDdnTyCQy7eX3iQDu1AkXH4l/TzhjlStCpPmYh31EJ6vU2JS0YNSWF1FSoLzBwpQ/kHP4pVG2OGQ09WMeMd3gp6LuN0c/sQ3pM1VTM+3KOjBBvEmgyHAcjMtRsavKlzAsRJGCioJm/FRG2OUSO7eNVrBFI9Q7kU3HNZwCYYU2E9kJJX3BacWL0xfVUVAofWr+T57lML7SkQWLuoE6DbGwnfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eiZcJOw/Rdqbcxw/aFtmjNXtEWbSyoeadqVjC97LY1g=; b=QtnrpoEQj7E6rRzZnGoJ8l2rLmLa2kzGcWLqIwPJn9fvx9cvMzjZ/KF0L8/dyR5ce/RRlSjb+KqpblVhkgQtO/wQjqbNdFhwNRzvVvneYyhIA4mELxO+FWO9h95/X+XgBpvggBEt8eeMIXagmtvXl1jDtJlapf7hS03WJmoydDViYJN6BwmvetjC4CkMQM4qFMfzBTB5a/RSjVBaN4iiWMfL6d9X9BkCh/JtapJmPWvxk7F8VfzCM1Av5nseg+SWXsJ68BpPXZZ78LbbAMJtPVwjK+//Qb0V+YwDbGVFS3avVtK9t8PFl727EXncfN6cqAwILeZujM3He6lGbNKKxQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eiZcJOw/Rdqbcxw/aFtmjNXtEWbSyoeadqVjC97LY1g=; b=sgHnj4CYpJYMwiN15g41RE6v2YRVDjN1KRDQmqulsUFYkn1gHH1wwei9EjevxeJxQRNKg8lGVuexn7OZuBnZKoIqbjg9Vhk/N5NSK85IUZ5friPMgzIxf5PSK4l1QCfscOhsvP3+T3EILKNET/TnQ7L/ltLZf4EDGuSq3knatm040o2mo5hHKA9MVS1UZRh7XG3BTZqFXuCXSTsymsgHDh3UKtOwn5iHK8mAT5DP9XLkJEQL7YGCCjFWW8WKG1tY0RoRRyjW2nnVbDQjBoWsSCI/2lLEVIGNeR9YDF7yy6xoOFqodgWVb46MBo8N1Z2Nfb1iJnU1YMqsOLpfif9SpQ== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by DM4PR12MB5086.namprd12.prod.outlook.com (2603:10b6:5:389::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Fri, 2 Sep 2022 00:36:25 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::7432:2749:aa27:722c]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::7432:2749:aa27:722c%7]) with mapi id 15.20.5566.021; Fri, 2 Sep 2022 00:36:25 +0000 From: Alistair Popple To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: Peter Xu , Nadav Amit , Alistair Popple , huang ying , LKML , "Sierra Guiza, Alejandro (Alex)" , Felix Kuehling , Jason Gunthorpe , John Hubbard , David Hildenbrand , Ralph Campbell , Matthew Wilcox , Karol Herbst , Lyude Paul , Ben Skeggs , Logan Gunthorpe , paulus@ozlabs.org, linuxppc-dev@lists.ozlabs.org, "Huang, Ying" , stable@vger.kernel.org Subject: [PATCH v4 1/4] mm/migrate_device.c: Flush TLB while holding PTL Date: Fri, 2 Sep 2022 10:35:51 +1000 Message-Id: <9f801e9d8d830408f2ca27821f606e09aa856899.1662078528.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.35.1 X-ClientProxiedBy: BYAPR07CA0085.namprd07.prod.outlook.com (2603:10b6:a03:12b::26) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ac77a4c3-da93-448c-d75e-08da8c7b2aa6 X-MS-TrafficTypeDiagnostic: DM4PR12MB5086:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5By7JDjBV4FpvC0E2wNtdupNsf7r0E3W3wZqcO0NzjDSM98U210fvCJOK0aOZK/IRLsojvLDPE3s+Dd+fneNQbigsasxpJKUW+wlafT4P3yvbPOM+zTdBvw15cezTRvt9d0It4ijrCR8DR68oGjlcRsSlss1Tjy9Y0exWk7H/ECrJxrRqzHBUSjzRRgNkOiMJ20M5qf8O1AIABgfTrJxAZlPqvbeSGXU/96wyyh450esWAqo+pVzm3kQEB6skkRU5yM40oB0SOZPDaMXwhF2oHpLvou5SeZ1xbJ7liugzniVIMcRzZmLn4YKOZRL7EPedY6YLAfojneRJ+zVvlnco3BrL/hJ9TiZpkjrno+Wwdl9fU6KAKuAC13TkE+yrFoXQ/OUW18MTpVrvTOQb8C0gjVUjJ9ifYnRmCchvdfGupmwsY2l0JGnuUonGntMZhY/CVWpV/mYoG4UEkX21Q6EoD713Jiftchh1ftkq6/BiPhVm1+wNWd9CIQzSF51ERzZ5I7KIh9SaNrH04SU9keQibiCiXujNdxd7DntU2hdzNf3PAKj8vpFo+IYgukwne4VYZHNhZlWg0sbkxKy8OeV11vWr3wmCSyMGeXeEz6/nozJ36JrC0/+Or6Ou6z9aFcZx0RXWISV6ETYTbojWRJsGcBxXAJaaVho5PNPYmrq2PJk6jSyfFcFinkuWH9VMd8vdEemkhSrB7b39iC6ckLWHA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(366004)(376002)(346002)(39860400002)(396003)(136003)(83380400001)(38100700002)(66556008)(66946007)(66476007)(8676002)(4326008)(54906003)(316002)(2906002)(7416002)(8936002)(5660300002)(6506007)(6512007)(26005)(186003)(2616005)(478600001)(41300700001)(6666004)(6486002)(86362001)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3G2x+tFQF0Bd6VTDJFUQT1LTnTEEbkHbuy6CcZEs1eJts9JPjNQqzJuWE9tJNqandOfZGDks/GjTuplL3s2hW4EG2UUizk3a4WlEAdSD6XAu29kMuq0LpA6UIkm+IEJkhGzCS/rDv1gSPzO3Wj/N9lHVLLs+MRKnkl7vgGgNIRtpcM7giSrGBS8ZpnwAsWMitAZGM8YugyLq6LkgDKzVxbMxV/HwMeOPDjOS4ayw61Ia1g/ihfTgU4aTcWRcnvzKp2eRkxqTbnJ9/EVFm64DTfpG8bOYofqhX5z3YgjiNnz2XnBwjFzFqtaEMcAz6WqPoTXaqV9pAvIzas7hQVbve3EYbcTdiRRsZ/LqNurgiBSmL4Iz50KvTEvGw+ex7yvOi756drYiGCY2rM06pbxZERL9gN8CrqU1WhzPBYODex4jQt3CkODbeM+GcLJxSUtIMJo/pimK3806kM53dXfGZMekTknIHi73YurtmvhsmR9gWli11ne+RkiGFDqIIlVto13vjmYOHW7SawNmJUVhXvsdvEFIrnIbpulaKmIhqphB4HTY/tTdLYVz4NuxCdEvArMh6E+JXl2k9+ysh+FxA+SkKfXmWj3eOs4oH9eXjL+53sA63OuGrvzPjnn2Cx60fit7DhjvM2j/lQPdm4a4TW+tV04BleB8PwkTvK6b4F2eY5eDn1mcaJcvi5re+aS++vraCtjnmXcIDVLcyC9LcjSabN7tjr4tiMIaMITdmoayoYJKFM4sOJY9/eayuVku+ZRZivfGGLMK3TQl7ZILxemCWNOpVv2Y1KR5Y19JKgKftAONIq8Yz5mm84aIAxg06UxmMK5042EoacEpAiDkj0/9roTGK9oPQTgLevGRrnfRgyNMrpuai8tGdM/740JsiuIapNi5ydxSJnUNB8dnAiWnZZQeqhc5bNSILdfFX3sYY/BCl9Adaw+m9mymhXhCwGcxA3ECuUbqcEO5vPhs9JWqe8U97YNa9fc9gog1Q1lz7Ldy2Ty3WDUOrHzxHNL+A/oE3dXuHvQE3/oI+pl9ShiMcvypr7E7A5puPAAhPSUN5cyCdStLtbgC/zroSH6e0wh/Kot5ZsXyw2xYCfCh00TklNLo6IFokxZZrEdtnWTaReFL0dgaN9SiMhxVb7TmAGi/3zv7T3Xeyjf6AZezHEvrJsRPAAwShbY3YFv4CzPfiWgB83K7LxUCuB4wZyrE0qKbnV+AxTjOzJntmccKSKPDVcAE6MKIJ2ysXC3uAQuM2cuRc67RnMxfdzOZi2CLzdDRDcqJnC7zFfg6YxrzPa5rg7KUnK+/D0FaezbNr6lZDd/iTo+ZZLGs/AOEAxEltIUrqL9Aw/qLilCnHtuOLE6vjsdtF1H1/sPx8ZABXJvfI/sJX1Ca0O8zY09+EIfh/3EHvR7uIoKtfuMkiGSlVC4DvBhDtx/Yda2joV8UgQJjKdE8PqzvOb4YrZ3X7jGKMCyRr8vje3VpebsakdHNSGEDzPpIvxmg/ATy89bpV7kgM1UNdL8y8EAkcaawDg2bN5not0GyQHBbqFkHUZ4MsF+d6hHr3kO6erSoqZOvnsFbyx2vHnIvrK1eWmplbZ1W X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ac77a4c3-da93-448c-d75e-08da8c7b2aa6 X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Sep 2022 00:36:25.2820 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 01V0ss6H0tvrrxartWe58iu8sKaOouzy1+HsWOLX3ATOVpRHxzv1QacREU11LQuJrg/2fFsEOSRavEoexZJtCg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5086 ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1662078987; a=rsa-sha256; cv=pass; b=aBLDBNsNNkrkZgKk07nQr+M+1QuEro+SWzgpuna+NfJK4Sdi/dSo1JOSqTO82AwbAi7Lch M7FmL/11dl9UJpXBmVQtA/T65DDdFsXS29guW+id3yaoZ+Vz+qvAMs5twj6c1xuGHyNZ2t 1CoLQt2iQa9b8gZ+UMwwFU2UhNasiRk= ARC-Authentication-Results: i=2; imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=sgHnj4CY; spf=pass (imf23.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.59 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662078987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=eiZcJOw/Rdqbcxw/aFtmjNXtEWbSyoeadqVjC97LY1g=; b=rzMwtH/bO9TjLalb/uRZyK6NksYDpGs7Zn999TXWT+u1h6ZuFBnFrDvoh9h6He01433GsI +kOyJfPM4140fSDgrMWekGlORX476+HtB/gfNWJNVpPzgFi8cdhznRmuTSn64bwiPz329I cs64ZXKuLC9qAnWfjiSrQVsZzWDaBFs= Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=sgHnj4CY; spf=pass (imf23.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.59 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com X-Stat-Signature: kkukf9aw3aq9sooqc93embk8kzquy3jy X-Rspamd-Queue-Id: 591E014005F X-Rspamd-Server: rspam04 X-Rspam-User: X-HE-Tag: 1662078987-246113 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When clearing a PTE the TLB should be flushed whilst still holding the PTL to avoid a potential race with madvise/munmap/etc. For example consider the following sequence: CPU0 CPU1 ---- ---- migrate_vma_collect_pmd() pte_unmap_unlock() madvise(MADV_DONTNEED) -> zap_pte_range() pte_offset_map_lock() [ PTE not present, TLB not flushed ] pte_unmap_unlock() [ page is still accessible via stale TLB ] flush_tlb_range() In this case the page may still be accessed via the stale TLB entry after madvise returns. Fix this by flushing the TLB while holding the PTL. Signed-off-by: Alistair Popple Reported-by: Nadav Amit Reviewed-by: "Huang, Ying" Fixes: 8c3328f1f36a ("mm/migrate: migrate_vma() unmap page from vma while collecting pages") Cc: stable@vger.kernel.org Acked-by: David Hildenbrand Acked-by: Peter Xu --- Changes for v4: - Added Review-by Changes for v3: - New for v3 --- mm/migrate_device.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) base-commit: ffcf9c5700e49c0aee42dcba9a12ba21338e8136 diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 27fb37d..6a5ef9f 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -254,13 +254,14 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, migrate->dst[migrate->npages] = 0; migrate->src[migrate->npages++] = mpfn; } - arch_leave_lazy_mmu_mode(); - pte_unmap_unlock(ptep - 1, ptl); /* Only flush the TLB if we actually modified any entries */ if (unmapped) flush_tlb_range(walk->vma, start, end); + arch_leave_lazy_mmu_mode(); + pte_unmap_unlock(ptep - 1, ptl); + return 0; }