From patchwork Fri May 14 09:50:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 12257307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1D69C433ED for ; Fri, 14 May 2021 09:51:40 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9178861108 for ; Fri, 14 May 2021 09:51:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9178861108 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=aY7xuX445BK7AkvlZnlz8UJUmjct+Go5BpThvsZmNkM=; b=I72shZJOvcDAwtMpQZl5Y5Jk26 gMLOzYyr4fOKBc92QfuzPCDebGCNIGIRPsD3BVffHMjiLPkVM9NhxCreOrK3c1d1sn3OiMsFLopX5 ulkW5IJT9k+uomsFi/KhftKVB9tKewOw+0uiQh6yXg34qk8ezgdSPsuMpN2kssTpcF0RARF3Lva/C YsBGHSNPoq4YIMJ3gTG8EFkhUWHkvLegWCgd/q3XdDG2FT8FMXNUY1CZIkWq3pMJXqG/1yfPvV9Rz 7RfU+daOcToT/0z2Nu0oFXtNHw5OGUQMYeePuebv5lDAHMiZCIEhaWopcEQNH7a7/yulodD5rm1AK iAMN5xww==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lhUSR-007gdj-Ow; Fri, 14 May 2021 09:50:11 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lhUSO-007gdQ-VE for linux-arm-kernel@desiato.infradead.org; Fri, 14 May 2021 09:50:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=Pl6FW4vG3/wAJ8aQqkmsdlgj5BFZ6jlyyLo3J/1W1A0=; b=rE97FQZWrczrseSIEgYSsC2YKI V2HKDjbrwn3QVcSTbU6RFwgzMvIZjkw8fUt3cGoc/bv48laNwBYSIQktY8QHGT184fMmUhHCIA/4m TXllrNGiN81/JiaLnxBYvyZRDb1GXprpL4USup6ib1bpXuR+sSxRsTKDudfKB2PrGqOxOf9yTbeTX jUp761VVlqSqh3Pl2q/E0MdHeeHtakc6lvkyigFUMUffTzQtQq1V6hJibfZkG2cab+bbwIT3QveX9 BISid6uJx3MWPHHZZAO3CyPWS4N4Xg3Io4cUOmmP176O0U5HIZAwZLHUV6KSG2Gg5dXMIdmaaneo4 vXR0twdw==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lhUSL-00BsIe-SG for linux-arm-kernel@lists.infradead.org; Fri, 14 May 2021 09:50:07 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id A429D613B6; Fri, 14 May 2021 09:50:03 +0000 (UTC) From: Catalin Marinas To: linux-arm-kernel@lists.infradead.org Cc: Mark Rutland , stable@vger.kernel.org, Will Deacon , Steven Price Subject: [PATCH] arm64: Fix race condition on PG_dcache_clean in __sync_icache_dcache() Date: Fri, 14 May 2021 10:50:01 +0100 Message-Id: <20210514095001.13236-1-catalin.marinas@arm.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210514_025005_958382_2B8557F8 X-CRM114-Status: GOOD ( 18.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To ensure that instructions are observable in a new mapping, the arm64 set_pte_at() implementation cleans the D-cache and invalidates the I-cache to the PoU. As an optimisation, this is only done on executable mappings and the PG_dcache_clean page flag is set to avoid future cache maintenance on the same page. When two different processes map the same page (e.g. private executable file or shared mapping) there's a potential race on checking and setting PG_dcache_clean via set_pte_at() -> __sync_icache_dcache(). While on the fault paths the page is locked (PG_locked), mprotect() does not take the page lock. The result is that one process may see the PG_dcache_clean flag set but the I/D cache maintenance not yet performed. Avoid test_and_set_bit(PG_dcache_clean) in favour of separate test_bit() and set_bit(). In the rare event of a race, the cache maintenance is done twice. Signed-off-by: Catalin Marinas Cc: Cc: Will Deacon Cc: Steven Price Reviewed-by: Steven Price Acked-by: Will Deacon --- Found while debating with Steven a similar race on PG_mte_tagged. For the latter we'll have to take a lock but hopefully in practice it will only happen when restoring from swap. Separate thread anyway. There's at least arch/arm with a similar race. Powerpc seems to do it properly with separate test/set. Other architectures have a bigger problem as they do a similar check in update_mmu_cache(), called after the pte was already exposed to user. I looked at fixing this in the mprotect() code but taking the page lock will slow it down, so not sure how popular this would be for such a rare race. arch/arm64/mm/flush.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index ac485163a4a7..6d44c028d1c9 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -55,8 +55,10 @@ void __sync_icache_dcache(pte_t pte) { struct page *page = pte_page(pte); - if (!test_and_set_bit(PG_dcache_clean, &page->flags)) + if (!test_bit(PG_dcache_clean, &page->flags)) { sync_icache_aliases(page_address(page), page_size(page)); + set_bit(PG_dcache_clean, &page->flags); + } } EXPORT_SYMBOL_GPL(__sync_icache_dcache);