From patchwork Fri Oct 30 10:08:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Daniel Vetter X-Patchwork-Id: 11869133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F83AC63699 for ; Fri, 30 Oct 2020 10:11:23 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16BB7221FA for ; Fri, 30 Oct 2020 10:11:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="C/7mltnq"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="OI5SurF9"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=ffwll.ch header.i=@ffwll.ch header.b="Gcc++IpN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16BB7221FA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=au7Gq0zZHF0igdH6k3aGXf2d7NF5tWiIW1WceYJHAPY=; b=C/7mltnqgCaSkdtB9PlPebTi2 GFC3w54nN2ZrUJif7BtTz8x+TKnWhLD0Pnx1CN++dY1fgxwtkahg60YUaq38DaWWatBGrd6Mewal4 yEU9isUSXvOIRe3LHhkMVUka7vpldkF8dIPyjvxhovqNvf2Oc44oLATejP1WOrWGFmqHOOb2TlFUp F9f/zjcb0EkDUiQKe3VFeoRqFxcg/NsvKqrdbo+xlPHEMnPZttQ1IeppUPA7Y1nFKK7AOOpor0Hh/ KK2sY9+YnVBRJhBRltfA9iJ7Owa/WkbcdNmOwSNyJgX17XwYBM0MERBe3Ka/hJhT8SDNjP3ltsQWk CKeT5vtYQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kYRMi-000843-8M; Fri, 30 Oct 2020 10:10:36 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kYRL8-0007WI-2d for linux-arm-kernel@merlin.infradead.org; Fri, 30 Oct 2020 10:08:58 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description; bh=HiI1+lLgMnsDQrYrNhRZjnVGpkNPEsmr7OdQwQZ1pJM=; b=OI5SurF9MEG7QC7EttJBWWhGiv cX4HnZ2eYA3yPmd3uZ5Fhl+TxkD5xOiBdDjYvC99u7Eum3FqoI9jRBQ4zPo7BQDSfz+94K1HADkKt pubkroWkquospjaqPaNAmDGFAL0Z1wxwrfjz7Hn4XoB7cki6zpNoEh/QfpvD47jug9YE+Eybw0yWj GfzOSRj1NlewXhV0NZdUzA8avRbXkd2nAPi1aUoAxvKtjLRKIiOlLRm8j6Y5eWITvtpXZ67cypG1J 5brny+cNyPZdomBqibk4NV+IWa0Clh0TUr/CZue0HYlL3uBxyqhMp5qaGWxZ62B5jKK8wDiGSytoI xTocywaQ==; Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kYRL1-0007qI-7N for linux-arm-kernel@lists.infradead.org; Fri, 30 Oct 2020 10:08:56 +0000 Received: by mail-wr1-x443.google.com with SMTP id k10so4459557wrw.13 for ; Fri, 30 Oct 2020 03:08:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HiI1+lLgMnsDQrYrNhRZjnVGpkNPEsmr7OdQwQZ1pJM=; b=Gcc++IpNNygNqtJu4MOWlOxXDTLkz8kWRrWBTguDYU0QSDOK1lmStqYhCY80JP1x87 nnoIM/kxhaqSCxhTbtNStA1+u8N8i8vvYmmr+2qlMbzhOp1A6XcFYXNVmwMv8phCp320 uR7glXXilJAM58uJT7pnvEqFSZFFav+qZ48X4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HiI1+lLgMnsDQrYrNhRZjnVGpkNPEsmr7OdQwQZ1pJM=; b=dSPiqZBpyff+PLfbqPrzx23q3NrDi3W3u5w2o9FqNhL6IKCcUcmNQfDtxMN9WEW72R yAou6wECu7YfnY0CalmKVbFk+D3uxRLluSG8Erdl+dEi8QmwrufChNgQatYyJFqHYaAL cX2ws2XXfjv7Th+2LB92bbyasjY9g5DkKwSUjEqnIjFktDvg+oEcFVI/3oDjvDHfq9dl 5ntuhfVxXDhWUhUYt2gyYmTNoKkUqrfHC8THxZf0WMO73Y9eyH+IY4Qgi39l9ga3eCu7 jnKxGsRr13Bhmz7ExEmhHB3xeMVzWdYutbzetYnJgWShQRS8niMVYXc00XWNUzm4m8rp zung== X-Gm-Message-State: AOAM533PhFwJRucRKLSlGx9Axq3r1nycwqCdsB7xwVcwrBMbBE9EyaFP 7rUqaIrQlKext+IL5dJuUy9vtw== X-Google-Smtp-Source: ABdhPJz2EbQLLd9y23l7XD4h1NQjvIRRCqqxxXIiiwxNslHpfrfcptIiI2iHPCasY7SN1JKAQAi14g== X-Received: by 2002:a5d:4d8a:: with SMTP id b10mr2128836wru.5.1604052527823; Fri, 30 Oct 2020 03:08:47 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id v189sm4430947wmg.14.2020.10.30.03.08.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Oct 2020 03:08:46 -0700 (PDT) From: Daniel Vetter To: DRI Development , LKML Subject: [PATCH v5 07/15] mm: Close race in generic_access_phys Date: Fri, 30 Oct 2020 11:08:07 +0100 Message-Id: <20201030100815.2269-8-daniel.vetter@ffwll.ch> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201030100815.2269-1-daniel.vetter@ffwll.ch> References: <20201030100815.2269-1-daniel.vetter@ffwll.ch> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201030_100852_535132_BC4D90A9 X-CRM114-Status: GOOD ( 24.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-samsung-soc@vger.kernel.org, Jan Kara , Kees Cook , kvm@vger.kernel.org, Jason Gunthorpe , Dave Airlie , Daniel Vetter , Chris Wilson , linux-mm@kvack.org, =?utf-8?b?SsOp?= =?utf-8?b?csO0bWUgR2xpc3Nl?= , John Hubbard , Benjamin Herrensmidt , Daniel Vetter , Dan Williams , Andrew Morton , linux-arm-kernel@lists.infradead.org, linux-media@vger.kernel.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Way back it was a reasonable assumptions that iomem mappings never change the pfn range they point at. But this has changed: - gpu drivers dynamically manage their memory nowadays, invalidating ptes with unmap_mapping_range when buffers get moved - contiguous dma allocations have moved from dedicated carvetouts to cma regions. This means if we miss the unmap the pfn might contain pagecache or anon memory (well anything allocated with GFP_MOVEABLE) - even /dev/mem now invalidates mappings when the kernel requests that iomem region when CONFIG_IO_STRICT_DEVMEM is set, see 3234ac664a87 ("/dev/mem: Revoke mappings when a driver claims the region") Accessing pfns obtained from ptes without holding all the locks is therefore no longer a good idea. Fix this. Since ioremap might need to manipulate pagetables too we need to drop the pt lock and have a retry loop if we raced. While at it, also add kerneldoc and improve the comment for the vma_ops->access function. It's for accessing, not for moving the memory from iomem to system memory, as the old comment seemed to suggest. References: 28b2ee20c7cb ("access_process_vm device memory infrastructure") Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Dan Williams Cc: Kees Cook Cc: Benjamin Herrensmidt Cc: Dave Airlie Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-samsung-soc@vger.kernel.org Cc: linux-media@vger.kernel.org Cc: Chris Wilson Signed-off-by: Daniel Vetter --- v2: Fix inversion in the retry check (John). v4: While at it, use offset_in_page (Chris Wilson) --- include/linux/mm.h | 3 ++- mm/memory.c | 46 +++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 45 insertions(+), 4 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 179dbb78d08d..83d0be101a38 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -574,7 +574,8 @@ struct vm_operations_struct { vm_fault_t (*pfn_mkwrite)(struct vm_fault *vmf); /* called by access_process_vm when get_user_pages() fails, typically - * for use by special VMAs that can switch between memory and hardware + * for use by special VMAs. See also generic_access_phys() for a generic + * implementation useful for any iomem mapping. */ int (*access)(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); diff --git a/mm/memory.c b/mm/memory.c index c48f8df6e502..ac32039ce941 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4847,28 +4847,68 @@ int follow_phys(struct vm_area_struct *vma, return ret; } +/** + * generic_access_phys - generic implementation for iomem mmap access + * @vma: the vma to access + * @addr: userspace addres, not relative offset within @vma + * @buf: buffer to read/write + * @len: length of transfer + * @write: set to FOLL_WRITE when writing, otherwise reading + * + * This is a generic implementation for &vm_operations_struct.access for an + * iomem mapping. This callback is used by access_process_vm() when the @vma is + * not page based. + */ int generic_access_phys(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write) { resource_size_t phys_addr; unsigned long prot = 0; void __iomem *maddr; - int offset = addr & (PAGE_SIZE-1); + pte_t *ptep, pte; + spinlock_t *ptl; + int offset = offset_in_page(addr); + int ret = -EINVAL; + + if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) + return -EINVAL; + +retry: + if (follow_pte(vma->vm_mm, addr, &ptep, &ptl)) + return -EINVAL; + pte = *ptep; + pte_unmap_unlock(ptep, ptl); - if (follow_phys(vma, addr, write, &prot, &phys_addr)) + prot = pgprot_val(pte_pgprot(pte)); + phys_addr = (resource_size_t)pte_pfn(pte) << PAGE_SHIFT; + + if ((write & FOLL_WRITE) && !pte_write(pte)) return -EINVAL; maddr = ioremap_prot(phys_addr, PAGE_ALIGN(len + offset), prot); if (!maddr) return -ENOMEM; + if (follow_pte(vma->vm_mm, addr, &ptep, &ptl)) + goto out_unmap; + + if (!pte_same(pte, *ptep)) { + pte_unmap_unlock(ptep, ptl); + iounmap(maddr); + + goto retry; + } + if (write) memcpy_toio(maddr + offset, buf, len); else memcpy_fromio(buf, maddr + offset, len); + ret = len; + pte_unmap_unlock(ptep, ptl); +out_unmap: iounmap(maddr); - return len; + return ret; } EXPORT_SYMBOL_GPL(generic_access_phys); #endif