From patchwork Sun Mar 21 18:45:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28Intel=29?= X-Patchwork-Id: 12153243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D1C1C433C1 for ; Sun, 21 Mar 2021 18:46:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 14EC36194F for ; Sun, 21 Mar 2021 18:46:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14EC36194F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shipmail.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1E2DB6B0070; Sun, 21 Mar 2021 14:46:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 118976B0075; Sun, 21 Mar 2021 14:46:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D68FE6B0074; Sun, 21 Mar 2021 14:46:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0213.hostedemail.com [216.40.44.213]) by kanga.kvack.org (Postfix) with ESMTP id AF2216B0071 for ; Sun, 21 Mar 2021 14:46:15 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 69E031E01 for ; Sun, 21 Mar 2021 18:46:15 +0000 (UTC) X-FDA: 77944761510.03.2766213 Received: from ste-pvt-msa1.bahnhof.se (ste-pvt-msa1.bahnhof.se [213.80.101.70]) by imf23.hostedemail.com (Postfix) with ESMTP id 9D1E6A0009D5 for ; Sun, 21 Mar 2021 18:46:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTP id 60C4042376; Sun, 21 Mar 2021 19:46:11 +0100 (CET) Authentication-Results: ste-pvt-msa1.bahnhof.se; dkim=pass (1024-bit key; unprotected) header.d=shipmail.org header.i=@shipmail.org header.b=PQ83JxXd; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at bahnhof.se Received: from ste-pvt-msa1.bahnhof.se ([127.0.0.1]) by localhost (ste-pvt-msa1.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DYYkbKdnSTuh; Sun, 21 Mar 2021 19:46:09 +0100 (CET) Received: by ste-pvt-msa1.bahnhof.se (Postfix) with ESMTPA id 11C0141474; Sun, 21 Mar 2021 19:46:09 +0100 (CET) Received: from localhost.localdomain (2.70.38.73.mobile.tre.se [2.70.38.73]) by mail1.shipmail.org (Postfix) with ESMTPSA id 33797361E5A; Sun, 21 Mar 2021 19:46:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=shipmail.org; s=mail; t=1616352368; bh=eIp5oHPDAcmaQRleCtZI+6MjuyOrnVz5ZEJhMg48fNo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PQ83JxXd/yVYagz6sx67TBDGljF2mEwjfTIVFv928dIHgfA4GwaNO+MXLPeRFMWdo hdLKCdJ9AwSB8a8T8cfCBNouOq8LIVNboC9LIv6jdRK4YaCKCXNl13mJBfxiF1SLUA 5fIvG6eW8tTml8wQJbPkv3RRBJ6QpCwV6RCQwnk8= From: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28Intel=29?= To: dri-devel@lists.freedesktop.org Cc: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28Intel=29?= , Christian Koenig , David Airlie , Daniel Vetter , Andrew Morton , Jason Gunthorpe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 1/2] mm,drm/ttm: Block fast GUP to TTM huge pages Date: Sun, 21 Mar 2021 19:45:28 +0100 Message-Id: <20210321184529.59006-2-thomas_os@shipmail.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210321184529.59006-1-thomas_os@shipmail.org> References: <20210321184529.59006-1-thomas_os@shipmail.org> MIME-Version: 1.0 X-Stat-Signature: osobmaukn65r94enwz4rm988rgkapna9 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9D1E6A0009D5 Received-SPF: none (shipmail.org>: No applicable sender policy available) receiver=imf23; identity=mailfrom; envelope-from=""; helo=ste-pvt-msa1.bahnhof.se; client-ip=213.80.101.70 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616352372-911438 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: TTM sets up huge page-table-entries both to system- and device memory, and we don't want gup to assume there are always valid backing struct pages for these. For PTEs this is handled by setting the pte_special bit, but for the huge PUDs and PMDs, we have neither pmd_special nor pud_special. Normally, huge TTM entries are identified by looking at vma_is_special_huge(), but fast gup can't do that, so as an alternative define _devmap entries for which there are no backing dev_pagemap as special, update documentation and make huge TTM entries _devmap, after verifying that there is no backing dev_pagemap. One other alternative would be to block TTM huge page-table-entries completely, and while currently only vmwgfx use them, they would be beneficial to other graphis drivers moving forward as well. Cc: Christian Koenig Cc: David Airlie Cc: Daniel Vetter Cc: Andrew Morton Cc: Jason Gunthorpe Cc: linux-mm@kvack.org Cc: dri-devel@lists.freedesktop.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Thomas Hellström (Intel) --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 17 ++++++++++++++++- mm/gup.c | 21 +++++++++++---------- mm/memremap.c | 5 +++++ 3 files changed, 32 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 6dc96cf66744..1c34983480e5 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -195,6 +195,7 @@ static vm_fault_t ttm_bo_vm_insert_huge(struct vm_fault *vmf, pfn_t pfnt; struct ttm_tt *ttm = bo->ttm; bool write = vmf->flags & FAULT_FLAG_WRITE; + struct dev_pagemap *pagemap; /* Fault should not cross bo boundary. */ page_offset &= ~(fault_page_size - 1); @@ -210,6 +211,20 @@ static vm_fault_t ttm_bo_vm_insert_huge(struct vm_fault *vmf, if ((pfn & (fault_page_size - 1)) != 0) goto out_fallback; + /* + * Huge entries must be special, that is marking them as devmap + * with no backing device map range. If there is a backing + * range, Don't insert a huge entry. + * If this check turns out to be too much of a performance hit, + * we can instead have drivers indicate whether they may have + * backing device map ranges and if not, skip this lookup. + */ + pagemap = get_dev_pagemap(pfn, NULL); + if (pagemap) { + put_dev_pagemap(pagemap); + goto out_fallback; + } + /* Check that memory is contiguous. */ if (!bo->mem.bus.is_iomem) { for (i = 1; i < fault_page_size; ++i) { @@ -223,7 +238,7 @@ static vm_fault_t ttm_bo_vm_insert_huge(struct vm_fault *vmf, } } - pfnt = __pfn_to_pfn_t(pfn, PFN_DEV); + pfnt = __pfn_to_pfn_t(pfn, PFN_DEV | PFN_MAP); if (fault_page_size == (HPAGE_PMD_SIZE >> PAGE_SHIFT)) ret = vmf_insert_pfn_pmd_prot(vmf, pfnt, pgprot, write); #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD diff --git a/mm/gup.c b/mm/gup.c index e40579624f10..1b6a127f0bdd 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1993,6 +1993,17 @@ static void __maybe_unused undo_dev_pagemap(int *nr, int nr_start, } #ifdef CONFIG_ARCH_HAS_PTE_SPECIAL +/* + * If we can't determine whether or not a pte is special, then fail immediately + * for ptes. Note, we can still pin HugeTLB as it is guaranteed not to be + * special. For THP, special huge entries are indicated by xxx_devmap() + * returning true, but a corresponding call to get_dev_pagemap() will + * return NULL. + * + * For a futex to be placed on a THP tail page, get_futex_key requires a + * get_user_pages_fast_only implementation that can pin pages. Thus it's still + * useful to have gup_huge_pmd even if we can't operate on ptes. + */ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, unsigned int flags, struct page **pages, int *nr) { @@ -2069,16 +2080,6 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, return ret; } #else - -/* - * If we can't determine whether or not a pte is special, then fail immediately - * for ptes. Note, we can still pin HugeTLB and THP as these are guaranteed not - * to be special. - * - * For a futex to be placed on a THP tail page, get_futex_key requires a - * get_user_pages_fast_only implementation that can pin pages. Thus it's still - * useful to have gup_huge_pmd even if we can't operate on ptes. - */ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, unsigned int flags, struct page **pages, int *nr) { diff --git a/mm/memremap.c b/mm/memremap.c index 7aa7d6e80ee5..757551cd2a4d 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -471,6 +471,11 @@ void vmem_altmap_free(struct vmem_altmap *altmap, unsigned long nr_pfns) * * If @pgmap is non-NULL and covers @pfn it will be returned as-is. If @pgmap * is non-NULL but does not cover @pfn the reference to it will be released. + * + * Return: A referenced pointer to a struct dev_pagemap containing @pfn, + * or NULL if there was no such pagemap registered. For interpretion + * of NULL returns for pfns extracted from valid huge page table entries, + * please see gup_pte_range(). */ struct dev_pagemap *get_dev_pagemap(unsigned long pfn, struct dev_pagemap *pgmap)