From patchwork Fri Apr 29 13:35:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 12832015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28A6FC433EF for ; Fri, 29 Apr 2022 13:37:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7D876B0075; Fri, 29 Apr 2022 09:37:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B2D556B007E; Fri, 29 Apr 2022 09:37:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CE1A6B0080; Fri, 29 Apr 2022 09:37:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 90E966B0075 for ; Fri, 29 Apr 2022 09:37:16 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 72DE960401 for ; Fri, 29 Apr 2022 13:37:16 +0000 (UTC) X-FDA: 79410018072.03.2814407 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by imf17.hostedemail.com (Postfix) with ESMTP id B7AD64002B for ; Fri, 29 Apr 2022 13:37:06 +0000 (UTC) Received: by mail-pg1-f170.google.com with SMTP id i62so6551362pgd.6 for ; Fri, 29 Apr 2022 06:37:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ti47oOTvJFCNXT5uBN/QcWBt/mztyaSPv2YCzaUIEks=; b=3g9d4G3AVSSQh+94A2C6QzQqKt1PRF8js91ZL0nJcPA3Z2pqrsS736awrXrhB5qPNL sUEJ1N/4NDvBYvnmd6ICC5/nmqG4YbNK2tVYSwte7tSPMcq+itTorKQlC5NV96S78vve 49KHlqBtvfS6ikyeLIE98STuW16ozIxTWc4DS49i0ZPiYyllxwnGRYZcxC6V30CKju9o CL9fjfO7XWQ7Yvc9MmOIn0mD/1tlgEEmvZfM2zZieTtyLlokUilgkaABmZykB1TcatHq pjHEGHSkjNa+P+/dvoKyd/UP7gMxeI4LAtJltzdrP0xKNx+P2UQhK6u7RqFWZSdZOLE4 X8LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ti47oOTvJFCNXT5uBN/QcWBt/mztyaSPv2YCzaUIEks=; b=eWVqRNSYp0bXeEuAmPuCWSFrJ514ibKYMQ4deYi7b0QcXXcxF/UNYmEv1w1VYsl7hM Urs8YD8GSwqJmWQxRJE9bDRg4KeiraCJEhAGhdEB51ORL7kEpHLZ0ilv7iYafiX9MHHN T39u+f0/0hu+jf9+y2qb2zNN5UDm8g1ibOtFqxh+Z49FolwQK1MTegXR1qYr4CSGSnOQ 7+GTuWeBa9Uqt982iTsx5mZlf6j9ZOc1ZcphFX340G5tWoB6UYWj5/1a+ZZpRszcxxFa 1U0fM9MpQymRN1XLDQGincoXUq5FecB2drD6pLudAAX/3Zi6C8fTT4vLkr/TU3E9sN/S iawA== X-Gm-Message-State: AOAM533U/eo9e3ERIdTpv6i1LUbKhoP0jf9n8ACKAWubqIX2Ar1zCJoQ 7VEtH+//zxxCblhgRPOYuPBX/g== X-Google-Smtp-Source: ABdhPJxMIKA/P0RDHv9STrZjiOvbOSFfoH4N5N3pT1umtFqilIbIuxsCtayDjWtKg053GReSSR2iXA== X-Received: by 2002:a65:4006:0:b0:3aa:1cb6:e2f8 with SMTP id f6-20020a654006000000b003aa1cb6e2f8mr32435549pgp.274.1651239435027; Fri, 29 Apr 2022 06:37:15 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.240]) by smtp.gmail.com with ESMTPSA id m8-20020a17090a414800b001d81a30c437sm10681977pjg.50.2022.04.29.06.37.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Apr 2022 06:37:14 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, tglx@linutronix.de, kirill.shutemov@linux.intel.com, mika.penttila@nextfour.com, david@redhat.com, jgg@nvidia.com, tj@kernel.org, dennis@kernel.org, ming.lei@redhat.com Cc: linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, songmuchun@bytedance.com, zhouchengming@bytedance.com, Qi Zheng Subject: [RFC PATCH 10/18] mm: add pte_tryget_map{_lock}() helper Date: Fri, 29 Apr 2022 21:35:44 +0800 Message-Id: <20220429133552.33768-11-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20220429133552.33768-1-zhengqi.arch@bytedance.com> References: <20220429133552.33768-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: B7AD64002B Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=3g9d4G3A; spf=pass (imf17.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.170 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspam-User: X-Stat-Signature: y9fubddgzk5au4sghrcxpmcqi71pgq37 X-HE-Tag: 1651239426-131811 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now, we usually use pte_offset_map{_lock}() to get the pte_t pointer before accessing the PTE page table page. After adding the FREE_USER_PTE, we also need to call the pte_tryget() before calling pte_offset_map{_lock}(), which is used to try to get the reference count of the PTE to prevent the PTE page table page from being freed during the access process. This patch adds pte_tryget_map{_lock}() to help us to do that. A return value of NULL indicates that we failed to get the percpu_ref, and there is a concurrent thread that is releasing this PTE (or has already been released). It needs to be treated as the case of pte_none(). Signed-off-by: Qi Zheng --- include/linux/pgtable.h | 37 +++++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index d1218cb1013e..6f205fee6348 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -228,6 +228,8 @@ static inline spinlock_t *pud_lock(struct mm_struct *mm, pud_t *pud) return ptl; } +#include + #ifndef pte_offset_kernel static inline pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address) { @@ -240,12 +242,38 @@ static inline pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address) #define pte_offset_map(dir, address) \ ((pte_t *)kmap_atomic(pmd_page(*(dir))) + \ pte_index((address))) -#define pte_unmap(pte) kunmap_atomic((pte)) +#define __pte_unmap(pte) kunmap_atomic((pte)) #else #define pte_offset_map(dir, address) pte_offset_kernel((dir), (address)) -#define pte_unmap(pte) ((void)(pte)) /* NOP */ +#define __pte_unmap(pte) ((void)(pte)) /* NOP */ #endif +#define pte_tryget_map(mm, pmd, address) \ +({ \ + pte_t *__pte = NULL; \ + if (pte_tryget(mm, pmd, address)) \ + __pte = pte_offset_map(pmd, address); \ + __pte; \ +}) + +#define pte_unmap(pte) do { \ + pte_put(pte); \ + __pte_unmap(pte); \ +} while (0) + +#define pte_tryget_map_lock(mm, pmd, address, ptlp) \ +({ \ + spinlock_t *__ptl = NULL; \ + pte_t *__pte = NULL; \ + if (pte_tryget(mm, pmd, address)) { \ + __ptl = pte_lockptr(mm, pmd); \ + __pte = pte_offset_map(pmd, address); \ + *(ptlp) = __ptl; \ + spin_lock(__ptl); \ + } \ + __pte; \ +}) + #define pte_offset_map_lock(mm, pmd, address, ptlp) \ ({ \ spinlock_t *__ptl = pte_lockptr(mm, pmd); \ @@ -260,6 +288,11 @@ static inline pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address) pte_unmap(pte); \ } while (0) +#define __pte_unmap_unlock(pte, ptl) do { \ + spin_unlock(ptl); \ + __pte_unmap(pte); \ +} while (0) + /* Find an entry in the second-level page table.. */ #ifndef pmd_offset static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)