From patchwork Mon Mar 25 13:41:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13602231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E7D3C54E58 for ; Mon, 25 Mar 2024 13:41:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B96576B0095; Mon, 25 Mar 2024 09:41:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B1EC76B0096; Mon, 25 Mar 2024 09:41:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A3966B0098; Mon, 25 Mar 2024 09:41:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 86A5F6B0095 for ; Mon, 25 Mar 2024 09:41:40 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1552E406EB for ; Mon, 25 Mar 2024 13:41:40 +0000 (UTC) X-FDA: 81935673960.23.A9F0586 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf18.hostedemail.com (Postfix) with ESMTP id 386441C0003 for ; Mon, 25 Mar 2024 13:41:38 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IJEf6P3d; spf=pass (imf18.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711374098; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CokJpVlNHQkDeOM8Ax0bVh3dvCSI52pFoSUWrkagN08=; b=uvsIwaX0hliYOoOnt84dbq9UJtaf5Dej3B+7X8mKxKQRa41B/c3KLhSh3rUsg6Ei8RIXf4 2nVJepOD0YmiVZDFA2qDuXenobAxvXrnrFzJZJpizLVsW6Gm0eRfS+D7VR51j64QYLvAmQ 6C7WeTvvfxdB52TabroD7LySrGhitUM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711374098; a=rsa-sha256; cv=none; b=rUjHPYtYPz7AkHh+hmmDZcksYfBKJo0VOwMT6tZb0UvyM7058XWFhMjcDDpUbizTsCp5Sm oy6eK9A8IL4+Xd9LvdwaScx/4lKaQUiNKMu524T5InezZgpzD/YLOoQM0Wqt7B2Zo7r6yR 0ZovCFgKiXoRVZtbJ7gt468bUnpqhbw= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IJEf6P3d; spf=pass (imf18.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711374097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CokJpVlNHQkDeOM8Ax0bVh3dvCSI52pFoSUWrkagN08=; b=IJEf6P3dq/q+8xLpM+JAKdIqazpas6s+waaZhbuEq9tjR6NO3b+gJtf+8vqg3kIVe3vHNx hmR/kQbm/XGISq5hmxMtdRUZqcsgOxU59p5mF7hY4Ubg59T6hBI9fiW2pEo2omGw7uzj+z wzBylNfSsCxw8mvzb6xfEjgzZTr0sME= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-373-tEBRYUiQPDClGkgaC9S8ug-1; Mon, 25 Mar 2024 09:41:33 -0400 X-MC-Unique: tEBRYUiQPDClGkgaC9S8ug-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4A2D5891E69; Mon, 25 Mar 2024 13:41:33 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.193.143]) by smtp.corp.redhat.com (Postfix) with ESMTP id B5DA62166B35; Mon, 25 Mar 2024 13:41:31 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Rapoport , Miklos Szeredi , Lorenzo Stoakes , xingwei lee , yue sun Subject: [PATCH v1 3/3] mm: merge folio_is_secretmem() into folio_fast_pin_allowed() Date: Mon, 25 Mar 2024 14:41:14 +0100 Message-ID: <20240325134114.257544-4-david@redhat.com> In-Reply-To: <20240325134114.257544-1-david@redhat.com> References: <20240325134114.257544-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 X-Stat-Signature: pxhwwpfnn4egkzhdg9jgd3w799u5mh9o X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 386441C0003 X-Rspam-User: X-HE-Tag: 1711374098-161761 X-HE-Meta: U2FsdGVkX19gAmUaw14UsFuBtvZ5eO9tat1JU6p8C/ET69vmHUNe5Zgsz5H+hD+Rgfwiqo87qgDkGHvzt4WmqIskIupK+clJvdbVnQZS885tjwHRuERslYQAZ8sGXNqpbUemi8q5J5G+KGEM6/BYaminzZammOzAVgMarVuwY8C6G6w0wukDezohL9IFQYtHuTeifKg/ftFLhoiSgUD7OKH98Xudu2oV1lpQLC/bltziCnMXv6IYmE+zLmYcihpY8ZcPlnSeN2jeqcM7ffS/qvHrZDcYlmcy+wXYuIatCdYCdkYCgzRA2tTNfdQfHe57/3sw+pX4xupc4gpf/k4nBaYDm03bcrTIFcRn2joZwgYpcCKwOnkuUAKTwHCUTP5ARb3Kv+8ZmMFXLSH1DKcl7CJup/rU+Kqjf1CjIh24T/u13AKIgrqHl1c647qcpIt9sMOaCtRAh6mk2jA23Zo8aX9LFklfKFUSXemjUE4PD52cRzAUb6L2vVMvZ2E+C48mXNbVQljpF81hk2Cu6S/ZizAFPern1Y1l2V2hzfamfJIkiuoqyrRBlBMSx3+N0EHNiGM0NjXciJ64ljKhIT4Rg4pVORX8bQbTyktHtst0M0I9fskp0P3vNcfRoUGrtfnt9ARiBdKSETYSbbPeBkM1N1/bOL7CV6Yki4FEismOXKW75YkPGQpAqwtAbnMKGFqgzKkWKdMzWpcgQ6KUdN8WECvQg4kI8TgCHJWpS6O3rVU8oMWlk1KuxxTueu49+LZb8nzlfvVDyQlIvzIlkWBBgBwrxIPnynDTuzBQAuLsjzmzO/fWUrJOIrBosVm3FcFhKiWNiggqG9M5HCU13IIwu502vGtK5MDf7BkfZR03+I3v1JEd7ICsFjqABFW+S4MFWvDcMMJOVB+5xJ/Cn5q2tYu51r7D95/6QzEUEgSh8vaAxq3mP33Jr5CwnC7pkr53jFhu3F6wsckSrydaIFo ZMHzUefS poS7j17KNEQR++mHzhn6IaywOFOIdbWXvuWPtdOzCx42tgtilkN0QoZbBobi75oPZkwsgADoURpsb0Hi9hfN8+O3AA2DjvJSAoa9gI0/1g6hLEDBUAE7YWg6+xkwic/q+zzPk9LJoH/gwAeUUqqf85wPh5A+IJpsE/ze+o49jit63k50P1y2sOFlW22QDIsHSLa6iUgPaYvmPxEyp2ZZiObaCGQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: folio_is_secretmem() is currently only used during GUP-fast, and using it in wrong context where concurrent truncation might happen, could be problematic. Nowadays, folio_fast_pin_allowed() performs similar checks during GUP-fast and contains a lot of careful handling -- READ_ONCE( -- ), sanity checks -- lockdep_assert_irqs_disabled() -- and helpful comments on how this handling is safe and correct. So let's merge folio_is_secretmem() into folio_fast_pin_allowed(), still avoiding checking the actual mapping only if really required. Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) --- include/linux/secretmem.h | 21 ++------------------- mm/gup.c | 33 +++++++++++++++++++++------------ 2 files changed, 23 insertions(+), 31 deletions(-) diff --git a/include/linux/secretmem.h b/include/linux/secretmem.h index 6996f1f53f14..e918f96881f5 100644 --- a/include/linux/secretmem.h +++ b/include/linux/secretmem.h @@ -6,25 +6,8 @@ extern const struct address_space_operations secretmem_aops; -static inline bool folio_is_secretmem(struct folio *folio) +static inline bool secretmem_mapping(struct address_space *mapping) { - struct address_space *mapping; - - /* - * Using folio_mapping() is quite slow because of the actual call - * instruction. - * We know that secretmem pages are not compound and LRU so we can - * save a couple of cycles here. - */ - if (folio_test_large(folio) || folio_test_lru(folio)) - return false; - - mapping = (struct address_space *) - ((unsigned long)folio->mapping & ~PAGE_MAPPING_FLAGS); - - if (!mapping || mapping != folio->mapping) - return false; - return mapping->a_ops == &secretmem_aops; } @@ -38,7 +21,7 @@ static inline bool vma_is_secretmem(struct vm_area_struct *vma) return false; } -static inline bool folio_is_secretmem(struct folio *folio) +static inline bool secretmem_mapping(struct address_space *mapping) { return false; } diff --git a/mm/gup.c b/mm/gup.c index e7510b6ce765..69d8bc8e4451 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2472,6 +2472,8 @@ EXPORT_SYMBOL(get_user_pages_unlocked); * This call assumes the caller has pinned the folio, that the lowest page table * level still points to this folio, and that interrupts have been disabled. * + * GUP-fast must reject all secretmem folios. + * * Writing to pinned file-backed dirty tracked folios is inherently problematic * (see comment describing the writable_file_mapping_allowed() function). We * therefore try to avoid the most egregious case of a long-term mapping doing @@ -2484,22 +2486,32 @@ EXPORT_SYMBOL(get_user_pages_unlocked); static bool folio_fast_pin_allowed(struct folio *folio, unsigned int flags) { struct address_space *mapping; + bool check_secretmem = false; + bool reject_file_backed = false; unsigned long mapping_flags; /* * If we aren't pinning then no problematic write can occur. A long term * pin is the most egregious case so this is the one we disallow. */ - if ((flags & (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) != + if ((flags & (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) == (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) - return true; + reject_file_backed = true; + + /* We hold a folio reference, so we can safely access folio fields. */ - /* The folio is pinned, so we can safely access folio fields. */ + /* secretmem folios are only order-0 folios and never LRU folios. */ + if (IS_ENABLED(CONFIG_SECRETMEM) && !folio_test_large(folio) && + !folio_test_lru(folio)) + check_secretmem = true; + + if (!reject_file_backed && !check_secretmem) + return true; if (WARN_ON_ONCE(folio_test_slab(folio))) return false; - /* hugetlb mappings do not require dirty-tracking. */ + /* hugetlb neither requires dirty-tracking nor can be secretmem. */ if (folio_test_hugetlb(folio)) return true; @@ -2535,10 +2547,12 @@ static bool folio_fast_pin_allowed(struct folio *folio, unsigned int flags) /* * At this point, we know the mapping is non-null and points to an - * address_space object. The only remaining whitelisted file system is - * shmem. + * address_space object. */ - return shmem_mapping(mapping); + if (check_secretmem && secretmem_mapping(mapping)) + return false; + /* The only remaining allowed file system is shmem. */ + return !reject_file_backed || shmem_mapping(mapping); } static void __maybe_unused undo_dev_pagemap(int *nr, int nr_start, @@ -2624,11 +2638,6 @@ static int gup_pte_range(pmd_t pmd, pmd_t *pmdp, unsigned long addr, if (!folio) goto pte_unmap; - if (unlikely(folio_is_secretmem(folio))) { - gup_put_folio(folio, 1, flags); - goto pte_unmap; - } - if (unlikely(pmd_val(pmd) != pmd_val(*pmdp)) || unlikely(pte_val(pte) != pte_val(ptep_get(ptep)))) { gup_put_folio(folio, 1, flags);