From patchwork Thu Apr 4 16:36:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13618058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4174CD1292 for ; Thu, 4 Apr 2024 16:37:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 351E76B009C; Thu, 4 Apr 2024 12:37:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3019C6B009D; Thu, 4 Apr 2024 12:37:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A40A6B00A0; Thu, 4 Apr 2024 12:37:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EA6486B009C for ; Thu, 4 Apr 2024 12:37:24 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A93941402B0 for ; Thu, 4 Apr 2024 16:37:24 +0000 (UTC) X-FDA: 81972404808.12.A86A356 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 194B22000E for ; Thu, 4 Apr 2024 16:37:22 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CnmRsrOd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712248643; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YqMhB+a0hiEG+4rkQzSAJvqocE9H8po4mJ+mzkWYDBs=; b=QNkIHHOgfnwtvxJAiAdlkqehDwxd/kLMXpsGGFZGAcMnWowC+z25BiP4xsNE5yw4BMvEfi lZNJjM+4UKzx/MmWqTmcfhuEhKJktKSVK0tXAFXpOWMKZ+K4movy2nRxubYSvjDGfWR+5f pgqzk41Tjv6OhP6Kt0sJoYVuTzJJgjQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=CnmRsrOd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712248643; a=rsa-sha256; cv=none; b=ux+P6q9c3UtUjAsvutsSpEBpuQDdj5cyLkB9PozyXe5/y4dJT/RzeFb5F9FD8gGNKD3A+F fKld86SHxIhkcttsRLjbAhzIXJIooTsSwDZMYAZdCCEv/tKuOCJElPb1O3gwCVlPmdoLnI c/D5/NNekJa51oSrOMdZQSwkIn2VV7M= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1712248642; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YqMhB+a0hiEG+4rkQzSAJvqocE9H8po4mJ+mzkWYDBs=; b=CnmRsrOdUB8nXoGrApte72Yukj6PtTjyEDGptq90T2YG4X/FVkDykOFhYIQA5dpEimeJwx ed5+OTp1H/rHwTt5QCt2exioWh8y1T93qi7eTQm1j9M7Y/o1LCvyTnwivWGh4MqS7sg0V4 ZRnDRoUewnU0PZrQfHszqa6wudP7VeM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-41-eqwsHOp0NrySS1NYBlY5wA-1; Thu, 04 Apr 2024 12:37:18 -0400 X-MC-Unique: eqwsHOp0NrySS1NYBlY5wA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2581E28116C5; Thu, 4 Apr 2024 16:37:17 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.101]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8E09A5827; Thu, 4 Apr 2024 16:37:13 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, linux-s390@vger.kernel.org, kvm@vger.kernel.org, David Hildenbrand , Matthew Wilcox , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Janosch Frank , Claudio Imbrenda , Gerald Schaefer , Thomas Huth Subject: [PATCH v1 2/5] s390/uv: convert gmap_make_secure() to work on folios Date: Thu, 4 Apr 2024 18:36:39 +0200 Message-ID: <20240404163642.1125529-3-david@redhat.com> In-Reply-To: <20240404163642.1125529-1-david@redhat.com> References: <20240404163642.1125529-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspamd-Queue-Id: 194B22000E X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: gyaxms85c4rbiuxq7q9noqzwt67imed9 X-HE-Tag: 1712248642-285385 X-HE-Meta: U2FsdGVkX180lJ5WlGDB4l7ao+5371mNXkq1AePsUoiycK6bRftv4cY+6BhHgrKw5xnOmjXRPaVnRJvu+obVMO2/YAPmQmgjZK5bor4TENjBZbtZ3tI+qNtKiwBOjUZjptIigypqlC0Fta90iYb7M+9la5jiRrKm8F5nFCahrlkCMYN+lNfw1VvP4yW8lXOgAGNl83Bg9Snf/ceJZecN3e0XQwfCdE2kg6B/X77gxBJ6JA6p4NdmOOh5tlZxg0QVyI6UaRT7y1arPqdebNGHxBcpZt6WteWbX5x+MdnoApWD+DiQ8nqak5MX3+cKFC+urtsm6RiX6Q4cGt0/aYkpEDrmO07L/lH90Pyf8xqOypSUIvUxIsmc+1oCUt8W9YUY/sT6QnpTEipXFT2RqVhH0lQSEe0BVJbEh4atrneQQR9MYJUkgP6j2tfutKTW9VpsMx+1hosDqeKGCIUM1SrUFJML9Ot4KmOsoL3+qhazsmPzp0PAQQqAYbiJZlVWtlngqtlveu7pKR3Pso+esskNQ5LGBTFfG8KCD9hH7rBhZK/HzNqPYqVVGmO5J23imWfl6Tiejte9k+fHnKvEYzlppdApa59RQ9/01NTlfPj0HAKFZ7bZ+TawL/bpe7TUsPxN+jAnDAnBfAzazZlU6oNEGoB4hR4578+X6XtbAkXFGri7IslesRk5p9yalneZcHF68M0TI3zCCbzJI8h28oG1MJYaYISpNLiFlYWOFfYJXLr6RhTElNNca6kb+vN1ev9sl6g2gEdxuQRVLNtDsFxJCbxuumfJx5bYedzxZPwLMJem/IdzqiZZPoos/8oj1qEj6qmUwEx28/EO8q51y0Rj8TZQwAYm0fgN8pDbqKsPVEW/tn1hIY6t9bO5u54U3mA4JyLaUsyzidgIrXs2Lcggo+TSaotEAMOWOo76Q4qPmxBc0XWNuTrR4RD9m4nqK8AQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We have various goals that require gmap_make_secure() to only work on folios. We want to limit the use of page_mapcount() to the places where it is absolutely necessary, we want to avoid using page flags of tail pages, and we want to remove page_has_private(). So, let's convert gmap_make_secure() to folios. While s390x makes sure to never have PMD-mapped THP in processes that use KVM -- by remapping them using PTEs in thp_split_walk_pmd_entry()->split_huge_pmd() -- we might still find PTE-mapped THPs and could end up working on tail pages of such large folios for now. To handle that cleanly, let's simply split any PTE-mapped large folio, so we can be sure that we are always working with small folios and never on tail pages. There is no real change: splitting will similarly fail on unexpected folio references, just like it would already when we try to freeze the folio refcount. Signed-off-by: David Hildenbrand --- arch/s390/include/asm/page.h | 1 + arch/s390/kernel/uv.c | 66 ++++++++++++++++++++++-------------- 2 files changed, 42 insertions(+), 25 deletions(-) diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index 9381879f7ecf..54d015bcd8e3 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -215,6 +215,7 @@ static inline unsigned long __phys_addr(unsigned long x, bool is_31bit) #define phys_to_page(phys) pfn_to_page(phys_to_pfn(phys)) #define page_to_phys(page) pfn_to_phys(page_to_pfn(page)) +#define folio_to_phys(page) pfn_to_phys(folio_pfn(folio)) static inline void *pfn_to_virt(unsigned long pfn) { diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c index 7401838b960b..adcbd4b13035 100644 --- a/arch/s390/kernel/uv.c +++ b/arch/s390/kernel/uv.c @@ -181,36 +181,36 @@ int uv_convert_owned_from_secure(unsigned long paddr) } /* - * Calculate the expected ref_count for a page that would otherwise have no + * Calculate the expected ref_count for a folio that would otherwise have no * further pins. This was cribbed from similar functions in other places in * the kernel, but with some slight modifications. We know that a secure - * page can not be a huge page for example. + * folio can only be a small folio for example. */ -static int expected_page_refs(struct page *page) +static int expected_folio_refs(struct folio *folio) { int res; - res = page_mapcount(page); - if (PageSwapCache(page)) { + res = folio_mapcount(folio); + if (folio_test_swapcache(folio)) { res++; - } else if (page_mapping(page)) { + } else if (folio_mapping(folio)) { res++; - if (page_has_private(page)) + if (folio_has_private(folio)) res++; } return res; } -static int make_page_secure(struct page *page, struct uv_cb_header *uvcb) +static int make_folio_secure(struct folio *folio, struct uv_cb_header *uvcb) { int expected, cc = 0; - if (PageWriteback(page)) + if (folio_test_writeback(folio)) return -EAGAIN; - expected = expected_page_refs(page); - if (!page_ref_freeze(page, expected)) + expected = expected_folio_refs(folio); + if (!folio_ref_freeze(folio, expected)) return -EBUSY; - set_bit(PG_arch_1, &page->flags); + set_bit(PG_arch_1, &folio->flags); /* * If the UVC does not succeed or fail immediately, we don't want to * loop for long, or we might get stall notifications. @@ -220,9 +220,9 @@ static int make_page_secure(struct page *page, struct uv_cb_header *uvcb) * -EAGAIN and we let the callers deal with it. */ cc = __uv_call(0, (u64)uvcb); - page_ref_unfreeze(page, expected); + folio_ref_unfreeze(folio, expected); /* - * Return -ENXIO if the page was not mapped, -EINVAL for other errors. + * Return -ENXIO if the folio was not mapped, -EINVAL for other errors. * If busy or partially completed, return -EAGAIN. */ if (cc == UVC_CC_OK) @@ -277,7 +277,7 @@ int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb) bool local_drain = false; spinlock_t *ptelock; unsigned long uaddr; - struct page *page; + struct folio *folio; pte_t *ptep; int rc; @@ -306,33 +306,49 @@ int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb) if (!ptep) goto out; if (pte_present(*ptep) && !(pte_val(*ptep) & _PAGE_INVALID) && pte_write(*ptep)) { - page = pte_page(*ptep); + folio = page_folio(pte_page(*ptep)); rc = -EAGAIN; - if (trylock_page(page)) { + + /* We might get PTE-mapped large folios; split them first. */ + if (folio_test_large(folio)) { + rc = -E2BIG; + } else if (folio_trylock(folio)) { if (should_export_before_import(uvcb, gmap->mm)) - uv_convert_from_secure(page_to_phys(page)); - rc = make_page_secure(page, uvcb); - unlock_page(page); + uv_convert_from_secure(folio_to_phys(folio)); + rc = make_folio_secure(folio, uvcb); + folio_unlock(folio); } /* - * Once we drop the PTL, the page may get unmapped and + * Once we drop the PTL, the folio may get unmapped and * freed immediately. We need a temporary reference. */ - if (rc == -EAGAIN) - get_page(page); + if (rc == -EAGAIN || rc == -E2BIG) + folio_get(folio); } pte_unmap_unlock(ptep, ptelock); out: mmap_read_unlock(gmap->mm); + if (rc == -E2BIG) { + /* + * Splitting might fail with -EBUSY due to unexpected folio + * references, just like make_folio_secure(). So handle it + * ahead of time without the PTL being held. + */ + folio_lock(folio); + rc = split_folio(folio); + folio_unlock(folio); + folio_put(folio); + } + if (rc == -EAGAIN) { /* * If we are here because the UVC returned busy or partial * completion, this is just a useless check, but it is safe. */ - wait_on_page_writeback(page); - put_page(page); + folio_wait_writeback(folio); + folio_put(folio); } else if (rc == -EBUSY) { /* * If we have tried a local drain and the page refcount