From patchwork Mon Apr 17 20:50:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13214795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 11F61C77B76 for ; Mon, 17 Apr 2023 21:53:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ydDxXL085sa8J8AKEuW5Qu3SP+6YnVL8o+lVD+Lu76E=; b=4ef9DA5vKEpGX6 CELDDqEaojVd6vNnhwjSP8X4wq1zb5IB0lfOUYwEQQgOOdEvK3SuzQjFjSkEz8Erl0VVBG7gTr5rf gjyK6Rbpe6QEaO5bGt8Jbt+l5mBa+6dchVY3WDG9pBzXGH4isHpYDfM6Mlr4lbAHFcaQumtuuf/Dp 0tdF90QnxYAKBYa8DH6H21EP9E5KpHuKYrojfeyEtoyiap75r8N6jSfdF3bNFIRZgThzFSwaoCS3m dZEwZ1yV92b8PcY2wPIcwYCZ4xTGB2Gr7q9CWwNtgdrYdfdUHm/hq+aIxBUqqU/mUEDMCSFC+9E7z SCtiL7TGiY3brVAdZf3A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1poWma-0009Qu-1E; Mon, 17 Apr 2023 21:53:08 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1poWmU-0009O3-1K; Mon, 17 Apr 2023 21:53:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bmbsS8DkaHNij/v9bdafzBlktB7f+f7NAeK2Q02GvbI=; b=iuONInTB9P3ifamdjSqB8ueUZp O6cHCkgEo2dF8hR3tlTe+jCZpiu38607UoIdL8XxLEM+6qKdXGHy31VgAA7/diMFXUMASuqQPnoZI LoOjv+9cb5+YDxZlordODIEQNuV1NVSC/OEE42PdU0h14DjQ4tDacfxJqvUiGDrr+IobkO/sdvS5A 8apfk5txOgrumQVTDZwHp+yLQq9J+pJmgWfJ0ePUYlvTy0lzYxdDrMjKw/acUkU8QeGAm97890YfQ D0B9roxC6fUOwaScvqeOjqRYqbvaKI9Ffie8t4f7W/2Xzu+xeQvBd7Cbo4BlSoW8RsIvpv0biT5nS uv8hTLgw==; Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by desiato.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1poVqN-00HS3h-09; Mon, 17 Apr 2023 20:53:12 +0000 Received: by mail-pj1-x102c.google.com with SMTP id s6-20020a17090a698600b00247a7794e78so3556258pjj.0; Mon, 17 Apr 2023 13:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681764768; x=1684356768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bmbsS8DkaHNij/v9bdafzBlktB7f+f7NAeK2Q02GvbI=; b=BOUYQbRtprz0j+QE5FspkyGNnQXo7tua/LH56ZXhTbzE3CAOV8yTgXemF8QTCY9h4g g6V6zIZ7VFlE3ubnJkZ/yYqYyanZgx2V65Z6ltF6wcw0210IAZlR8To7Sk4y8TLWY4Hz xNxTo6TOWAdqeGCeg1Pit+CsJNU/MS0gvSxN85F+x2sLQC0IMYUPA6am8ZQrYVGpPzga 8IeLZRTxr8/8Yg1lDFCUFpg2PfjK4sGTGgffRI7OQ6kx1dS8UMGUuVs3EuNQkywAoLYX E+HKq98y8quQjgUskaGG1Ic02lqe4ZtSQ+Zea8JL+rlDVF597bKHVbSZetgCXEY/xB7J hYgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681764768; x=1684356768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bmbsS8DkaHNij/v9bdafzBlktB7f+f7NAeK2Q02GvbI=; b=DWyLklPLg9/u5KTmn8J93Jg0Q63cpwh+SWmvjZn8bMcK3tpob3ytOPf4qkjRqzIInc U7kC50wLxd+UrZl1yj2GF0T1d0XQ+BFLBLOerHBbk21fi3a05VT7mpAamppjxXvjIptP QKeJyhkRdu9CAqsWRWns8ZOpSm3aRQuXyUb9vgjwTuL29IwQagZc11hwKCqg+bXsDA8N ytXdhJY2tuO5Sh6kMvN8fNCG94WelhlzV/0olrKAy4LFmHLDt5rpQNyjkj3+jS0qMy1h vzspJlkOeJvgR5PqzTVC/V6VIEqqzijgfuDNXAYb6TvYMeIMB7bK+J3birgR2xjPxbOO 862Q== X-Gm-Message-State: AAQBX9cAbvlfgX8MWKAx/7sgGZ67ioCF4qTRkPmik1xNO2iMwenZMuB2 4v/CBhuh+5H9UXVxyQxZ3/w= X-Google-Smtp-Source: AKy350ZtXMgj2Ot8GUfxTmnnYjtYP4oOp+wIKL46L+o6DOMpet2qdkXsjbB50RH+WV9/LkMm+8nkDA== X-Received: by 2002:a17:90b:3907:b0:247:8f24:eb31 with SMTP id ob7-20020a17090b390700b002478f24eb31mr4943020pjb.48.1681764767923; Mon, 17 Apr 2023 13:52:47 -0700 (PDT) Received: from fedora.hsd1.ca.comcast.net ([2601:644:937f:7f20::c139]) by smtp.googlemail.com with ESMTPSA id h7-20020a17090ac38700b0022335f1dae2sm7609707pjt.22.2023.04.17.13.52.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Apr 2023 13:52:47 -0700 (PDT) From: "Vishal Moola (Oracle)" To: Andrew Morton , Matthew Wilcox Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, "Vishal Moola (Oracle)" Subject: [PATCH 02/33] s390: Use pt_frag_refcount for pagetables Date: Mon, 17 Apr 2023 13:50:17 -0700 Message-Id: <20230417205048.15870-3-vishal.moola@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230417205048.15870-1-vishal.moola@gmail.com> References: <20230417205048.15870-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230417_215301_845802_5506B998 X-CRM114-Status: GOOD ( 18.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org s390 currently uses _refcount to identify fragmented page tables. The page table struct already has a member pt_frag_refcount used by powerpc, so have s390 use that instead of the _refcount field as well. This improves the safety for _refcount and the page table tracking. This also allows us to simplify the tracking since we can once again use the lower byte of pt_frag_refcount instead of the upper byte of _refcount. Signed-off-by: Vishal Moola (Oracle) --- arch/s390/mm/pgalloc.c | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c index 66ab68db9842..6b99932abc66 100644 --- a/arch/s390/mm/pgalloc.c +++ b/arch/s390/mm/pgalloc.c @@ -182,20 +182,17 @@ void page_table_free_pgste(struct page *page) * As follows from the above, no unallocated or fully allocated parent * pages are contained in mm_context_t::pgtable_list. * - * The upper byte (bits 24-31) of the parent page _refcount is used + * The lower byte (bits 0-7) of the parent page pt_frag_refcount is used * for tracking contained 2KB-pgtables and has the following format: * * PP AA - * 01234567 upper byte (bits 24-31) of struct page::_refcount + * 01234567 upper byte (bits 0-7) of struct page::pt_frag_refcount * || || * || |+--- upper 2KB-pgtable is allocated * || +---- lower 2KB-pgtable is allocated * |+------- upper 2KB-pgtable is pending for removal * +-------- lower 2KB-pgtable is pending for removal * - * (See commit 620b4e903179 ("s390: use _refcount for pgtables") on why - * using _refcount is possible). - * * When 2KB-pgtable is allocated the corresponding AA bit is set to 1. * The parent page is either: * - added to mm_context_t::pgtable_list in case the second half of the @@ -243,11 +240,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm) if (!list_empty(&mm->context.pgtable_list)) { page = list_first_entry(&mm->context.pgtable_list, struct page, lru); - mask = atomic_read(&page->_refcount) >> 24; + mask = atomic_read(&page->pt_frag_refcount); /* * The pending removal bits must also be checked. * Failure to do so might lead to an impossible - * value of (i.e 0x13 or 0x23) written to _refcount. + * value of (i.e 0x13 or 0x23) written to + * pt_frag_refcount. * Such values violate the assumption that pending and * allocation bits are mutually exclusive, and the rest * of the code unrails as result. That could lead to @@ -259,8 +257,8 @@ unsigned long *page_table_alloc(struct mm_struct *mm) bit = mask & 1; /* =1 -> second 2K */ if (bit) table += PTRS_PER_PTE; - atomic_xor_bits(&page->_refcount, - 0x01U << (bit + 24)); + atomic_xor_bits(&page->pt_frag_refcount, + 0x01U << bit); list_del(&page->lru); } } @@ -281,12 +279,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm) table = (unsigned long *) page_to_virt(page); if (mm_alloc_pgste(mm)) { /* Return 4K page table with PGSTEs */ - atomic_xor_bits(&page->_refcount, 0x03U << 24); + atomic_xor_bits(&page->pt_frag_refcount, 0x03U); memset64((u64 *)table, _PAGE_INVALID, PTRS_PER_PTE); memset64((u64 *)table + PTRS_PER_PTE, 0, PTRS_PER_PTE); } else { /* Return the first 2K fragment of the page */ - atomic_xor_bits(&page->_refcount, 0x01U << 24); + atomic_xor_bits(&page->pt_frag_refcount, 0x01U); memset64((u64 *)table, _PAGE_INVALID, 2 * PTRS_PER_PTE); spin_lock_bh(&mm->context.lock); list_add(&page->lru, &mm->context.pgtable_list); @@ -323,22 +321,19 @@ void page_table_free(struct mm_struct *mm, unsigned long *table) * will happen outside of the critical section from this * function or from __tlb_remove_table() */ - mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x11U << bit); if (mask & 0x03U) list_add(&page->lru, &mm->context.pgtable_list); else list_del(&page->lru); spin_unlock_bh(&mm->context.lock); - mask = atomic_xor_bits(&page->_refcount, 0x10U << (bit + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x10U << bit); if (mask != 0x00U) return; half = 0x01U << bit; } else { half = 0x03U; - mask = atomic_xor_bits(&page->_refcount, 0x03U << 24); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x03U); } page_table_release_check(page, table, half, mask); @@ -368,8 +363,7 @@ void page_table_free_rcu(struct mmu_gather *tlb, unsigned long *table, * outside of the critical section from __tlb_remove_table() or from * page_table_free() */ - mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x11U << bit); if (mask & 0x03U) list_add_tail(&page->lru, &mm->context.pgtable_list); else @@ -391,14 +385,12 @@ void __tlb_remove_table(void *_table) return; case 0x01U: /* lower 2K of a 4K page table */ case 0x02U: /* higher 2K of a 4K page table */ - mask = atomic_xor_bits(&page->_refcount, mask << (4 + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, mask << 4); if (mask != 0x00U) return; break; case 0x03U: /* 4K page table with pgstes */ - mask = atomic_xor_bits(&page->_refcount, 0x03U << 24); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x03U); break; }