From patchwork Mon Jul 10 20:43:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13307610 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5078EB64D9 for ; Mon, 10 Jul 2023 20:44:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33F588E000C; Mon, 10 Jul 2023 16:43:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 033A88E0011; Mon, 10 Jul 2023 16:43:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BA848E000B; Mon, 10 Jul 2023 16:43:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2F1388E0001 for ; Mon, 10 Jul 2023 16:43:48 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0BDA6A028B for ; Mon, 10 Jul 2023 20:43:48 +0000 (UTC) X-FDA: 80996878536.13.697DEF8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id 5AD98140014 for ; Mon, 10 Jul 2023 20:43:46 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tiHU73Gt; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689021826; a=rsa-sha256; cv=none; b=NimU1+vgcxnHEhJE9pDqO5yxqZGmC0MNX/KjtswJbmyH6OG1uLUOyy8nbLV+1WY2/jG69S x9dIuQik2QNlwW2ym18MyxLPeEA6CVnAeAqKdAZJMrnY64oEE6ZnAixHnYccjlhMdyRqGt ZXKbnytL3Jg606PtS7bX/cTM/5mZ4Qg= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=tiHU73Gt; dmarc=none; spf=none (imf23.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689021826; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9mxVcCkW583JrlzbOMVQaZnB0bjf4NUO5YrFMq87KVA=; b=FKpyXssm7zJGkHaQxMqCPtX26JIAdkr53AUnW+0j/nY1PmijG69D1QxGXN0WSv/gKrxFlp 7AWvjV3f2UxDbHJldBj3ol9n2QXkvc4H8Av4prKF/iij/bLM5yvifoOa1cgDA6C4ymmHZ3 s+XUehJidfVbrHG9vwqqAWled2kd/GE= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9mxVcCkW583JrlzbOMVQaZnB0bjf4NUO5YrFMq87KVA=; b=tiHU73GtBk0CdV4R8NECb7nrWG rlw/hDC8XD9b2zjAZwna48outBRoVxu2IWG4ZicdNlUuRYBDKZRUemkoPgiXlrkf0kVB/sx3qjfs0 KkxCLgeBb0dGeLCejf0eJpZyZUb0QAlMhBZF+E3fcYghEbOccZYeYRtjGSz0QvTXnX52XdAJw1TXs dD9qFxvpT4uZslcdHGUINiZkGgdG7bUfWf+eV3k8ld7aPrcsNPTW5Ce0qlVAzV58FK0Y85gBRJU9t +h+io5vedYA2iPQhdC9XJcsDOkWmkXsYFq638vvxWop58dOIBFBXxPiLMZOQ82sbNU0OuAiljU1om x2DyixKQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qIxjT-00EuqC-Se; Mon, 10 Jul 2023 20:43:43 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Gerald Schaefer , Mike Rapoport , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , linux-s390@vger.kernel.org Subject: [PATCH v5 23/38] s390: Implement the new page table range API Date: Mon, 10 Jul 2023 21:43:24 +0100 Message-Id: <20230710204339.3554919-24-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230710204339.3554919-1-willy@infradead.org> References: <20230710204339.3554919-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5AD98140014 X-Stat-Signature: 7wni9c96qphpc65jwjmkgn17qsdzu8qk X-HE-Tag: 1689021826-30848 X-HE-Meta: U2FsdGVkX18lM+xzplgQmE0e0ugHKJ+DCvLBU9j2lwhRlis5jqgnRD717h+g0PVkPR8AjenEt7kBY4TflWdYJXVDRvjAcQMIBHZFgGBgbghcT27dXBUSZ2RD+NP5phwe5IBf7mIpDhLBHNeMAFeEXpVXhrdJo+0Dfxie41jcl0OLi9OYXq0WIf+qOWprScDmkz9QocYSujarrRyACVtuPSdv1Bjn6dRpGw0y0OygfBAbVt2/QLaZR0OfnVeNWzcJyekN1Ty4+VFN2gMFd+2viaLZo/je2TqC/aUVbgyKp/nOeftoF9ScBIh+uFvxNGzzKAhBsrcgeJIlVyTKjZz/Oz90MzwErOP2ai8PmZP5hgvNSXlA41D2E7Ewu7J52QyBpemRqJooweZc0VDz9BgWSnWgeoAhTXwu1FZLqm2tVX3/PwK6eKvLef9/2kQjuCfBOuVkGLlT2E2yd9wHDUgmQyn3oVg9oI2pTn1h3eV1kzDlfUVuW1diP4waxqybJBKMRsMh8PxGUNUF1lEUBm97xfhA/qhiCLkJCOHdKpk4nkuxY/bfqjIA5umo0resnxVtoXYGoNIz0AOXIUpwqW7m9PV6Ked5ypubBqkO3MBofhmRUstfXCXRKi0AfeifFkTPg4lpGYOsKnYANV9RYjSlE/vPqPdH+pN+m60lP7TPFGYVbK8Y7tJSoZUxsOqd+Ky7jBoAsf/91prGyJoYcClQmd797B7+i3VSEmDlRrMAPc410a8cIw6jdJVwmCMZnu+F0VeQdU7pWzeAjAqhoUjp6zg74t+qA0N0wkZLbnL4nHfOwsrdNj7cq/74566BGa5ajcMZjvhKobj/KwBUTkN3XImsJX+cdwqmO78QPAHl00oTmORObW4LouqxvEMXcqvyYfVqtbtfd+xERelu4sV4cWp9KdeMl/8DXQO7KJWUeniwXgl7uQfkZZljPAl9vU1WfXnkB+SAv6BzphrlLZX o1G9d/ug YZywUGSMYzLervA+DN7zccPzRy6hxQ4k/C0Ygyaz9ZvsR7CJK9NQxDQqz3WHQPoa+AkUuAr5GnQG4R+MT3qyk8f6ubpkypAM5DHF8UKpitTOQXO9rBWVeSFbxExgiv39n2Emo3HIs9CgyUks1ST0P4m7aueKajoI9jfCZFFyfwPwAncfCkxxDsMTDCX2aTDDGHnMN9uassaHsZt+hmcVBUt390Z58Rm/oFuEahrrNIm1aUK2fIqGK9B7auRzQ3CbpXUn5DOiMS8z/sAM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add set_ptes() and update_mmu_cache_range(). Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Gerald Schaefer Acked-by: Mike Rapoport (IBM) Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Alexander Gordeev Cc: Christian Borntraeger Cc: Sven Schnelle Cc: linux-s390@vger.kernel.org --- arch/s390/include/asm/pgtable.h | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index c55f3c3365af..02973c740a5b 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -47,6 +47,7 @@ static inline void update_page_count(int level, long count) * tables contain all the necessary information. */ #define update_mmu_cache(vma, address, ptep) do { } while (0) +#define update_mmu_cache_range(vmf, vma, addr, ptep, nr) do { } while (0) #define update_mmu_cache_pmd(vma, address, ptep) do { } while (0) /* @@ -1316,20 +1317,34 @@ pgprot_t pgprot_writecombine(pgprot_t prot); pgprot_t pgprot_writethrough(pgprot_t prot); /* - * Certain architectures need to do special things when PTEs - * within a page table are directly modified. Thus, the following - * hook is made available. + * Set multiple PTEs to consecutive pages with a single call. All PTEs + * are within the same folio, PMD and VMA. */ -static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, - pte_t *ptep, pte_t entry) +static inline void set_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t entry, unsigned int nr) { if (pte_present(entry)) entry = clear_pte_bit(entry, __pgprot(_PAGE_UNUSED)); - if (mm_has_pgste(mm)) - ptep_set_pte_at(mm, addr, ptep, entry); - else - set_pte(ptep, entry); + if (mm_has_pgste(mm)) { + for (;;) { + ptep_set_pte_at(mm, addr, ptep, entry); + if (--nr == 0) + break; + ptep++; + entry = __pte(pte_val(entry) + PAGE_SIZE); + addr += PAGE_SIZE; + } + } else { + for (;;) { + set_pte(ptep, entry); + if (--nr == 0) + break; + ptep++; + entry = __pte(pte_val(entry) + PAGE_SIZE); + } + } } +#define set_ptes set_ptes /* * Conversion functions: convert a page and protection to a page entry,