From patchwork Thu Aug 25 15:24:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 12955010 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92E04C32774 for ; Thu, 25 Aug 2022 15:24:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 559AF940007; Thu, 25 Aug 2022 11:24:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50BC694000B; Thu, 25 Aug 2022 11:24:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 341CB94000A; Thu, 25 Aug 2022 11:24:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 033DC6B0075 for ; Thu, 25 Aug 2022 11:24:47 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id C867D1403F2 for ; Thu, 25 Aug 2022 15:24:46 +0000 (UTC) X-FDA: 79838487372.20.D2AA12E Received: from ale.deltatee.com (ale.deltatee.com [204.191.154.188]) by imf14.hostedemail.com (Postfix) with ESMTP id 8503710000E for ; Thu, 25 Aug 2022 15:24:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=PNE4B8sXwOVMOk+3rqBkaNjhAgYrAxODM+UtKodjnl4=; b=blWJgVk3dMLXB9Hn8RuPfUoC0X 14dFNFjLX5NPIJM12TD3x1Ff5i8KItpOFIT6ngkqLr6P5dN+ZseUqea/oX0Jh6mbgxeufH94pmnxR YWNj1cKBdx6e89DpKvvqrl4M3NyL8YZeMHTJjoriIU82VWUxxsalH6aH3+dxyBMzORat8o9RYJ3su dRKf03bADuprGQ05RTIOCvqU8rIuPgfxIC3/k5rLPBXfZbxvyCfTrHn0Cs8V1r2nuN82JKFPGvNx6 ARAzsSfmXrNkHGfrMewamAQWvZgPLup5H2IFOL4sPhpUsJ/BBP0wHhvwu2+9BRzCGMyc6UVSukB28 Hb9Dh65g==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1oREij-0086MC-QX; Thu, 25 Aug 2022 09:24:38 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.94.2) (envelope-from ) id 1oREie-0001ef-EU; Thu, 25 Aug 2022 09:24:32 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org Cc: Christoph Hellwig , Greg Kroah-Hartman , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Matthew Wilcox , Daniel Vetter , Minturn Dave B , Jason Ekstrand , Dave Hansen , Xiong Jianxin , Bjorn Helgaas , Ira Weiny , Robin Murphy , Martin Oliveira , Chaitanya Kulkarni , Ralph Campbell , Stephen Bates , Logan Gunthorpe Date: Thu, 25 Aug 2022 09:24:21 -0600 Message-Id: <20220825152425.6296-5-logang@deltatee.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220825152425.6296-1-logang@deltatee.com> References: <20220825152425.6296-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, hch@lst.de, gregkh@linuxfoundation.org, jgg@ziepe.ca, christian.koenig@amd.com, ddutile@redhat.com, willy@infradead.org, daniel.vetter@ffwll.ch, jason@jlekstrand.net, dave.hansen@linux.intel.com, helgaas@kernel.org, dan.j.williams@intel.com, dave.b.minturn@intel.com, jianxin.xiong@intel.com, ira.weiny@intel.com, robin.murphy@arm.com, martin.oliveira@eideticom.com, ckulkarnilinux@gmail.com, jhubbard@nvidia.com, rcampbell@nvidia.com, sbates@raithlin.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v9 4/8] lib/scatterlist: add check when merging zone device pages X-SA-Exim-Version: 4.2.1 (built Sat, 13 Feb 2021 17:57:42 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661441086; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PNE4B8sXwOVMOk+3rqBkaNjhAgYrAxODM+UtKodjnl4=; b=vyGZhIP1BWyElwyx/oDkiljFUaWR+FTxEn3vAK8V6GM2B9i339ODJeo/U73zq8SzSakP4X QO4iAq6og91djlixzhZuIBXHkUHQu3gdh+JazScYU4LuA5gTQOaoH8egsQw8r+VdyO9GXP YJWr4qLAO96n/rJlhrfK0glwN2qr+pY= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=deltatee.com header.s=20200525 header.b=blWJgVk3; dmarc=pass (policy=none) header.from=deltatee.com; spf=pass (imf14.hostedemail.com: domain of gunthorp@deltatee.com designates 204.191.154.188 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661441086; a=rsa-sha256; cv=none; b=XDZ9wxajey49hCe2KPAGbhQrD+v1FavmqCULGtJrqP2wRrbPUY2tsBtsFevoiDV1ETHPim GjQ2SNXdG/wzaJXfQi7tESPwGkkzA9/QLou59MsCZmev+qKIY7ANi40aTUPSR5W/TyFKM+ cjpCvLZHgVhR5jOQmMSIOTE1InP6oWk= Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=deltatee.com header.s=20200525 header.b=blWJgVk3; dmarc=pass (policy=none) header.from=deltatee.com; spf=pass (imf14.hostedemail.com: domain of gunthorp@deltatee.com designates 204.191.154.188 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com X-Rspam-User: X-Rspamd-Queue-Id: 8503710000E X-Rspamd-Server: rspam10 X-Stat-Signature: g5dtpbn71dehkdazjw67a9pun8cfeydu X-HE-Tag: 1661441084-100096 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Consecutive zone device pages should not be merged into the same sgl or bvec segment with other types of pages or if they belong to different pgmaps. Otherwise getting the pgmap of a given segment is not possible without scanning the entire segment. This helper returns true either if both pages are not zone device pages or both pages are zone device pages with the same pgmap. Factor out the check for page mergability into a pages_are_mergable() helper and add a check with zone_device_pages_are_mergeable(). Signed-off-by: Logan Gunthorpe Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard --- lib/scatterlist.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/lib/scatterlist.c b/lib/scatterlist.c index c8c3d675845c..a0ad2a7959b5 100644 --- a/lib/scatterlist.c +++ b/lib/scatterlist.c @@ -410,6 +410,15 @@ static struct scatterlist *get_next_sg(struct sg_append_table *table, return new_sg; } +static bool pages_are_mergeable(struct page *a, struct page *b) +{ + if (page_to_pfn(a) != page_to_pfn(b) + 1) + return false; + if (!zone_device_pages_have_same_pgmap(a, b)) + return false; + return true; +} + /** * sg_alloc_append_table_from_pages - Allocate and initialize an append sg * table from an array of pages @@ -447,6 +456,7 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, unsigned int chunks, cur_page, seg_len, i, prv_len = 0; unsigned int added_nents = 0; struct scatterlist *s = sgt_append->prv; + struct page *last_pg; /* * The algorithm below requires max_segment to be aligned to PAGE_SIZE @@ -460,21 +470,17 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, return -EOPNOTSUPP; if (sgt_append->prv) { - unsigned long paddr = - (page_to_pfn(sg_page(sgt_append->prv)) * PAGE_SIZE + - sgt_append->prv->offset + sgt_append->prv->length) / - PAGE_SIZE; - if (WARN_ON(offset)) return -EINVAL; /* Merge contiguous pages into the last SG */ prv_len = sgt_append->prv->length; - while (n_pages && page_to_pfn(pages[0]) == paddr) { + last_pg = sg_page(sgt_append->prv); + while (n_pages && pages_are_mergeable(last_pg, pages[0])) { if (sgt_append->prv->length + PAGE_SIZE > max_segment) break; sgt_append->prv->length += PAGE_SIZE; - paddr++; + last_pg = pages[0]; pages++; n_pages--; } @@ -488,7 +494,7 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, for (i = 1; i < n_pages; i++) { seg_len += PAGE_SIZE; if (seg_len >= max_segment || - page_to_pfn(pages[i]) != page_to_pfn(pages[i - 1]) + 1) { + !pages_are_mergeable(pages[i], pages[i - 1])) { chunks++; seg_len = 0; } @@ -504,8 +510,7 @@ int sg_alloc_append_table_from_pages(struct sg_append_table *sgt_append, for (j = cur_page + 1; j < n_pages; j++) { seg_len += PAGE_SIZE; if (seg_len >= max_segment || - page_to_pfn(pages[j]) != - page_to_pfn(pages[j - 1]) + 1) + !pages_are_mergeable(pages[j], pages[j - 1])) break; }