From patchwork Fri Feb 5 20:41:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12070887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66408C43381 for ; Fri, 5 Feb 2021 20:42:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E0EB864FBB for ; Fri, 5 Feb 2021 20:42:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0EB864FBB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E5BA96B0072; Fri, 5 Feb 2021 15:42:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DE9FC6B0073; Fri, 5 Feb 2021 15:42:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA1A66B0074; Fri, 5 Feb 2021 15:42:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id 957786B0072 for ; Fri, 5 Feb 2021 15:42:10 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 5DAAE2493 for ; Fri, 5 Feb 2021 20:42:10 +0000 (UTC) X-FDA: 77785386420.29.flame94_4c0165d275e7 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 0F73618086CDA for ; Fri, 5 Feb 2021 20:42:10 +0000 (UTC) X-HE-Tag: flame94_4c0165d275e7 X-Filterd-Recvd-Size: 11835 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Fri, 5 Feb 2021 20:42:09 +0000 (UTC) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 115KdQZR195498; Fri, 5 Feb 2021 20:42:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=0LTNyLhfcQU8WurGe4uk1JQtTl/ZkrEkTZA6jP+zRYs=; b=k+w+qDHd8qSYcAXon/ucFHWY4VaerhKw/R4O5HqYGoG5c0xhw9HPTLeALNXtOeGMSDCf /yo65lmCFDfeNTCE/txbWKkkaBnTBr76bcJWz7IRwjkOQMQx3oywIlF2MxbVpiqqS8ZR WTn1LfWGVMiDVg+ooeey6e6UAboepZu1le7O+7qQK5KlSNtDSPxDNQVdcyWIWJgAjbcs ZCtyShVRY3DTdzQ15wPhdCJIZRMmZvBeJfYIxiVYOonWsgIWJiizpL/sv8zbRT3JLhuS XsSuZklT30W5tCOB/1CcNLgxXO0A/7vEIxKVtaDUf+PFraRcVgUzOQfcyNKVzIfv0bqf Ng== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 36cydmb22e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 05 Feb 2021 20:42:03 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 115KfBmB034035; Fri, 5 Feb 2021 20:42:03 GMT Received: from nam04-sn1-obe.outbound.protection.outlook.com (mail-sn1nam04lp2053.outbound.protection.outlook.com [104.47.44.53]) by aserp3020.oracle.com with ESMTP id 36dhc4p4pd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 05 Feb 2021 20:42:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZAZkrkEf+z6BSGmYS2mfQ+6y+yDr6sG9UOeUY5TAVIRTP4D+9tWL9I0bsk227jBUOa2vm/oyPsSSOnZDgKS8ArGQd5siPLs1aB2bF4GXevAN+QrzZbe655al29o8eoTbvmoLNj5GYFYZO2AsVePvfNAgpEIEjfMUg4TXsC9N2ur3hF6UYjx9zUYK8/wSZM3iC8RHlM6Nhe5MHYBKtJd+vYeJ50E6HSCum/ZkIVVq4cAtteKmMecuFOXQkh562CLBKnwlvLpzWBHYm0J1julD+LfVqu6m+vMwIGq9R6NNx8Hoiv3OosT7eGyxGKXbywRjnPqBNaPvpSJl19gRhXEYNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0LTNyLhfcQU8WurGe4uk1JQtTl/ZkrEkTZA6jP+zRYs=; b=nXAEj/afdFeYAKz2De3O8VwPJ33nBOnWI4eJcoZU4GdwEVbpiF6+inyoBjSdTt/5r6a4rxhxeWsC5Co97kwUX6iLnvaDOpvHCKglIRliRl/e6wJVy4OoXwKAqK7YBn56an9eLHyaefniZgS2ZNKY6oLNZQgDlA+143/MW5bTZN7T5NdPATjkPIHRwrYbFGMqEvS22lDoTMwAuTdWT2JvsKvGJJLiCHIhR7Qz92lfYlCMPMspxCD8lzURKoTjmIHE2Vfu/Ne8uBo2VRqD+sJphqJ3EGTSqmFU2W6ns95nXBTcHDxi5bY9HhTCkK0EF+0Z38+1nS+F4IFxZ8PdlI9z9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0LTNyLhfcQU8WurGe4uk1JQtTl/ZkrEkTZA6jP+zRYs=; b=EOiYzQAV6LpDQNLRS6uUPoxIl54flngKnUpUu/LrbnueQh6/MNANQvspBNLleSUQmoLV8uLI8tm9fnGLmDjLQDcU07R7H4hcY64039RcsQT5pQD8IKMcJOn3CGZtdNNKwmOL1DVq3t8k82niBrKuojXYkylcQAGn2BWQSHYQONc= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3077.namprd10.prod.outlook.com (2603:10b6:a03:8c::12) by SJ0PR10MB4432.namprd10.prod.outlook.com (2603:10b6:a03:2df::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.21; Fri, 5 Feb 2021 20:42:01 +0000 Received: from BYAPR10MB3077.namprd10.prod.outlook.com ([fe80::74a8:8649:e20b:d571]) by BYAPR10MB3077.namprd10.prod.outlook.com ([fe80::74a8:8649:e20b:d571%7]) with mapi id 15.20.3805.024; Fri, 5 Feb 2021 20:42:01 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, Andrew Morton , Jason Gunthorpe , Doug Ledford , John Hubbard , Matthew Wilcox , Joao Martins Subject: [PATCH v3 4/4] RDMA/umem: batch page unpin in __ib_umem_release() Date: Fri, 5 Feb 2021 20:41:27 +0000 Message-Id: <20210205204127.29441-5-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210205204127.29441-1-joao.m.martins@oracle.com> References: <20210205204127.29441-1-joao.m.martins@oracle.com> X-Originating-IP: [94.61.1.144] X-ClientProxiedBy: AM3PR03CA0058.eurprd03.prod.outlook.com (2603:10a6:207:5::16) To BYAPR10MB3077.namprd10.prod.outlook.com (2603:10b6:a03:8c::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from paddy.uk.oracle.com (94.61.1.144) by AM3PR03CA0058.eurprd03.prod.outlook.com (2603:10a6:207:5::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3825.19 via Frontend Transport; Fri, 5 Feb 2021 20:41:59 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dc7463f9-eaa7-490d-877d-08d8ca167d58 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4432: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:370; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NVhnKcxoi+1YeQUlTXA/AghpDMvhCmuUaWzwKcebGrkbaVZl5nd7ZIMyEYlZaj3hgp8sI3BX4mQ2mV1rsLeCcldrZnCRnegOdQJjnmoV18UGYn9pJyxLBLU7ZfG+k0D9I4ztHJN4V2L3DzCPBews1dt3FkU9zqe4qU8YZc8XFV0hpe8CYVkhPHZnw+z/r0U0CGPChnuP+9lJ8Q5SDDPv1ZXFDCtLY65WjFzuhonRU7oAhOMzfyp2+pxDnBU8u16gBQPpNNxUkS3IzpzffM3SdL0EXwGGZEmjYwNBfKygLU5jzKrEJGckERy6oH8OCd1m+MSTgf/Tsv3txkPOsq1Y7bnRM2oOubhkBm/6B9JoRHkT6+pmUbMayeAmTmeg2b6ybWuRqETz0r2d9ghY+EGXlfPr9QV47NxUFQ/QZH9Ee3kThbirmHB9fZGt807QleAm5uuUNA0dsD8f13R2qXtFi9FRMqfpD/k+B+zqJMfhCTFdgqek5FkLuEDv/yS2snxjlBofc/0ND4t17YXbPMnFag== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3077.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(396003)(366004)(136003)(376002)(39860400002)(6916009)(1076003)(5660300002)(186003)(26005)(54906003)(52116002)(103116003)(4326008)(8676002)(83380400001)(66946007)(2906002)(36756003)(6486002)(478600001)(6666004)(66476007)(8936002)(107886003)(66556008)(16526019)(2616005)(7696005)(86362001)(316002)(956004);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: JRDoVXDvElOlBQAF/3+e+BtRTf7ya9ry3liTSNoCHQyfmJ/ZL1eRIxFSEjw04X6JADjKLmqPhVHLfDbggidYPbWJ3TMU2V1zUGVUUp2rocfo+m0OGqFw0tPfgjmQJV6qVF6o7lOR61ZGfhYUPSRcxSxqVeGzphGJBV/SWjl1SF4VHzorsTloKnfZHPa8i2rDaA/4RNiC6GhbaM0k5/DkyOn5aIigIRPgljyHzu+mUzwantmEm1vqvcn2J6e9lyuUf6RMZ5+MEYIOq+BPXTDHb33OjED1oCbSy40gF57BiOLJp0UitbFdzPwCbs5aPHHjqZF3sgeywlqfpzCpgzhm1ZuXuuGLTXQ1KhiZJGxfRIL5eBscNMG/cYgwUxfu41+jfZoFbXQEuXQZ2esg1UdeEoSMKyns+3fzknn15X29xSEvFP4el0G8mkG1c64dW4epwxce+xDaN+56I20Odsa2H7uvWJ7vHQFBS5yio6Cyuk/CqF4f0rSdF9WmZvqe4gTLDF9PAIR4kCgTtwtJeDcg0I94CsYFkV8JeJ9GfvLuMp0QMxS423qgZ5W8OUPlHZEjVGOfVZW7uk7jX/Awclpxij9Fw50YVw3ufxFB6VUaGOel+r3vSNWDCIy6F5I6avlDxGT4ub+IBLyCK+xPGMWTVOVrlFm663vjjib3OfUO2+3U9gP5plMa92lhXtHKyYKmm7eIp88L5t65vKp6n+B+H2RVm9FZDPOn6mdjzwMZxhrjP3CiRnZrMOaS1AXs6Teehs+1eVgqt4HHnBiIXhRHX4AXJ1I/u64Jjig2J1wgKbgO/bgSIDH9+SYJXd8nnQKvXv1Bin+81kKTS1SQjmnSzloTdfrVZZaDUInfW2DcEsJ2xC5Dqn5GvOnXBbstuxOLiJsLzOHcCW4SxosHi+8+FkrhTOHnQw9Mkbjn+PhI8KMetecOmgQr2aUNBtHzWJIRiyXqbaCoj1BzzQ0cQ0+2t3C44FBDpPaDbRWVGTfZ0W/U2VEIpBnRlKPCN8nJwikCc/KD/6V0ibp1OB63QFqCS+4Xmp3dGZdmdH5TlPXKCDOx732mfkAwh/sR/8hU9w4GZwmkFjeJ71r/Z1D6Y+ojObVJhhgGnMdwzbFGoisKnZ+GC8xwxVD1pKMSX90bYqVzsfmyo/jKvBJ+CeiU9oV3ol96bhde+i5Wj8KsZ6ICxQBtINzlEfkLw15DiznArIN60n1PO3A2gptyz3dlwrLIUd/l2x6OpyhOlPkRcYphi28N5fbVjax6rtmyFnQzc/XykCs/KlfrvYY7cC1UwHqNk37IWM3S8D+zAIM08OoowAD3moo4Y1EjjVEBIKcct1Cm X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: dc7463f9-eaa7-490d-877d-08d8ca167d58 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3077.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Feb 2021 20:42:01.5819 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sWfPLzAY2chy3d8qstbtM/v0vQ+9ps1JlaVay5eudAKev0rvZx/z3P3MnuUQq8TZImI8wj6cSPT+DpHul2tXMWcyzjJ7CzD0fph+GUmB9S0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4432 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9886 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 adultscore=0 suspectscore=0 spamscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102050129 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9886 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 adultscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 clxscore=1015 spamscore=0 lowpriorityscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102050129 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the newly added unpin_user_page_range_dirty_lock() for more quickly unpinning a consecutive range of pages represented as compound pages. This will also calculate number of pages to unpin (for the tail pages which matching head page) and thus batch the refcount update. Running a test program which calls mr reg/unreg on a 1G in size and measures cost of both operations together (in a guest using rxe) with THP and hugetlbfs: Before: 590 rounds in 5.003 sec: 8480.335 usec / round 6898 rounds in 60.001 sec: 8698.367 usec / round After: 2688 rounds in 5.002 sec: 1860.786 usec / round 32517 rounds in 60.001 sec: 1845.225 usec / round Signed-off-by: Joao Martins Acked-by: Jason Gunthorpe --- drivers/infiniband/core/umem.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 2dde99a9ba07..9b607013e2a2 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -47,17 +47,17 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int dirty) { - struct sg_page_iter sg_iter; - struct page *page; + bool make_dirty = umem->writable && dirty; + struct scatterlist *sg; + unsigned int i; if (umem->nmap > 0) ib_dma_unmap_sg(dev, umem->sg_head.sgl, umem->sg_nents, DMA_BIDIRECTIONAL); - for_each_sg_page(umem->sg_head.sgl, &sg_iter, umem->sg_nents, 0) { - page = sg_page_iter_page(&sg_iter); - unpin_user_pages_dirty_lock(&page, 1, umem->writable && dirty); - } + for_each_sg(umem->sg_head.sgl, sg, umem->sg_nents, i) + unpin_user_page_range_dirty_lock(sg_page(sg), + DIV_ROUND_UP(sg->length, PAGE_SIZE), make_dirty); sg_free_table(&umem->sg_head); }