From patchwork Tue Oct 26 16:01:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 12585035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A99BFC433FE for ; Tue, 26 Oct 2021 16:01:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0A7EA60EFE for ; Tue, 26 Oct 2021 16:01:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0A7EA60EFE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 73629940009; Tue, 26 Oct 2021 12:01:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E60F940007; Tue, 26 Oct 2021 12:01:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5AD04940009; Tue, 26 Oct 2021 12:01:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 47422940007 for ; Tue, 26 Oct 2021 12:01:27 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6829A31E42 for ; Tue, 26 Oct 2021 16:01:25 +0000 (UTC) X-FDA: 78739053330.20.26FBE9C Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) by imf23.hostedemail.com (Postfix) with ESMTP id 1BAC09000392 for ; Tue, 26 Oct 2021 16:01:16 +0000 (UTC) Received: by mail-il1-f182.google.com with SMTP id h20so10901318ila.4 for ; Tue, 26 Oct 2021 09:01:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20210112.gappssmtp.com; s=20210112; h=to:cc:from:subject:message-id:date:user-agent:mime-version :content-language:content-transfer-encoding; bh=/gA0KFvGXSdxex+fMXRM3sBjJt0PuBVd+JsL/yBoWAs=; b=S2fAGG4HkJg/fX1LYM0MZHwW0KGnqfM8mcWfWbDuklh/p8T3M0K9OUsOyHCDcUo5hs INpxWCPxr0ZzFb6vISnwFI0rxaDZxM0JcjPAmnRlfNf9cFDOHJSj4YRxEnbJo8fdhEA+ d9WqhLodR0Xsuk8pRJN6axTuUpRZ4WKmhq5ehLAhdU1W9AnuenkcSxUlMkv7jWm/nip3 /f4l3FT2KoW6kspGePK1qi0LLUfhnHsbNtQJ5opVIHgTRjU/rBe/iBu2m8Whuu46zsQe OLSuBzE3f9zbjSNcTDnfSGkLkD0iVMTodt6QjUOXrYZQJJhJo6UVB9uTNHCrbH3pkxO5 +/WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:to:cc:from:subject:message-id:date:user-agent :mime-version:content-language:content-transfer-encoding; bh=/gA0KFvGXSdxex+fMXRM3sBjJt0PuBVd+JsL/yBoWAs=; b=QYAm0q/QGsVcnBz+Spcu+pGx/v/irz3nTBprwAuXHwi37lpmOKPL8pieclJd5zo7DO FUQSucxiGt+VPpLB7+4u0dcVk3h3oP7Ge4Prlqb89973ShmVRoeWOMo8XPtrif5PGmrT P2GpWbhyZf8u20rXe55q0bOYuH2jCdznZJMzCthVtPSCzoqWFTwJU3hmIuws+BtwDjSh NA+4VZTnN653dNCt1IqZcAYAtp+oPCLboGrC159TfhcQWFn8gUkmg+d6x8qDOC30kC6r XVn3q3TgigSxJbsW5Xx52fvBi9O4jTipLQmBqajBf22rnZjAmHJCmLUJ4rIZCO7+0BUW CSvQ== X-Gm-Message-State: AOAM533Rswnyt8S99Bmm97nPxjJRcnkX+HcJZuAKVmIn07zou2Oopyql 767SZwR06nyU7x+Dy57Mi+NipcGxjm2NKQ== X-Google-Smtp-Source: ABdhPJzscMM+0lKAzjxwvb/dte7Th6IVJJ3LtihsTRKHETvzzKy+HR/MB+hqRtLf1ZgzcVZbTXUkCA== X-Received: by 2002:a05:6e02:bc9:: with SMTP id c9mr14809334ilu.309.1635264081808; Tue, 26 Oct 2021 09:01:21 -0700 (PDT) Received: from [192.168.1.30] ([207.135.234.126]) by smtp.gmail.com with ESMTPSA id u4sm5406249ilv.39.2021.10.26.09.01.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 26 Oct 2021 09:01:21 -0700 (PDT) To: Andrew Morton Cc: Linux Memory Management List , LKML From: Jens Axboe Subject: [PATCH] mm: move more expensive part of XA setup out of mapping check Message-ID: <49f67983-b802-8929-edab-d807f745c9ca@kernel.dk> Date: Tue, 26 Oct 2021 10:01:20 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 Content-Language: en-US X-Stat-Signature: 978df7hrjc5zqmke5bohsdociitt7x9r Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel-dk.20210112.gappssmtp.com header.s=20210112 header.b=S2fAGG4H; spf=pass (imf23.hostedemail.com: domain of axboe@kernel.dk designates 209.85.166.182 as permitted sender) smtp.mailfrom=axboe@kernel.dk; dmarc=none X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1BAC09000392 X-HE-Tag: 1635264076-978378 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The fast path here is not needing any writeback, yet we spend time setting up the xarray lookup data upfront. Move the part that actually needs to iterate the address space mapping into a separate helper, saving ~30% of the time here. Signed-off-by: Jens Axboe diff --git a/mm/filemap.c b/mm/filemap.c index 850920276846..0e4021edac0b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -638,6 +638,30 @@ static bool mapping_needs_writeback(struct address_space *mapping) return mapping->nrpages; } +static bool filemap_range_has_writeback(struct address_space *mapping, + loff_t start_byte, loff_t end_byte) +{ + XA_STATE(xas, &mapping->i_pages, start_byte >> PAGE_SHIFT); + pgoff_t max = end_byte >> PAGE_SHIFT; + struct page *page; + + if (end_byte < start_byte) + return false; + + rcu_read_lock(); + xas_for_each(&xas, page, max) { + if (xas_retry(&xas, page)) + continue; + if (xa_is_value(page)) + continue; + if (PageDirty(page) || PageLocked(page) || PageWriteback(page)) + break; + } + rcu_read_unlock(); + return page != NULL; + +} + /** * filemap_range_needs_writeback - check if range potentially needs writeback * @mapping: address space within which to check @@ -655,29 +679,12 @@ static bool mapping_needs_writeback(struct address_space *mapping) bool filemap_range_needs_writeback(struct address_space *mapping, loff_t start_byte, loff_t end_byte) { - XA_STATE(xas, &mapping->i_pages, start_byte >> PAGE_SHIFT); - pgoff_t max = end_byte >> PAGE_SHIFT; - struct page *page; - if (!mapping_needs_writeback(mapping)) return false; if (!mapping_tagged(mapping, PAGECACHE_TAG_DIRTY) && !mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK)) return false; - if (end_byte < start_byte) - return false; - - rcu_read_lock(); - xas_for_each(&xas, page, max) { - if (xas_retry(&xas, page)) - continue; - if (xa_is_value(page)) - continue; - if (PageDirty(page) || PageLocked(page) || PageWriteback(page)) - break; - } - rcu_read_unlock(); - return page != NULL; + return filemap_range_has_writeback(mapping, start_byte, end_byte); } EXPORT_SYMBOL_GPL(filemap_range_needs_writeback);