From patchwork Wed Sep 13 16:56:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13383565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C314EE01E0 for ; Wed, 13 Sep 2023 16:57:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D1D8D6B01F2; Wed, 13 Sep 2023 12:57:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CCC736B01F3; Wed, 13 Sep 2023 12:57:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B94F96B01F4; Wed, 13 Sep 2023 12:57:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A52D86B01F2 for ; Wed, 13 Sep 2023 12:57:04 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 63B9D1404C7 for ; Wed, 13 Sep 2023 16:57:04 +0000 (UTC) X-FDA: 81232179168.14.345E286 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id A649B12001B for ; Wed, 13 Sep 2023 16:57:02 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IVV5jXBj; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694624222; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iCJtvcwek/PzSsuqar23vXotUlaq5CuwRpVtwMRiY9g=; b=sCnzoYuFggWVdzX92ghhk1jI5IVpsD1peDZyUmTDHHtKayWbce9nffNqeFXxRETCQZ/Fxf dLONj/MGk0Dv6Kp115p8fSiN5N9JKDC1TPouHfpWn09F0QEmW4Szplvcuybr6S1iN7UPwk 1At4DLcLXqKP5cZnptZp1YRsjajv0xw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694624222; a=rsa-sha256; cv=none; b=Ju52/a4we6GLAkTvQ7kgdOXfWC6DZNEyQdKkS/Ds6e0eZsvszm/KNnwIoua/WFXI5mZf6T f0UpkHljbeotCXsp5eUqjlYAXcQfHW7F0uw2dJfJVWuVT/d/5AWJkLLtNwEPOd6DFVfGea CpZhYj86e9sBr8w7t8OaFhTUJmDckqs= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IVV5jXBj; spf=pass (imf29.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694624222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iCJtvcwek/PzSsuqar23vXotUlaq5CuwRpVtwMRiY9g=; b=IVV5jXBjjY2Q0i1MmIoZpQkTxsy71CmhlIPVFmxgLxgmi1puahc6w9/sDiuVlT6wVGE3Gc 6ZAcYD8FGGwFzeMTdazrha2qs/sf4AbrSbQcvgS9DIhWwR4MkZ0OHsE/evZx+WzGIODiJt kMvzIX6H1lsEO9ua75+x1Lvu94vGitM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-674-3UG6PQn5O3qmluWTH-Ob_w-1; Wed, 13 Sep 2023 12:56:56 -0400 X-MC-Unique: 3UG6PQn5O3qmluWTH-Ob_w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EA3DC185A79B; Wed, 13 Sep 2023 16:56:55 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D05340C6EA8; Wed, 13 Sep 2023 16:56:54 +0000 (UTC) From: David Howells To: Al Viro , Linus Torvalds Cc: David Howells , Jens Axboe , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 01/13] iov_iter: Add a benchmarking kunit test Date: Wed, 13 Sep 2023 17:56:36 +0100 Message-ID: <20230913165648.2570623-2-dhowells@redhat.com> In-Reply-To: <20230913165648.2570623-1-dhowells@redhat.com> References: <20230913165648.2570623-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspamd-Queue-Id: A649B12001B X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: bgxojosdxtdccenit4igu11feehyist1 X-HE-Tag: 1694624222-306430 X-HE-Meta: U2FsdGVkX19xGbKToS9zswI4hkr+pFGmJ5lkkJO5iZPgjjO1R/eRLAz5Yuatm9xXEQScbYH2csaJLLv63vviVO5NGqvjisObcFkZVBUH3/wPwuRsR0Rnjv22J0SvwdYxAbpSIysu/SWGueoMOQbufM91RUS9QqBb0//ENShhPIiFU6KF6QNuzl1Yo9QCiSfqooPSAY0Fp1P6zDn2JwL/xJ12ZVb4SUJcfKiE1yfXs/Eqz6gJsU68MGiehfNXeoET4CiFKdbV9uRCrng414AHtZsbzo1OvJQmkiRbX88fVHcsen2gZn6Y6mQJEMuYamfFD6WwSbZTOhi9zlEWS0WlfV+0bRL1edy5EVZ/8eQuRdDkK86olGTEE4RfIkC5a+j3RWdgM1+781jc+aGlbWZpebgrZSr6jH/ESLvaQgX1ChbNWoA066EpgyLafbW77N2A9Muk6mspwSmy2EwwflcFdjBBrkuO0EPp3IdmlGIphMDxFPGdK1c7mmTEs+h3BKmNHxC1n1kx4kixETuPT04ZUMJcFKl9O0z5FYIf3J/P9/ZRvN3hkM8YMDg5MbG+wrGEUheulCKd81PwtDPKavFuQzdBJvKreLULX8T+hM/mH7PoWCB9DMmkxdq64JX+kDzmOMXylel5yVuVxyclhNPgOH92Dv3mW5v5EAAV8bt1wAkjwCKnU9h+5cB9tDhVdCH3oW0lwKhE7bCrfKovjB0csoL8AgCtBdFUBU2QPwHP1gp4Oi7CO5ucVdUYkSnJw3fa5LMKytbjXTxagWEf9I3sMEHMwr6q2AFMSQurQDnCAfJbo8VSXkeE3gOIsZalPp+EDXi4mB3XAQIWu9ziOD1qtvGyzDvnbeNax7JcTrPECFIKGgGX3NdJDwkvZVytSQtgosdARC9MTFSVncT/LtNZce3wilHgI5peE3tV+qO/2QM5yp5bzLVmJrx/tCqKWGsHAUdqLutQcAay8BtL9Em ZKq5p8vE 4kDd01eIruKdw7GpG6sfD6fz9VELQebuFXArHskiLfS/tUJed841WgDiS6wPwptwiI4l8vVQatcDoE7xWLcZyUVclEsVycHpR+tGk1ujLmG+QfebPlbIHEJq4QwUrEAKgqOlEH0mjtqtCS9ZVSeZVVPZ+mX+KckJOFbtrx4UQhznawhoGr2hivADifyBa2fObTi52f5E8Nu/qUL4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --- lib/kunit_iov_iter.c | 181 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 181 insertions(+) diff --git a/lib/kunit_iov_iter.c b/lib/kunit_iov_iter.c index 859b67c4d697..478fea956f58 100644 --- a/lib/kunit_iov_iter.c +++ b/lib/kunit_iov_iter.c @@ -756,6 +756,184 @@ static void __init iov_kunit_extract_pages_xarray(struct kunit *test) KUNIT_SUCCEED(); } +static void iov_kunit_free_page(void *data) +{ + __free_page(data); +} + +static void __init iov_kunit_benchmark_print_stats(struct kunit *test, + unsigned int *samples) +{ + unsigned long total = 0; + int i; + + for (i = 0; i < 16; i++) { + total += samples[i]; + kunit_info(test, "run %x: %u uS\n", i, samples[i]); + } + + kunit_info(test, "avg %lu uS\n", total / 16); +} + +/* + * Time copying 256MiB through an ITER_BVEC. + */ +static void __init iov_kunit_benchmark_bvec(struct kunit *test) +{ + struct iov_iter iter; + struct bio_vec *bvec; + struct page *page, **pages; + unsigned int samples[16]; + ktime_t a, b; + ssize_t copied; + size_t size = 256 * 1024 * 1024, npages = size / PAGE_SIZE; + void *scratch; + int i; + + /* Allocate a page and tile it repeatedly in the buffer. */ + page = alloc_page(GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, page); + kunit_add_action_or_reset(test, iov_kunit_free_page, page); + + bvec = kunit_kmalloc_array(test, npages, sizeof(bvec[0]), GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, bvec); + for (i = 0; i < npages; i++) + bvec_set_page(&bvec[i], page, PAGE_SIZE, 0); + + /* Create a single large buffer to copy to/from. */ + pages = kunit_kmalloc_array(test, npages, sizeof(pages[0]), GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, pages); + for (i = 0; i < npages; i++) + pages[i] = page; + + scratch = vmap(pages, npages, VM_MAP | VM_MAP_PUT_PAGES, PAGE_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, scratch); + kunit_add_action_or_reset(test, iov_kunit_unmap, scratch); + + /* Perform and time a bunch of copies. */ + kunit_info(test, "Benchmarking copy_to_iter() over BVEC:\n"); + for (i = 0; i < 16; i++) { + iov_iter_bvec(&iter, ITER_DEST, bvec, npages, size); + a = ktime_get_real(); + copied = copy_to_iter(scratch, size, &iter); + b = ktime_get_real(); + KUNIT_EXPECT_EQ(test, copied, size); + samples[i] = ktime_to_us(ktime_sub(b, a)); + } + + iov_kunit_benchmark_print_stats(test, samples); + KUNIT_SUCCEED(); +} + +/* + * Time copying 256MiB through an ITER_BVEC in 256 page chunks. + */ +static void __init iov_kunit_benchmark_bvec_split(struct kunit *test) +{ + struct iov_iter iter; + struct bio_vec *bvec; + struct page *page, **pages; + unsigned int samples[16]; + ktime_t a, b; + ssize_t copied; + size_t size, npages = 64; + void *scratch; + int i, j; + + /* Allocate a page and tile it repeatedly in the buffer. */ + page = alloc_page(GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, page); + kunit_add_action_or_reset(test, iov_kunit_free_page, page); + + /* Create a single large buffer to copy to/from. */ + pages = kunit_kmalloc_array(test, npages, sizeof(pages[0]), GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, pages); + for (i = 0; i < npages; i++) + pages[i] = page; + + scratch = vmap(pages, npages, VM_MAP | VM_MAP_PUT_PAGES, PAGE_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, scratch); + kunit_add_action_or_reset(test, iov_kunit_unmap, scratch); + + /* Perform and time a bunch of copies. */ + kunit_info(test, "Benchmarking copy_to_iter() over BVEC:\n"); + for (i = 0; i < 16; i++) { + size = 256 * 1024 * 1024; + a = ktime_get_real(); + do { + size_t part = min(size, npages * PAGE_SIZE); + + bvec = kunit_kmalloc_array(test, npages, sizeof(bvec[0]), GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, bvec); + for (j = 0; j < npages; j++) + bvec_set_page(&bvec[j], page, PAGE_SIZE, 0); + + iov_iter_bvec(&iter, ITER_DEST, bvec, npages, part); + copied = copy_to_iter(scratch, part, &iter); + KUNIT_EXPECT_EQ(test, copied, part); + size -= part; + } while (size > 0); + b = ktime_get_real(); + samples[i] = ktime_to_us(ktime_sub(b, a)); + } + + iov_kunit_benchmark_print_stats(test, samples); + KUNIT_SUCCEED(); +} + +/* + * Time copying 256MiB through an ITER_XARRAY. + */ +static void __init iov_kunit_benchmark_xarray(struct kunit *test) +{ + struct iov_iter iter; + struct xarray *xarray; + struct page *page, **pages; + unsigned int samples[16]; + ktime_t a, b; + ssize_t copied; + size_t size = 256 * 1024 * 1024, npages = size / PAGE_SIZE; + void *scratch; + int i; + + /* Allocate a page and tile it repeatedly in the buffer. */ + page = alloc_page(GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, page); + kunit_add_action_or_reset(test, iov_kunit_free_page, page); + + xarray = iov_kunit_create_xarray(test); + + for (i = 0; i < npages; i++) { + void *x = xa_store(xarray, i, page, GFP_KERNEL); + + KUNIT_ASSERT_FALSE(test, xa_is_err(x)); + } + + /* Create a single large buffer to copy to/from. */ + pages = kunit_kmalloc_array(test, npages, sizeof(pages[0]), GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, pages); + for (i = 0; i < npages; i++) + pages[i] = page; + + scratch = vmap(pages, npages, VM_MAP | VM_MAP_PUT_PAGES, PAGE_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, scratch); + kunit_add_action_or_reset(test, iov_kunit_unmap, scratch); + + /* Perform and time a bunch of copies. */ + kunit_info(test, "Benchmarking copy_to_iter() over XARRAY:\n"); + for (i = 0; i < 16; i++) { + iov_iter_xarray(&iter, ITER_DEST, xarray, 0, size); + a = ktime_get_real(); + copied = copy_to_iter(scratch, size, &iter); + b = ktime_get_real(); + KUNIT_EXPECT_EQ(test, copied, size); + samples[i] = ktime_to_us(ktime_sub(b, a)); + } + + iov_kunit_benchmark_print_stats(test, samples); + KUNIT_SUCCEED(); +} + static struct kunit_case __refdata iov_kunit_cases[] = { KUNIT_CASE(iov_kunit_copy_to_kvec), KUNIT_CASE(iov_kunit_copy_from_kvec), @@ -766,6 +944,9 @@ static struct kunit_case __refdata iov_kunit_cases[] = { KUNIT_CASE(iov_kunit_extract_pages_kvec), KUNIT_CASE(iov_kunit_extract_pages_bvec), KUNIT_CASE(iov_kunit_extract_pages_xarray), + KUNIT_CASE(iov_kunit_benchmark_bvec), + KUNIT_CASE(iov_kunit_benchmark_bvec_split), + KUNIT_CASE(iov_kunit_benchmark_xarray), {} };