From patchwork Mon Sep 25 12:02:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D32ECE7A95 for ; Mon, 25 Sep 2023 12:03:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A2DE6B0136; Mon, 25 Sep 2023 08:03:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F1C966B0137; Mon, 25 Sep 2023 08:03:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D96966B0138; Mon, 25 Sep 2023 08:03:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C363B6B0136 for ; Mon, 25 Sep 2023 08:03:25 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8BFE5120C83 for ; Mon, 25 Sep 2023 12:03:25 +0000 (UTC) X-FDA: 81274984770.08.55F039C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 1CC531C0012 for ; Mon, 25 Sep 2023 12:03:21 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EiGOB+WN; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LZvLwoc939E6B4TZcwRPSf2buY4OlPtMZfRg4M/9mP0=; b=Oar+kwwgNHTQYEPgerj9aymbewjMqjKAxmIDfEXM7oIvotIxlsl81IIPW0eRoJHVgpVQjt jaU75jJ3+8mjRlbKEuaSUBiM5Gmw2gdEy+/Z2C7E/02S6CMcf9PXHNX8iad7DF3/R2u1pu /hhhTHKBwvkV8UTVlcAA9QYc0J84aSw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EiGOB+WN; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf21.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643402; a=rsa-sha256; cv=none; b=FBX4Z3PPSu+PsHkalOHE/UGezdHXQt1YF5Nbx5JT23LrNf0KgtA7QCtKxHWtnfYp2JCAfu dEp4lAETc+djXuQy4b1nzVN2GZrrnEEJfbRyzRWpt0D+PeHu0WqqrPEs6HgkjvGTlnXX/a TJVdJRcrfdK84fbyfTAt9VGtlPe0pO8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643401; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LZvLwoc939E6B4TZcwRPSf2buY4OlPtMZfRg4M/9mP0=; b=EiGOB+WNm82wr9IbM1uFhpJfLehdgUvX2kDCys7AjPInMDdJCwy0rGdnpsdRJ5Ky+2BS+4 D3mjdrMRdKERyYzLKYxfHAPAb+WUljATej6Jh4DcLCVnbi85ShV/GfpK2ufZSQKU5Br3IU wh6+J8C3lijsZ9P4pw64bPbDEEQs7lE= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-564-j_NYb_xaP4-1I6Rd3LFSKw-1; Mon, 25 Sep 2023 08:03:19 -0400 X-MC-Unique: j_NYb_xaP4-1I6Rd3LFSKw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 50026382135D; Mon, 25 Sep 2023 12:03:18 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id A718C51E3; Mon, 25 Sep 2023 12:03:16 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 01/12] iov_iter: Remove last_offset from iov_iter as it was for ITER_PIPE Date: Mon, 25 Sep 2023 13:02:58 +0100 Message-ID: <20230925120309.1731676-2-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Queue-Id: 1CC531C0012 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 74ust3qic89i9wyfu16ka57fcr5o77nd X-HE-Tag: 1695643401-787573 X-HE-Meta: U2FsdGVkX19cZNbUeaG3Yc8SY8/DQs7J2bfWGT0jcv7UGwz2rzAPNGyisslX3VX3/6+DHRGqxo273ocUBgJMGKJG07CBmGH0fabmA/Hyxl0nsH+7mFhJsixkwK/BGQx3qJz4aI9ctNAQ73cpPQdugvfdBdl91yYJffKeAH8NefNsQq21j+sMtzduVeEYT/rSGIxzzQOzK29y6J0u1TfojgdfkhAXzUZFaAflCp7liXJQWLxiPCS8rJtoPgn7NTMneWzXtj9ffv3Gpc2RIZqI1+DkrrktuLkwlbFW71zYiOdDiIfSEWBv9szlTDe1q4ikXc3Ag+tHMnmVXWY9MvR00fUgsEwCSnQG3BZvKvHDGfQlMC7ddD7HD3glazrGYVI1OzCNwk0JCr7K1t4jzFh3DMPVJ2x94S/iFaIzSG6am9pvrPfKDb5kXkWWDm/cm9h+WyWVIbFxQ/6n9CdpSPeIznFS2IlrIq7DH2vybhaD1fEr+GdDO0aWcgmAh2PeGDq2AYPDmnTqEEKVQ6mxv/60KIoblnYyNRM29qAHlCeCYn1DjevWWd/ieK0mrTyd8PYklJQkRdhemADb1n0I+QwhKWpLWFfp+bpdsIzIp5hF2JucHR9O+B4gVXYQFxeDe8gl6Pjt52tj+zKVU/nI/4itx3kJbmxPmdQF/7ug5vg59jJFRSX71K7Gp8X0Twl25DVqtS2trSHggNtKARv73KoZIkvUnmISV992/0cXr6c7tf/FTLsG98G5H+QHdc7TCqmPY6nzJ8inS3FykCHywqWnyzb/UAUKNqUGTmTCiJCJNlciylrLUaUqUdPn+ZuZopgAWlEZY/CConUocAvp8WuWhusKxpb7MdNp9oL1di8jkAKZtvG+OONYF2G7CsfdDPTH9BXj8jRiuwiZT5gyO5yhcMJSioU5n+vbodT2A3+5tx54D6JZsZBn4USLm0iRlhDhQpxX6kFFlZYwzn9ZQ7x 1dSbKJ+K s0LeSIPhhfVaixtNZouLwX7BDTK/+uwCujiw/5G78YVYC2Ek+V4H9Gn5PI4Clf7xnApjExMBitNAQh2rjp1wjt2a3Haj6ajcYAaNPKNxqVFqEtL5Ec4Ar2z6Bp7U/nDVEpxqO+RUdIr71myLuN9aXE7u4RaSZUY6xbGdrwVflMplBS9ew7mlpj+E7TvbVHcBfk/e/8MFi37RIVXhYWtkhISC7hRz+JUjyRD/Wc28lYF21qG1RcRFOmSN6t6UFUtx07rs86gM9F9UY+X5BWQboFiX96Tispx/OIlZqBQKQ5sT0j4rQbuXHUWWgcx0/LkWwxF9h2ekH3Do1++fcbwr243odJqRRA1/ibnTsDzS/ZlJkY9z6p5RNuW4sPHFBbbxkIJtjZQwpslr1L1q9Snl153r7EA0Q2uaphprncLNm0k7cQWQv5AYn2CcEWpbxhveuGqeiOlV8U1sXTAaFoxGF6Q/Zpvk4o5fhSqgLq0HnBRbYxwdrPBIHRzQ4hgpYn0RxWlEFJF43/2bG/RGDzKMDWW4Bj8M9d23APt+t87seIZJbz6sUIfJ1ygZPoKXuxMNMY73+J7e4piS/mXg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that ITER_PIPE has been removed, iov_iter::last_offset is no longer used, so remove it. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- include/linux/uio.h | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 42bce38a8e87..2000e42a6586 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -44,10 +44,7 @@ struct iov_iter { bool nofault; bool data_source; bool user_backed; - union { - size_t iov_offset; - int last_offset; - }; + size_t iov_offset; /* * Hack alert: overlay ubuf_iovec with iovec + count, so * that the members resolve correctly regardless of the type From patchwork Mon Sep 25 12:02:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1630CE7A81 for ; Mon, 25 Sep 2023 12:03:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 449A46B0139; Mon, 25 Sep 2023 08:03:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D2046B013A; Mon, 25 Sep 2023 08:03:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 272416B013B; Mon, 25 Sep 2023 08:03:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 101AB6B0139 for ; Mon, 25 Sep 2023 08:03:32 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7F27E1A0B3F for ; Mon, 25 Sep 2023 12:03:31 +0000 (UTC) X-FDA: 81274985022.18.F205B74 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf12.hostedemail.com (Postfix) with ESMTP id 1DA5F4001B for ; Mon, 25 Sep 2023 12:03:27 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=I6wvvrUJ; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643408; a=rsa-sha256; cv=none; b=J26jb/OOPyhXbS1YIdpiHSfCBl/J6pywXxqyE3ZNW9bTbCcAEl9dVALsn4NJhGDqBJ99uL AVgIf8seZQqYVeUVA6NY3dtMvlsv+0yZdOJShO0onaps+MygDbLv4kk0uviLn/jXzTx/hi pGB4aUrsUWB/jfz+/s93Vk4LtqG6q2w= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=I6wvvrUJ; spf=pass (imf12.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643408; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=psu53B47Zw+z+D2u7wWqQ/a6N4tr+aS4fkyxU8KLlHI=; b=k+KRKihqR8fnMRnOoeBfYXyv8qcfFQIYRFglAmDpSt5l4wfFm2HsUY107ApMiUCOhhES3w 55UKr9Yc7Df8oLWmKztdWdPKTniHA66plAalOyFREXvYVd/FVsvTqJnckklxTrvI1curiC oGhsgSOWzZdw2xbDIJajl+2yY1Tn744= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643407; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=psu53B47Zw+z+D2u7wWqQ/a6N4tr+aS4fkyxU8KLlHI=; b=I6wvvrUJC9TMUqYJOTkyUz0UjhKya/gt25blpg+yEAMFj0M14fIlMZeud4i9uWMEqFkCBC iAn264SYq/8TVUYzUPONnGsKYM/SqpmLHHmJ0LjFBbuieq9LEoO4Oalkf40QcRzBc/fb4v 9SRcxlEBGgRkG6eMVvKghqJV6Hu6/Gw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-396-94it2JUUPB6shaAaGHHbuA-1; Mon, 25 Sep 2023 08:03:22 -0400 X-MC-Unique: 94it2JUUPB6shaAaGHHbuA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CE6CF8039D1; Mon, 25 Sep 2023 12:03:21 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0E5B2711293; Mon, 25 Sep 2023 12:03:18 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Dan Williams , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , x86@kernel.org Subject: [PATCH v7 02/12] iov_iter, x86: Be consistent about the __user tag on copy_mc_to_user() Date: Mon, 25 Sep 2023 13:02:59 +0100 Message-ID: <20230925120309.1731676-3-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 1DA5F4001B X-Stat-Signature: 7kf4o1zxozermzt7u8eqg854gtu1xcfd X-Rspam-User: X-HE-Tag: 1695643407-355467 X-HE-Meta: U2FsdGVkX19olRruel2oxq8eHYo+W9OlmAUZ+PPuItbGL83ZF+y79EO2j9sAdfx/+1McpPQA4ucw2ZLrnmvBn5AfAf4hKI7RGQdD0EZyLiD1VBHYwVxR55FyWzaBF9QWR8q4pXMfUBgWkKo3zWoKFcqfe15gkHK95dCYuBnRMNV9TG/ytwxPqyCGepIyH4P/epk6MAsxhgzAAQf/u/lM3J02LGBpwW3dFcpeAczmYRk6bSlsRY7a/aUqFyqZh6Xwx78UaGxJRNu6957pjdiKZix0jExDnZ+nUVI4//AJidZkPhBK6SKbNvN7uEG3kr4lS0NW82E2HtflWNLxm0Ocfe19t1GwTYa/q9GSTrpn2+D8LdRmzemHARl1bPOoo4wTdPEbUMsvK9GNJeUuAtDX5euxuvkQhzGJdszUL87EkC611iUuCwrimxOFVV74Dl58zlLY4IAYSOc6dbIl05YsEgAPrkYPSxH61tiQyk7m2m9Wmbq2jnDDu8K2umxTFnNRASqT3i8mjdzyJGD1EKhYjBwv7ffheSBc9a6Cghzdmb0HBbjRMdUcOye5Ygdkyv7ubCJueU9R25Rfyv8E4ncwiVFDzrofUeBRLmmrH2DMahPlfbGPq0/r/TXe/TysbCf9Z3ATiOkC+QvwcrTteCvv+esYfNX0KQTFd/0IDaoFcTCnhkA0zGx4hZOhck/8qU6xW1clDs7hoVrMDEHcPXzNEgXYy0x4uyl7S5EfblVQqOIswdXn7dtkWEpP2a6zQcGQTQYaG3RMHiBcuD+8NNxQQsInx9rtyrkSlBdZED9s6jTJg3wbcuEfAx6IFj2mxDBqH+1I4tjzQ/0xmxLtgRu25IOESw6uf8TJs6UaAw43OzDjs7wjpdjuVnoBRO28wkeK76q690nKJnZaDEkctbqd//mAxj5u9cH/AR3lu19l9LSKPNphZY+s3GuNp6WqVx5ajm+D7Y0qg9//Riu6fq/ L7mlXycX FPkuqpkai9PU0P6Hk46bsQbG7EjnBek5kKjRDBYB/t9CjKC/yaSqhOtBHqMQG4s072UprGu6xbc5aYYBIW3tZoM7rScRwdgRxlIQsZNfJvXHTlFzT/wsnTNmdCpX/Xw3xHvjzzoM4bDXmo+vKb9Ns5j7fi3lN7HqnvoEI1V59jNnd6hVSVcFwaAp3BSPj9hw/8QF3kJivAHGqNZzr7sIE0TtKh2PMwXTakugImBj9XMsC/ado9Anu+e/Y2lVmRb0qxdHkL4EiO6WUS0xj3woGmU0W33Yzg8unci7VPofSEd9VBiZYBK6k8O9uwswprz4ZJm2dPgleKIYv/BjEKK317ziAf9GurH1I7/Z4COMqfSCisH2C8gmeoZJyCctnwMApiO0D2HjwoJkOMvZP5cys6GpDPMEX9syNrPIopMkcabdQNZW+1KLsABdXIFfrwi//H+KAYJu9ajuRsGeuZXVEFIio6mJOnpKN7l5j2PduOXh1/2j+ByKFS8DeGbRWoqCMbd71a0NSallr+rjcrsGNRd/ZP6rD6edO+ZH9m2VreMDhQCvV3RtUX73ojwlU+fIBDU5fqrHvyh0gW6NanGuDw2cP1eHLh6Ru4RW2iuwqfUui1sh+Ir0TVLdAYVMoHtFH6dV5U1BUZSmHgjDQIQXDXlixcThMEVEfruAR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: copy_mc_to_user() has the destination marked __user on powerpc, but not on x86; the latter results in a sparse warning in lib/iov_iter.c. Fix this by applying the tag on x86 too. Fixes: ec6347bb4339 ("x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}()") Signed-off-by: David Howells cc: Dan Williams cc: Thomas Gleixner cc: Ingo Molnar cc: Borislav Petkov cc: Dave Hansen cc: "H. Peter Anvin" cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: x86@kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Acked-by: Borislav Petkov (AMD) --- arch/x86/include/asm/uaccess.h | 2 +- arch/x86/lib/copy_mc.c | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/uaccess.h b/arch/x86/include/asm/uaccess.h index 8bae40a66282..5c367c1290c3 100644 --- a/arch/x86/include/asm/uaccess.h +++ b/arch/x86/include/asm/uaccess.h @@ -496,7 +496,7 @@ copy_mc_to_kernel(void *to, const void *from, unsigned len); #define copy_mc_to_kernel copy_mc_to_kernel unsigned long __must_check -copy_mc_to_user(void *to, const void *from, unsigned len); +copy_mc_to_user(void __user *to, const void *from, unsigned len); #endif /* diff --git a/arch/x86/lib/copy_mc.c b/arch/x86/lib/copy_mc.c index 80efd45a7761..6e8b7e600def 100644 --- a/arch/x86/lib/copy_mc.c +++ b/arch/x86/lib/copy_mc.c @@ -70,23 +70,23 @@ unsigned long __must_check copy_mc_to_kernel(void *dst, const void *src, unsigne } EXPORT_SYMBOL_GPL(copy_mc_to_kernel); -unsigned long __must_check copy_mc_to_user(void *dst, const void *src, unsigned len) +unsigned long __must_check copy_mc_to_user(void __user *dst, const void *src, unsigned len) { unsigned long ret; if (copy_mc_fragile_enabled) { __uaccess_begin(); - ret = copy_mc_fragile(dst, src, len); + ret = copy_mc_fragile((__force void *)dst, src, len); __uaccess_end(); return ret; } if (static_cpu_has(X86_FEATURE_ERMS)) { __uaccess_begin(); - ret = copy_mc_enhanced_fast_string(dst, src, len); + ret = copy_mc_enhanced_fast_string((__force void *)dst, src, len); __uaccess_end(); return ret; } - return copy_user_generic(dst, src, len); + return copy_user_generic((__force void *)dst, src, len); } From patchwork Mon Sep 25 12:03:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9E6DCE7A94 for ; Mon, 25 Sep 2023 12:03:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CA086B013B; Mon, 25 Sep 2023 08:03:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B4096B013E; Mon, 25 Sep 2023 08:03:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70B336B0140; Mon, 25 Sep 2023 08:03:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4C4E46B013B for ; Mon, 25 Sep 2023 08:03:36 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2745D1CA7A2 for ; Mon, 25 Sep 2023 12:03:36 +0000 (UTC) X-FDA: 81274985232.01.91BA34F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 09B842001A for ; Mon, 25 Sep 2023 12:03:33 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MSp2CwdP; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643414; a=rsa-sha256; cv=none; b=3yAMZZRVjpECb4RGuvza+jYU3papQHwJjARBKWM2Fmi6dH7cDBVeF5THggFkopw6koI2jT Jrq4KCQxmKcLpxRDFpNpVUN9M3CFmgRdpR6yFOwQ/KghqPuiPwuOSToWHH/oqM8J4bHgin jd8y9+uEhRLlnxkl5Y7d1jznCC7gw0k= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MSp2CwdP; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643414; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7BA8QLNemiJIXUC+XXC0PAivfRO0j2uaFr7AQHg4nPM=; b=KtvPgzLRKna5WSHl5IfKeygGySwgF7BoI3qaANhbqW1vYP+RGXEMUpLJfXTGnHjf7OA26E TVcckzRCRmY9siGBDLJ0dZffwm9wlUF8eQG8wiV2VXxapclbCjKW/UW8ySoex1GbvJO7uW MKrE00vjzvkWZVSDfWTKolNHzXRnTUQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7BA8QLNemiJIXUC+XXC0PAivfRO0j2uaFr7AQHg4nPM=; b=MSp2CwdPtH0l3YwVo3Vn6QrZPrKGkcGOhhcNdcQEixmQ8MJf5oHOGhgV8HFROtM7gnjV7k XuCfoM/Zkxo9NMo2gVJW+sDheFbhhRKvZ/FyHNHJ4D4eJDq13wtN2wsc9aghKy8D1Z92kf 6kITnKGEPjZ0Lb1ODm5cSZnG5+ncRs8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-443-3j0u1djBOj-K7pheSFVopw-1; Mon, 25 Sep 2023 08:03:26 -0400 X-MC-Unique: 3j0u1djBOj-K7pheSFVopw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0B212811E88; Mon, 25 Sep 2023 12:03:25 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7101940C2064; Mon, 25 Sep 2023 12:03:22 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Jaroslav Kysela , Takashi Iwai , Oswald Buddenhagen , Suren Baghdasaryan , Kuninori Morimoto , alsa-devel@alsa-project.org Subject: [PATCH v7 03/12] sound: Fix snd_pcm_readv()/writev() to use iov access functions Date: Mon, 25 Sep 2023 13:03:00 +0100 Message-ID: <20230925120309.1731676-4-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 09B842001A X-Stat-Signature: ioxq9ba5h3emstnxn4coskxsadytfxht X-Rspam-User: X-HE-Tag: 1695643413-559341 X-HE-Meta: U2FsdGVkX19eTXJUEq1L/xiW1rCu7DxmAlYOM7PxFbGBKfWVLYx6eO5zE9i8TtJ83Q9/etxCsLug7Do/IZUAYMriFctzWnrq7QpGSZXtoetNo8AM24WA48j5t4psDnDrPGDJZT4Wa/r9Bni4RLDkv/lqJtipEWS+E5ibjyYjdI2W2cElPWDD7dqXZ7Fqd2psldJ/+K1Ts3hBfvaThCVCQIBQQRrWqxaxoaR450CFpj6VuDMiymwGrNH3Kqd3sliGNPhF/O25TMFUTShBtRMppiWW3V4bnflY7/KAyTu2Zpk1VwprWqacLM+gwhZsM1raCpag2wFm9b8MgtLsa9zhdCGHfRXL56rkh8TTK5FvUITi+ZSST/RCLB0OuU0jiApLTRt/exmFnBrDO/5ikSaeaVnCV7sFD3DysPeNgjJY5ljf9xRSV1IV+TlVTrNhQDMu9A8GjgyAvhAI4qEk9lAHHxM87GUDpDOetQzpiqd1GoeNX5XA5x6RCs9Zva14c/HiZWYsxkQIQ7aPR3BBT+pr22vlEGHXC7JvDcWScl73XpjrWjV7B3+Z5aZZvbaG4OILF8T1EWWTtLgWKWG9Xz8j/tzXfl/7bTSaowIinm0vtOyfjw/ka/x/fIS1xpGeiFupBlucvuKOl319AYE4djmpv4pzl2Lf8QPBTAH7Uqe11nGRtJX6jp0lXYDIikYHmglfcU7Ru+FTI/ulwRTqpBZnscd9N666Q3NAYDuw0jmPd11Ya5liRyWj3mKoqrp0eNo/gc65D3HSqPyhWAH7PoKMAvoJAqytZ5dazJeJL4wFx55D2PYQTJrfK4T6SJ9OvSfWzVd6Sox/BqZ9Hyc3NyxIc4CDOq6SdTfn0tg4wpSU0eeUJCJwDntdg3SC4Y/JbQE2BoHWd51a/0Wz+tIOJ9M9LZm0ush3YHaNQ1Diq4Nw6pD9qP1HwhsTjiAbBtp/+UR75hGp+BSm/O55AfPJZlt CdNsrX9s kcLsoT3zgkScnJfUjZLIGKMm/qhUFuui6+BCjRKSuvvP5Zi9I763EH6+G8p5jWL2tn5cKnNC40yCZSrZixg6jjawOCHmm42iXUy0zwQgR5ZvghcBfxFq0lyHoEnKsD0Kr2YkTIKjwP9GgJMYDMn/6SPpQd8FVAsNFa1WoXqvzH+Ltly7MSVkYJdowtw6qspgdGTlngJ6hnFlhcjrdjPZX8dKmrTWPW7kV7KU0LjZuAQDQBUNCvT/DNeR8XlrG7BHbi5v67SMWvHRRt6VnM+JuAC/wC9LfGdiZmOut2pnTQqsDOyDWKLgFwiDEv3uPtLJGSJgCCShhkeiSDxD2CnWvF5tBenk3BUiPVHmD6sK/+I5YNZfVR4q27FbKe1LfdmeGqiGzXhwdJ6RDaW17jL0VBqLQaQX9wNlno0cGux9YNWf07IHiZwovn3lKJmRsSPezM+0x X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fix snd_pcm_readv()/writev() to use iov access functions rather than poking at the iov_iter internals directly. Signed-off-by: David Howells Reviewed-by: Jaroslav Kysela Reviewed-by: Takashi Iwai cc: Oswald Buddenhagen cc: Jens Axboe cc: Suren Baghdasaryan cc: Kuninori Morimoto cc: alsa-devel@alsa-project.org --- sound/core/pcm_native.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c index bd9ddf412b46..9a69236fa207 100644 --- a/sound/core/pcm_native.c +++ b/sound/core/pcm_native.c @@ -3527,7 +3527,7 @@ static ssize_t snd_pcm_readv(struct kiocb *iocb, struct iov_iter *to) if (runtime->state == SNDRV_PCM_STATE_OPEN || runtime->state == SNDRV_PCM_STATE_DISCONNECTED) return -EBADFD; - if (!to->user_backed) + if (!user_backed_iter(to)) return -EINVAL; if (to->nr_segs > 1024 || to->nr_segs != runtime->channels) return -EINVAL; @@ -3567,7 +3567,7 @@ static ssize_t snd_pcm_writev(struct kiocb *iocb, struct iov_iter *from) if (runtime->state == SNDRV_PCM_STATE_OPEN || runtime->state == SNDRV_PCM_STATE_DISCONNECTED) return -EBADFD; - if (!from->user_backed) + if (!user_backed_iter(from)) return -EINVAL; if (from->nr_segs > 128 || from->nr_segs != runtime->channels || !frame_aligned(runtime, iov->iov_len)) From patchwork Mon Sep 25 12:03:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13256CE7A95 for ; Mon, 25 Sep 2023 12:03:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C6E56B013A; Mon, 25 Sep 2023 08:03:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 928F26B013B; Mon, 25 Sep 2023 08:03:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A3C76B013E; Mon, 25 Sep 2023 08:03:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 64BFE6B013A for ; Mon, 25 Sep 2023 08:03:35 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 3BBA440BFA for ; Mon, 25 Sep 2023 12:03:35 +0000 (UTC) X-FDA: 81274985190.16.D41EDEC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf08.hostedemail.com (Postfix) with ESMTP id C646716002A for ; Mon, 25 Sep 2023 12:03:31 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=D7xWpvJz; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643411; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j/K7bby96tUMA/iC07lWusXqTlhgR9zlJgJ7++h0ZQI=; b=5yL4QkVP4RCNljG2YuH7ghrXGWSOXdG8ot8eukgNDz+KkWu5do6dd1Os+R/tw9yRzB816A 64oErO4u9i2lBfo+uf5blvHTAbjxK8TwXm08cKvsrLKSJAEQ3t536o47xhIexvMJDV8HfJ c16Y52FjgarCVBKs0tysLxONN5ZScDg= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=D7xWpvJz; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf08.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643411; a=rsa-sha256; cv=none; b=BucMTHwu1HSotIBrxutdBf5Q2J0ggiib9ipprjgVyKwFlUbTLCF/iPPCeyn6m79WM/DWSu 3eQxaFidIwec2iDJVAlNSg5zWfQWCI0XgFAvkTX53SlwTY1W/mn0E6UCqHvx2Zm78IATGH lkYpIwK7rZThJPsJg/jpcV4vEmgaHDg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643411; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j/K7bby96tUMA/iC07lWusXqTlhgR9zlJgJ7++h0ZQI=; b=D7xWpvJzU2dieu6MANtMV21oNUhIH+An7pAgX67s4qwHrSdfDfnd9pHiNcMV/iwqFUtLGD 2gI8A4KBmU5KgvoDLnUMYQIGJsOtHeZX0Ig9hO0oAB+Xkbj0d2av7cBkSoihv4vq9siVBY l/VqUDZAQ0vNcbWxnzZe01BnOo7CD+Q= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-322-N5vnkQuMM-ymsL1ps_d4Kg-1; Mon, 25 Sep 2023 08:03:29 -0400 X-MC-Unique: N5vnkQuMM-ymsL1ps_d4Kg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AC2E7802C1A; Mon, 25 Sep 2023 12:03:28 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB2892156701; Mon, 25 Sep 2023 12:03:25 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Dennis Dalessandro , Jason Gunthorpe , Leon Romanovsky , linux-rdma@vger.kernel.org Subject: [PATCH v7 04/12] infiniband: Use user_backed_iter() to see if iterator is UBUF/IOVEC Date: Mon, 25 Sep 2023 13:03:01 +0100 Message-ID: <20230925120309.1731676-5-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: C646716002A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: x3wteqxke5z6sm8nit3j16zgxc8kcqio X-HE-Tag: 1695643411-437808 X-HE-Meta: U2FsdGVkX1/iqMtOEfpxfYtQ16348pPmBTfT+WB43RMtRwBuxhKTcol93eG9Vj2cmc//qBPVE0BkubLsNHGcmrlN7+Sr+XKxvHijjuZ3bBnUbWFi6ut2DAdi6GT2Ky1vVw1/8ZlriaZPBVTjqNgthF8fpyIbxG6tOHV+RQ6ZmNtvfqnfz6c880KFS9eVPPKX6kPHQW69L5AXJuhfQzaYsZ8HtVKaAWfIIO54QSau+88pBu5a7IAtHFffFNfRXvDWwkgiOnreVXSLPmoIfNjBzBYgM5SgtNkUU+UTLZ8W1rzJabKz8Y+9+4GvvxacBxdQC4I6aO//Dg4mf8HYgAfxcTBiNRZSRc0wbeRGDL6HZ/0rnid2UO3nmoqnF/l1Xkm13qDx9uuzNvqxe7Q/1xh18DMFM2yVxPnLFLv3Nf3Ce0B3km0ZEUTvBNT+BRu/F4DP5KxgZlwTjQU84E5eXli3UbJ9XnqszKRGWz2h2YT4g0gy6aGjLzNS0c9mJP167zglEwavow6PBfIaLI0KXzyy970HGM0v/UTx+IicnoI6rZudYnBeVYZN1vGI5dTeIX5241kzOR55vpM/MFa6sl9D7zc87lA9uFBhHNgwEq6cs1lzscy7CKx4pNc1IK0N0E85Z/fmWJU688ta/50iOextULtR+Qb1j4ggvzKr5GzHBYdNCYZCO7iScpGan4cXRaVnyxKm3g/nEGsI4FMRldOn3ROj70DQPnuvEot8w2zcuWZSt3LHpMWINh2ISnSZp/Rjy7arzzTEmaQsi1Qy8UwTVKY7vMyRHankd54nfNyzd3620N2wCIQP5mPsaTAwjsmfKnmig2wA/gaGxkBsuxs2M6A99QsIJIfztKUbNhLK7dvqb0bdWeGNFBzPP5u0/BrsZ7aRPJvx0FxdEn+/Ru26x1joU8zh+vNIANay6agbbCOwpUzeqX84AERb2zZLkVwROb5QuVVqsK4kaL0o5MK te5Y4DVF zOx3oXISyYk3yKMWr8atHnwiSn1aEELiUY+SkQ8jTWtnMMupUuy45wI2M2jTgQAflS/LWO/oRLknASd06iuuTGNuzjAGM7klxoD8HZAI7FfwoQyJ8lcUsWLaDYhkKTFJB1B37XFNX5ioOSJC+KtxkjiN62L4R2AjcvATQnFN7yXDbDKj0TSgamwsgmUqoM/3P58yNaazMM0FRFU+x1UOtuMAb0qg8mCF5rJnnSLHx+8/2TkQ8cU7/6Hy1z5zB55p3ei5qJikXoCx1BNTIC77NVbcfLgWxV0mgh0W66OvkUBWtrg1PO/6l70Bh7HM1e/u8Sl3dS/txwnimkEeDcJr0WA7MY6MJdlS8wk2syRlvsWCIYHs8L3y++tjHWwvA4LlnV2ufjoXfOO0rYXA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use user_backed_iter() to see if iterator is UBUF/IOVEC rather than poking inside the iterator. Signed-off-by: David Howells cc: Dennis Dalessandro cc: Jason Gunthorpe cc: Leon Romanovsky cc: linux-rdma@vger.kernel.org --- drivers/infiniband/hw/hfi1/file_ops.c | 2 +- drivers/infiniband/hw/qib/qib_file_ops.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c index a5ab22cedd41..788fc249234f 100644 --- a/drivers/infiniband/hw/hfi1/file_ops.c +++ b/drivers/infiniband/hw/hfi1/file_ops.c @@ -267,7 +267,7 @@ static ssize_t hfi1_write_iter(struct kiocb *kiocb, struct iov_iter *from) if (!HFI1_CAP_IS_KSET(SDMA)) return -EINVAL; - if (!from->user_backed) + if (!user_backed_iter(from)) return -EINVAL; idx = srcu_read_lock(&fd->pq_srcu); pq = srcu_dereference(fd->pq, &fd->pq_srcu); diff --git a/drivers/infiniband/hw/qib/qib_file_ops.c b/drivers/infiniband/hw/qib/qib_file_ops.c index 152952127f13..29e4c59aa23b 100644 --- a/drivers/infiniband/hw/qib/qib_file_ops.c +++ b/drivers/infiniband/hw/qib/qib_file_ops.c @@ -2244,7 +2244,7 @@ static ssize_t qib_write_iter(struct kiocb *iocb, struct iov_iter *from) struct qib_ctxtdata *rcd = ctxt_fp(iocb->ki_filp); struct qib_user_sdma_queue *pq = fp->pq; - if (!from->user_backed || !from->nr_segs || !pq) + if (!user_backed_iter(from) || !from->nr_segs || !pq) return -EINVAL; return qib_user_sdma_writev(rcd, pq, iter_iov(from), from->nr_segs); From patchwork Mon Sep 25 12:03:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50DBBCE7A95 for ; Mon, 25 Sep 2023 12:03:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 053896B0140; Mon, 25 Sep 2023 08:03:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F1CE66B0143; Mon, 25 Sep 2023 08:03:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D24D76B0144; Mon, 25 Sep 2023 08:03:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B1BBD6B0140 for ; Mon, 25 Sep 2023 08:03:37 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 939EC4066D for ; Mon, 25 Sep 2023 12:03:37 +0000 (UTC) X-FDA: 81274985274.30.C9FFF82 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf02.hostedemail.com (Postfix) with ESMTP id C8F0D8000E for ; Mon, 25 Sep 2023 12:03:35 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RWzMWhiD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf02.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643415; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oQjxv/qWpzFGTlwOWGQlwUCcgnofDI2I99l4qJgz63Y=; b=vq3gWZsIFxROrHAV6BT3nTiKKAkIr48zcB4HExBPxI3yViHYT1SWMqGfyPQzOFu2Mnook/ ljSjdTGWXiBt7HHrKCSA8h8S2c/yJ193JCduaWLSVM9GAH11sPR+THPZNamn+hPUzj/YpA CAaOzBieWfHHT5UZGC9+zIkKI30g1FM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RWzMWhiD; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf02.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643415; a=rsa-sha256; cv=none; b=RDKZNah+Dk7SSqueyBY8QYuobC7YszWCwM5Ka/F0wILD9sD0OVVk7Fh8Up6mNS/bg6KeB/ Br9t7dHbea/Fuo383vZk8DOqNcmZLDPKeoERNFUl8s4JpkY6d4QKpxv5DcEf229GT/Spjb 8H7WqpRuwN2Hlxp0+B51oA83a5Psbs8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643415; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oQjxv/qWpzFGTlwOWGQlwUCcgnofDI2I99l4qJgz63Y=; b=RWzMWhiDR/6Y4XDNzlO3NIhL5wNvFQxGNetqxFkvFe9wOJusFsutXSiV80HavkfnTkTwZS TlKj6keQ1tBq6TM9gBj5NxPcrbeiUwgxUXmbQmuutD+Ux5a/bLVE4Ncoh5xQjNMpplJEKl g8FjtPwGnTJ9VPlCWd8WAAa5DhVdyVE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-269-DNPjLMOqM3-blp0lhNFq-g-1; Mon, 25 Sep 2023 08:03:31 -0400 X-MC-Unique: DNPjLMOqM3-blp0lhNFq-g-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EB143101AA42; Mon, 25 Sep 2023 12:03:30 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4D914711293; Mon, 25 Sep 2023 12:03:29 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 05/12] iov_iter: Renumber ITER_* constants Date: Mon, 25 Sep 2023 13:03:02 +0100 Message-ID: <20230925120309.1731676-6-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Queue-Id: C8F0D8000E X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: bc5fwu1inp8z3hqbzuuccpeb9e3jhy3u X-HE-Tag: 1695643415-53081 X-HE-Meta: U2FsdGVkX184aRVaQWA5gzFRxPHlWMzc3QgdPjLFOZwoLiV2Wy3m2gFJ577UtZu8ws2E1ykR5VsVMseR9uK1xX7/PREj1BZv/11r5omxyonaH8Lq6zDYYWA/Q5Ce7rfuQkOeYjNVkuOLSm+F+n3cPiXGj05XDNCJVM97RfLNbLNCHr8680ssOiuXwhB1cTzT8Caw8mc38KRmIxb7kAXfrsr50ZLwS1NWiQab82PE9UOmzQCMOgptbMs07pVnv4k4tLGHuEgON2ncBGyYblAAvuA7OpBDW6rggIO7MCj19EtTf1hJkINrvbaqpsv5R8pbHjmYmRc8iFVqrxhLM936AkoqMclfeyJ/2QixXXRTjwglwCL1FneqyZNl1KVJSk4i2a3wnwmylYvduS5xAOCgdeIWAuQZYFm/6vwI+TV6PBetFd1iwHtmnVY4/+D12RxYfTTP0oibTqBdqi3AtfIhdHRnb874lPe8SOsqMh9ehmau/NSplUJIBNRyYcrIFvSI8zE9nuEDk9oJPHQYfjnZls6dUroWBD5xgoIlpGODrEuESU5qM6Y6y1jfsS9OMPNr+/dz9sM8kZASxFxtqlQh/EUKz8gvA6quFyBpnsoZsdw+OQbl7eqkQozetRHAksPimUI5ZoXuuizrmBLwodVeXqS2ehhe5X/HRoX8kNQZMLiJvlDNOt8vQ1wNGAgSiHXGcmlxRKMoeYFqLynrt5U6Juq12SgJivwq+EI9KxJBfmrRLiiAUNPzO+I6KMr90XsM1xatB2ThiNdt04gGmZsp9DuNsY0OMO09/JgXL1GghgVulPKHk0L2omV7VSXT0JNdIWfO/tmema+a0Ow7mlB06kplX/H7AFort6/uS+uwE7aeWw6oHa7mc8GxoU86KmTUpTIRXFN0qxmm9nLehQZ2gWvl8emuW/ROdiYRoFEX2LwnAxSBt1LusY1XEzAiUMizR5NBZMsNJ2LjrgrSYno Uzvb3zEW zRZRxO5EQWz2xOMDzsEc4KZzv0IHyDT1UPdgoGV8QcGhJOsoXfe60DhH4XSbB05VEE+LDgttX38DRJESHRWTFGU7Ta6dZBnxQJ3rQQ5OFukKUSCB/+AaNfSLu/dNim44vtKMeC7/cJWUeK6z45iNRdWCqz4LT98C2eACmCMmgLrRh1mWya/y9GxIQBMmOCHauKMwNSVd/Rj+fiSEe/ndpNQNAR2kKoc5HyrWYq5AK+APXhbcy9IBSBn/7dGOxRR8vcTL+bs9WsMggaM59TxuGx83EAh5h7BgD0v5TYcnpPuU5SsZV7Ud8yBTDxu2FqtP0efpswN/89dV/jtIp9Y1Ar8O0LtRJ984GzuXJqadnVO3AW1hagfMVFdPXfujvmzm9tbUV72Y9boE3hjr3lMGE+2qR2bo/c/G4+q0ZKbrD8A9KciKOL2ViXN8xHgp3Hvwaut4ejuEexuJDOWCWSyLP9ZT6v9a6g9jbUdtKKg9nD5Hg+JaSKyCj7mgVmYIIV3PoOXf5iJrWJ6t1HgnMe8OJIUXSQo2L+JECJchX3aW7bNSGQzsgFnsyzptwLW4SltFalRgAUjvWUd+Jlxo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Renumber the ITER_* iterator-type constants to put things in the same order as in the iteration functions and to group user-backed iterators at the bottom. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- include/linux/uio.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 2000e42a6586..bef8e56aa45c 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -21,12 +21,12 @@ struct kvec { enum iter_type { /* iter types */ + ITER_UBUF, ITER_IOVEC, - ITER_KVEC, ITER_BVEC, + ITER_KVEC, ITER_XARRAY, ITER_DISCARD, - ITER_UBUF, }; #define ITER_SOURCE 1 // == WRITE From patchwork Mon Sep 25 12:03:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 294FBCE7A94 for ; Mon, 25 Sep 2023 12:03:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A8CC6B0143; Mon, 25 Sep 2023 08:03:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5338A6B0144; Mon, 25 Sep 2023 08:03:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 314A56B0148; Mon, 25 Sep 2023 08:03:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 176746B0143 for ; Mon, 25 Sep 2023 08:03:40 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D817CC0C61 for ; Mon, 25 Sep 2023 12:03:39 +0000 (UTC) X-FDA: 81274985358.21.A477BF8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 12575140022 for ; Mon, 25 Sep 2023 12:03:37 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gDctC5YI; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643418; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4obcSyEH8f0LI+hqbulIObwIk6RPpIYigX0Ml5LHEtM=; b=1/X4SkXNgQMJPYrpZzb356oLxFlpU76LOvh18bmuXLldThTRSCBkQCDyfqFf3PPMyb2PDs fb8KvY0AM/8jm1YrQN0EHTul/ZxQSuxsW9RDA/8F1lyFJIKcQ+nnkHWwr8JhTYgXmigdgZ FA5MmuKr/v/XKi9/99uNBShEQKd4kBo= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gDctC5YI; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643418; a=rsa-sha256; cv=none; b=hQxYqnBudnfdunu199Zccn4Wt8Ao2ObFe2uRH29qlqeZ0t30ipHNL0Nf7KhQGyHIqDjI81 fKO+BzjO1kXurhAZc6rXCakjm7WLEjqNL+5QWNNG2CH/nFbP8ivhA2CsAEPkDm7zURxXE4 ngZtLg1psoVlg/NPdwAUTXfGqSiE06s= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643417; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4obcSyEH8f0LI+hqbulIObwIk6RPpIYigX0Ml5LHEtM=; b=gDctC5YITbxq4Yuw31SKhxc0+ZoejdwlUZiw8izb1s/XLjKxDsNeNuSVm6vG/gNvGXOOHZ 1JY7iNjGvrl+NQQ94HJj/wiykyHngJDf32RGQ5yH5DvcpjH9VPeAk76DFh0ys/sYk+hR0K pt+idm3O3PHC/2HgVtUF34YKWx3WFhM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-136-cDj4weHpMsW5nB2aWNtwwg-1; Mon, 25 Sep 2023 08:03:34 -0400 X-MC-Unique: cDj4weHpMsW5nB2aWNtwwg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4F33E800B35; Mon, 25 Sep 2023 12:03:33 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id A8F8551E3; Mon, 25 Sep 2023 12:03:31 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 06/12] iov_iter: Derive user-backedness from the iterator type Date: Mon, 25 Sep 2023 13:03:03 +0100 Message-ID: <20230925120309.1731676-7-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Queue-Id: 12575140022 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: xwj1omjtfq9pdi1tuchgdun8tdzjwdt6 X-HE-Tag: 1695643417-532599 X-HE-Meta: U2FsdGVkX19lLzLx/xlu4eY9ADIvYiWweccElIg7ghN+A6Qlzz9lht1tQOq4H6ye3dD1WTlNwxohH1XPs/5L/2SKegmV8dAFS9f6xdIsycUJMtuWzxq6hlGYOi6Obij67wCmBDjJtsiS1WamJvbqHpLHiS878EuWIVX3npXta/AGQeZ/QGZ5GZKJWCegKEykPo6lUdbye3H2J4DG2c8xSexhTAyq4IDuZbyVWYdsaakRAG3Ukf7D1nyJPHK5L8PYPUhyMQ2KHlR+8w4quglOlA1fG1FVpYznIGor4MExqg6FWkWIqzCHgi3KVQu+RgR1U4xa3ib5EZt1FYmXJHdS1gnqHFPai4kFGRuf3J+kzY7UNsQ7z2776DXb1e9P2cZovjzXIjse2tZIdAVqMFzMeu0949GXJex3YfcTm6VagHjYdhhSimd4Dla7yh9vDey8lhsG2q2P3p/hLhgqFoWhT+6t2vOJKuzyo+xf7/Eo6kSI52nA+y0zg4PkfWIbtmo52ufVc+vqwsCOqLS/+TXtC9YsFm1dd3x+VVhw6+B9nAxDdMdG4bv3YlPpc4pNyKrCGRzquj/kYI/RoIwLD647+fl+JhZW5eTRB/sN3czZmjYV7zQuDPtgGpIzEED+wZk35Wb3rj2zowq9mZ2wzOUN34Bl1uCgJKLarDVsmi6niy3qQ7m4UOjdG9vkLLvm3naHDqaqFUPRyqDRJrtRlk3nh+RSkBlICQexpqcPyk77DYgkZK5rEOoJ6X1wnoAkqS70s9JAFhpo3Bw8feANoOLWhQlT1mi4t4J65fvGpCgMfBEJuZjz5tDgrpFZR2UgcWvZFqUKYMtrEhvcw90Xbyr2zMEm6LtOJPgLXmXI0LhLsytatAmLfXrySRsHmqMtja5lw7G9xDWTr+OL2OICXoHlrNtw6uiB4Na1QEG+zGkRy34yktTUh2QAtXarm14H7ptjSWyRGofeCK+rMs3C2Bs 4i10rahy Y/9vHTDTl0IqzTNSZQlSRpqgMNzi+HliWjwTDvyH3qYppqZGY0DsFhRh9EgdSkDr3VsWdlB3lUg+/VUY8dIGIKNSHlSH7tUvT6JNoYfVApDBZL5u6hlg4sJJgGInV1LkSMROWv9Hnzcpu7oUz9/gdj77UlcCBYpBAjO6JQUtItdCEITY+K9OnLkjuE6JwctMRM6skyQ35WEjo9IbxvExrDZIZFEpE6w4AxLaZRtIKGlBiUaNp5xBsVj0cPMCakrJH2Fq6QNw5JUcD82KXdPJ4bbknGyxyu46sYJ/bvlo9bFuvKD1dps2+iF1nMllBaLvHlXEmT5zEOyLwRIaYCsLCJqUlr1CWnGXs4t+tx5GaOGSvu/7R3ys2CDsQDyqr4wOIv+4aTD5PoccS2GwB1zfn6SxthLSd84Q3QbAzxzv4Db3rK3N0Wa2xOS1FO6XVKvXJ5KcFmWSFtRQNTDacwuK1kFPWwMv3+SGsgsezyMv1yVkzVi7dWYHn1qPRY9pacuh3MtpqxIQED2GcYLS0FANCS9pCoo4cJ528hanZdmPvjoKLJpdIdEDMhoLnsxDs2uPrYund2ozwDIO5zC8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the iterator type to determine whether an iterator is user-backed or not rather than using a special flag for it. Now that ITER_UBUF and ITER_IOVEC are 0 and 1, they can be checked with a single comparison. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- include/linux/uio.h | 4 +--- lib/iov_iter.c | 1 - 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index bef8e56aa45c..65d9143f83c8 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -43,7 +43,6 @@ struct iov_iter { bool copy_mc; bool nofault; bool data_source; - bool user_backed; size_t iov_offset; /* * Hack alert: overlay ubuf_iovec with iovec + count, so @@ -140,7 +139,7 @@ static inline unsigned char iov_iter_rw(const struct iov_iter *i) static inline bool user_backed_iter(const struct iov_iter *i) { - return i->user_backed; + return iter_is_ubuf(i) || iter_is_iovec(i); } /* @@ -380,7 +379,6 @@ static inline void iov_iter_ubuf(struct iov_iter *i, unsigned int direction, *i = (struct iov_iter) { .iter_type = ITER_UBUF, .copy_mc = false, - .user_backed = true, .data_source = direction, .ubuf = buf, .count = count, diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 27234a820eeb..227c9f536b94 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -290,7 +290,6 @@ void iov_iter_init(struct iov_iter *i, unsigned int direction, .iter_type = ITER_IOVEC, .copy_mc = false, .nofault = false, - .user_backed = true, .data_source = direction, .__iov = iov, .nr_segs = nr_segs, From patchwork Mon Sep 25 12:03:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DAEACE7A95 for ; Mon, 25 Sep 2023 12:03:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4224F6B014B; Mon, 25 Sep 2023 08:03:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3ABDB6B014C; Mon, 25 Sep 2023 08:03:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 161ED6B0150; Mon, 25 Sep 2023 08:03:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F1C956B014B for ; Mon, 25 Sep 2023 08:03:42 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B0ECF1C9744 for ; Mon, 25 Sep 2023 12:03:42 +0000 (UTC) X-FDA: 81274985484.20.1E4A153 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf02.hostedemail.com (Postfix) with ESMTP id DF9EA80023 for ; Mon, 25 Sep 2023 12:03:40 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Ia5zrAFF; spf=pass (imf02.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643421; a=rsa-sha256; cv=none; b=sdhHY5NsgEvaiTbbjoNj9fivSVwa/xZEATYPwdqRMH4YD0eIaxqsUaFdjyGE8bLg6XrX/1 JcN8mogngg8y3FoQFA3A9xYuAjZQ9SJenGbmf+YDxKBKuU66BdivfgnYVwFMvxZgP/YdGG Bc2Giy35aEeoVkuBXSLnJr8Vs4Z3dPg= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Ia5zrAFF; spf=pass (imf02.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643421; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=saXGiomC5bnv9RAKI0XpVlyY0MbcRBlM5OC3bKVXk+c=; b=uzVbJ5h7/QNRDIsoSt5fFrgeOH2JIelr4/vHzRP0/Gf+4ZtkmRhvhgJWQubbyiQl4FzsuP 4iewD2+BycXWOvyB0RKmK6th0Vep6MQC9v+N756zOHOTLYHovkJNMqnOMzJfF0wqx+D3go QEi91Z6CYU1D2TuBkXk83KMbd0eMHCk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643420; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=saXGiomC5bnv9RAKI0XpVlyY0MbcRBlM5OC3bKVXk+c=; b=Ia5zrAFFjSCoB0ZFYrtZUmvb1K2Cz0/680IQPbITjYh+GA5L/4wv0LGUFI2/wBKYNnDAd2 Do8upoCXvaMRSELJ1kZU4uFJ2gr9PygM/VD92/hZfVEZ4o5uejx1kj/NQNRz9OV+PRWmXf NxGpiQ1atW6W7fdCc8eZQiwujtJGNxM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-624-5kZIkTjKNJaXPLuiWr8tCQ-1; Mon, 25 Sep 2023 08:03:36 -0400 X-MC-Unique: 5kZIkTjKNJaXPLuiWr8tCQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C196D18312CF; Mon, 25 Sep 2023 12:03:35 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id E01C240C6EA8; Mon, 25 Sep 2023 12:03:33 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 07/12] iov_iter: Convert iterate*() to inline funcs Date: Mon, 25 Sep 2023 13:03:04 +0100 Message-ID: <20230925120309.1731676-8-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: DF9EA80023 X-Stat-Signature: emjj767jp6ep7igxtkidizeodi7d7hpw X-Rspam-User: X-HE-Tag: 1695643420-96616 X-HE-Meta: U2FsdGVkX19VuW4rErjmti+A/XffbqcvlY8pCFAssQ7kLTAC3iJlg2/H/OqRn0bx/XwAUcYgZvnX4ZSC+nL/u7bQEYTaEol7mbUJXd7BF/2WaUhLfz2Rqi8jcPAO9oN895wGN+UlCIpZLbDi//jaF0GWlw/tyCjobZH+fgvTrBf4LCwnn3AuF8V9XO7UOaFp89a0yn+wkaJgZHeNzmI7yIakLhUo2Tp5l0/9XUzuASKTLVee08pmR8SCQlOPGYx5qz82xjVJDTWHmk++Z1kEYHqY0kZ8G0hdzFnLeEO7g5k90A0SKesVdAzkxrCGGUeYNOfaUifsUpWxVH1omaZwPPYee3h90SIZHuDTggbwBIR12k0ruoIja9ozzAHx6GYN+nPMBJk+xpOwazv2TtcnoW5pOScnkUtxH56qM0MWqr5cz3PN8beVQcu/JVkOE48cj0zyYKU7X5ary0vvkA5tT1XCe5SX77BGORL3Wgl8ot4GQ6V+m4zH+DIJTsydCpU/uHlcvizLrVncLB/uIk8elBpAuvJkZN1hcgHeZ4IZ9/RIpGU6+nlqvFEObEKazvP+s+SDQA+0yg5tH8QJU5kNt44gIuL9AY4E1w0XEL5w1Ce3a0JXGcRkMXrDmH/zACVZ3qcHsmwh02VRKTP6AEaTdwMkGmfisRi85DD7yWr1Ln5D3fA/xiRz5MGgH7s1it7Q7zDLhF0COOXuxYW0Nlhs150mtnOQN7cPtGHZmygW9CpPryYhxc+CeajJ+Hp6zBzbgjNx+l5vHF9T4QDzMxKHBDNsBJ1iLE2xAS+Nmp2BQ5AGG+9DUuBk9lUWSYB2vI4K9/vPm5MzaYrxh0FSbb0vp92qp+qpxFnN/9fA/L2Hu+1Ni4Dfw5Pn/TObI8bPQeFBcqXyll8gATOsOVD9v8OOHNGPQTISH1aZm++2hs03QMWOZQytSt20wAG6/nZBEvJnGlaGjF0W+eWstpsTnQL /JKrLraF WgRh4xETLgTnK9/P0X5cADjWU70cIQeiV7FOEPQ8JImuhQiNvwaqupUA6/HP3Zffk10bSb/ZsOuG+/zC+fgiADRD6n3v2nzo3Vj3YjBwBJ/0uBcxoOkIlzw4s+yigSxw+VA3d8IThjFQXzKOtH34qxv/fE5z3NLhKfvBhEXQQ6PZSioDMy6c9imA+0VgnOLuX1U41ZSkj3EbZcjXNHHVH2YIq21SLfnNbNyM+275K9khEy46ySDhouyvkoU2gpI4eFgUOapcy0AbSX1pq5Ij1SKbMfg3sEXVviO1azaYm18sSO9G/DBC9qHZqzlTMpmIY4h2Fkl9zrDlr4So0+rMcezCe8KwL4yi675ARV9PzprzwHcWWeVxtN4j75R0FWliRR5APQChHdsHfocCWjj4nS1i9Anrzpq/lJZ6ygp4WVSDmrEQSw50TG2Iodvrf+OyuXVzsEWCBHiaCdIYsRO/kAEMXeKoxDSZkieMwrR4xOB8yfOO0ToaF8ZsMynlN8QMm8v4KdOY4AVmtnGGIoOmmBvLhEOdtWvcGL2qI/tei1aD8nuZsEKljwu/eMuXN05CZmb46WrdFvrM/hwOfJ3suqOJ9juPKMe2mpWkGVfxCQQCNpE355BmcY50luyyUZOGfH71ma9MdKquWxNTpgiY7lX+DyBMKaR4rnfM4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert the iov_iter iteration macros to inline functions to make the code easier to follow. The functions are marked __always_inline as we don't want to end up with indirect calls in the code. This, however, leaves dealing with ->copy_mc in an awkard situation since the step function (memcpy_from_iter_mc()) needs to test the flag in the iterator, but isn't passed the iterator. This will be dealt with in a follow-up patch. The variable names in the per-type iterator functions have been harmonised as much as possible and made clearer as to the variable purpose. The iterator functions are also moved to a header file so that other operations that need to scan over an iterator can be added. For instance, the rbd driver could use this to scan a buffer to see if it is all zeros and libceph could use this to generate a crc. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/3710261.1691764329@warthog.procyon.org.uk/ # v1 Link: https://lore.kernel.org/r/855.1692047347@warthog.procyon.org.uk/ # v2 Link: https://lore.kernel.org/r/20230816120741.534415-1-dhowells@redhat.com/ # v3 Reviewed-by: David Howells Tested-by: David Howells --- Notes: Changes ======= ver #5) - Merge in patch to move iteration framework to a header file. - Move "iter->count - progress" into individual iteration subfunctions. include/linux/iov_iter.h | 274 ++++++++++++++++++++++++++ lib/iov_iter.c | 416 ++++++++++++++++----------------------- 2 files changed, 449 insertions(+), 241 deletions(-) create mode 100644 include/linux/iov_iter.h diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h new file mode 100644 index 000000000000..270454a6703d --- /dev/null +++ b/include/linux/iov_iter.h @@ -0,0 +1,274 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* I/O iterator iteration building functions. + * + * Copyright (C) 2023 Red Hat, Inc. All Rights Reserved. + * Written by David Howells (dhowells@redhat.com) + */ + +#ifndef _LINUX_IOV_ITER_H +#define _LINUX_IOV_ITER_H + +#include +#include + +typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len, + void *priv, void *priv2); +typedef size_t (*iov_ustep_f)(void __user *iter_base, size_t progress, size_t len, + void *priv, void *priv2); + +/* + * Handle ITER_UBUF. + */ +static __always_inline +size_t iterate_ubuf(struct iov_iter *iter, size_t len, void *priv, void *priv2, + iov_ustep_f step) +{ + void __user *base = iter->ubuf; + size_t progress = 0, remain; + + remain = step(base + iter->iov_offset, 0, len, priv, priv2); + progress = len - remain; + iter->iov_offset += progress; + iter->count -= progress; + return progress; +} + +/* + * Handle ITER_IOVEC. + */ +static __always_inline +size_t iterate_iovec(struct iov_iter *iter, size_t len, void *priv, void *priv2, + iov_ustep_f step) +{ + const struct iovec *p = iter->__iov; + size_t progress = 0, skip = iter->iov_offset; + + do { + size_t remain, consumed; + size_t part = min(len, p->iov_len - skip); + + if (likely(part)) { + remain = step(p->iov_base + skip, progress, part, priv, priv2); + consumed = part - remain; + progress += consumed; + skip += consumed; + len -= consumed; + if (skip < p->iov_len) + break; + } + p++; + skip = 0; + } while (len); + + iter->nr_segs -= p - iter->__iov; + iter->__iov = p; + iter->iov_offset = skip; + iter->count -= progress; + return progress; +} + +/* + * Handle ITER_KVEC. + */ +static __always_inline +size_t iterate_kvec(struct iov_iter *iter, size_t len, void *priv, void *priv2, + iov_step_f step) +{ + const struct kvec *p = iter->kvec; + size_t progress = 0, skip = iter->iov_offset; + + do { + size_t remain, consumed; + size_t part = min(len, p->iov_len - skip); + + if (likely(part)) { + remain = step(p->iov_base + skip, progress, part, priv, priv2); + consumed = part - remain; + progress += consumed; + skip += consumed; + len -= consumed; + if (skip < p->iov_len) + break; + } + p++; + skip = 0; + } while (len); + + iter->nr_segs -= p - iter->kvec; + iter->kvec = p; + iter->iov_offset = skip; + iter->count -= progress; + return progress; +} + +/* + * Handle ITER_BVEC. + */ +static __always_inline +size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2, + iov_step_f step) +{ + const struct bio_vec *p = iter->bvec; + size_t progress = 0, skip = iter->iov_offset; + + do { + size_t remain, consumed; + size_t offset = p->bv_offset + skip, part; + void *kaddr = kmap_local_page(p->bv_page + offset / PAGE_SIZE); + + part = min3(len, + (size_t)(p->bv_len - skip), + (size_t)(PAGE_SIZE - offset % PAGE_SIZE)); + remain = step(kaddr + offset % PAGE_SIZE, progress, part, priv, priv2); + kunmap_local(kaddr); + consumed = part - remain; + len -= consumed; + progress += consumed; + skip += consumed; + if (skip >= p->bv_len) { + skip = 0; + p++; + } + if (remain) + break; + } while (len); + + iter->nr_segs -= p - iter->bvec; + iter->bvec = p; + iter->iov_offset = skip; + iter->count -= progress; + return progress; +} + +/* + * Handle ITER_XARRAY. + */ +static __always_inline +size_t iterate_xarray(struct iov_iter *iter, size_t len, void *priv, void *priv2, + iov_step_f step) +{ + struct folio *folio; + size_t progress = 0; + loff_t start = iter->xarray_start + iter->iov_offset; + pgoff_t index = start / PAGE_SIZE; + XA_STATE(xas, iter->xarray, index); + + rcu_read_lock(); + xas_for_each(&xas, folio, ULONG_MAX) { + size_t remain, consumed, offset, part, flen; + + if (xas_retry(&xas, folio)) + continue; + if (WARN_ON(xa_is_value(folio))) + break; + if (WARN_ON(folio_test_hugetlb(folio))) + break; + + offset = offset_in_folio(folio, start + progress); + flen = min(folio_size(folio) - offset, len); + + while (flen) { + void *base = kmap_local_folio(folio, offset); + + part = min_t(size_t, flen, + PAGE_SIZE - offset_in_page(offset)); + remain = step(base, progress, part, priv, priv2); + kunmap_local(base); + + consumed = part - remain; + progress += consumed; + len -= consumed; + + if (remain || len == 0) + goto out; + flen -= consumed; + offset += consumed; + } + } + +out: + rcu_read_unlock(); + iter->iov_offset += progress; + iter->count -= progress; + return progress; +} + +/* + * Handle ITER_DISCARD. + */ +static __always_inline +size_t iterate_discard(struct iov_iter *iter, size_t len, void *priv, void *priv2, + iov_step_f step) +{ + size_t progress = len; + + iter->count -= progress; + return progress; +} + +/** + * iterate_and_advance2 - Iterate over an iterator + * @iter: The iterator to iterate over. + * @len: The amount to iterate over. + * @priv: Data for the step functions. + * @priv2: More data for the step functions. + * @ustep: Function for UBUF/IOVEC iterators; given __user addresses. + * @step: Function for other iterators; given kernel addresses. + * + * Iterate over the next part of an iterator, up to the specified length. The + * buffer is presented in segments, which for kernel iteration are broken up by + * physical pages and mapped, with the mapped address being presented. + * + * Two step functions, @step and @ustep, must be provided, one for handling + * mapped kernel addresses and the other is given user addresses which have the + * potential to fault since no pinning is performed. + * + * The step functions are passed the address and length of the segment, @priv, + * @priv2 and the amount of data so far iterated over (which can, for example, + * be added to @priv to point to the right part of a second buffer). The step + * functions should return the amount of the segment they didn't process (ie. 0 + * indicates complete processsing). + * + * This function returns the amount of data processed (ie. 0 means nothing was + * processed and the value of @len means processes to completion). + */ +static __always_inline +size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv, + void *priv2, iov_ustep_f ustep, iov_step_f step) +{ + if (unlikely(iter->count < len)) + len = iter->count; + if (unlikely(!len)) + return 0; + + if (likely(iter_is_ubuf(iter))) + return iterate_ubuf(iter, len, priv, priv2, ustep); + if (likely(iter_is_iovec(iter))) + return iterate_iovec(iter, len, priv, priv2, ustep); + if (iov_iter_is_bvec(iter)) + return iterate_bvec(iter, len, priv, priv2, step); + if (iov_iter_is_kvec(iter)) + return iterate_kvec(iter, len, priv, priv2, step); + if (iov_iter_is_xarray(iter)) + return iterate_xarray(iter, len, priv, priv2, step); + return iterate_discard(iter, len, priv, priv2, step); +} + +/** + * iterate_and_advance - Iterate over an iterator + * @iter: The iterator to iterate over. + * @len: The amount to iterate over. + * @priv: Data for the step functions. + * @ustep: Function for UBUF/IOVEC iterators; given __user addresses. + * @step: Function for other iterators; given kernel addresses. + * + * As iterate_and_advance2(), but priv2 is always NULL. + */ +static __always_inline +size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv, + iov_ustep_f ustep, iov_step_f step) +{ + return iterate_and_advance2(iter, len, priv, NULL, ustep, step); +} + +#endif /* _LINUX_IOV_ITER_H */ diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 227c9f536b94..65374ee91ecd 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -13,189 +13,69 @@ #include #include #include +#include -/* covers ubuf and kbuf alike */ -#define iterate_buf(i, n, base, len, off, __p, STEP) { \ - size_t __maybe_unused off = 0; \ - len = n; \ - base = __p + i->iov_offset; \ - len -= (STEP); \ - i->iov_offset += len; \ - n = len; \ -} - -/* covers iovec and kvec alike */ -#define iterate_iovec(i, n, base, len, off, __p, STEP) { \ - size_t off = 0; \ - size_t skip = i->iov_offset; \ - do { \ - len = min(n, __p->iov_len - skip); \ - if (likely(len)) { \ - base = __p->iov_base + skip; \ - len -= (STEP); \ - off += len; \ - skip += len; \ - n -= len; \ - if (skip < __p->iov_len) \ - break; \ - } \ - __p++; \ - skip = 0; \ - } while (n); \ - i->iov_offset = skip; \ - n = off; \ -} - -#define iterate_bvec(i, n, base, len, off, p, STEP) { \ - size_t off = 0; \ - unsigned skip = i->iov_offset; \ - while (n) { \ - unsigned offset = p->bv_offset + skip; \ - unsigned left; \ - void *kaddr = kmap_local_page(p->bv_page + \ - offset / PAGE_SIZE); \ - base = kaddr + offset % PAGE_SIZE; \ - len = min(min(n, (size_t)(p->bv_len - skip)), \ - (size_t)(PAGE_SIZE - offset % PAGE_SIZE)); \ - left = (STEP); \ - kunmap_local(kaddr); \ - len -= left; \ - off += len; \ - skip += len; \ - if (skip == p->bv_len) { \ - skip = 0; \ - p++; \ - } \ - n -= len; \ - if (left) \ - break; \ - } \ - i->iov_offset = skip; \ - n = off; \ -} - -#define iterate_xarray(i, n, base, len, __off, STEP) { \ - __label__ __out; \ - size_t __off = 0; \ - struct folio *folio; \ - loff_t start = i->xarray_start + i->iov_offset; \ - pgoff_t index = start / PAGE_SIZE; \ - XA_STATE(xas, i->xarray, index); \ - \ - len = PAGE_SIZE - offset_in_page(start); \ - rcu_read_lock(); \ - xas_for_each(&xas, folio, ULONG_MAX) { \ - unsigned left; \ - size_t offset; \ - if (xas_retry(&xas, folio)) \ - continue; \ - if (WARN_ON(xa_is_value(folio))) \ - break; \ - if (WARN_ON(folio_test_hugetlb(folio))) \ - break; \ - offset = offset_in_folio(folio, start + __off); \ - while (offset < folio_size(folio)) { \ - base = kmap_local_folio(folio, offset); \ - len = min(n, len); \ - left = (STEP); \ - kunmap_local(base); \ - len -= left; \ - __off += len; \ - n -= len; \ - if (left || n == 0) \ - goto __out; \ - offset += len; \ - len = PAGE_SIZE; \ - } \ - } \ -__out: \ - rcu_read_unlock(); \ - i->iov_offset += __off; \ - n = __off; \ -} - -#define __iterate_and_advance(i, n, base, len, off, I, K) { \ - if (unlikely(i->count < n)) \ - n = i->count; \ - if (likely(n)) { \ - if (likely(iter_is_ubuf(i))) { \ - void __user *base; \ - size_t len; \ - iterate_buf(i, n, base, len, off, \ - i->ubuf, (I)) \ - } else if (likely(iter_is_iovec(i))) { \ - const struct iovec *iov = iter_iov(i); \ - void __user *base; \ - size_t len; \ - iterate_iovec(i, n, base, len, off, \ - iov, (I)) \ - i->nr_segs -= iov - iter_iov(i); \ - i->__iov = iov; \ - } else if (iov_iter_is_bvec(i)) { \ - const struct bio_vec *bvec = i->bvec; \ - void *base; \ - size_t len; \ - iterate_bvec(i, n, base, len, off, \ - bvec, (K)) \ - i->nr_segs -= bvec - i->bvec; \ - i->bvec = bvec; \ - } else if (iov_iter_is_kvec(i)) { \ - const struct kvec *kvec = i->kvec; \ - void *base; \ - size_t len; \ - iterate_iovec(i, n, base, len, off, \ - kvec, (K)) \ - i->nr_segs -= kvec - i->kvec; \ - i->kvec = kvec; \ - } else if (iov_iter_is_xarray(i)) { \ - void *base; \ - size_t len; \ - iterate_xarray(i, n, base, len, off, \ - (K)) \ - } \ - i->count -= n; \ - } \ -} -#define iterate_and_advance(i, n, base, len, off, I, K) \ - __iterate_and_advance(i, n, base, len, off, I, ((void)(K),0)) - -static int copyout(void __user *to, const void *from, size_t n) +static __always_inline +size_t copy_to_user_iter(void __user *iter_to, size_t progress, + size_t len, void *from, void *priv2) { if (should_fail_usercopy()) - return n; - if (access_ok(to, n)) { - instrument_copy_to_user(to, from, n); - n = raw_copy_to_user(to, from, n); + return len; + if (access_ok(iter_to, len)) { + from += progress; + instrument_copy_to_user(iter_to, from, len); + len = raw_copy_to_user(iter_to, from, len); } - return n; + return len; } -static int copyout_nofault(void __user *to, const void *from, size_t n) +static __always_inline +size_t copy_to_user_iter_nofault(void __user *iter_to, size_t progress, + size_t len, void *from, void *priv2) { - long res; + ssize_t res; if (should_fail_usercopy()) - return n; - - res = copy_to_user_nofault(to, from, n); + return len; - return res < 0 ? n : res; + from += progress; + res = copy_to_user_nofault(iter_to, from, len); + return res < 0 ? len : res; } -static int copyin(void *to, const void __user *from, size_t n) +static __always_inline +size_t copy_from_user_iter(void __user *iter_from, size_t progress, + size_t len, void *to, void *priv2) { - size_t res = n; + size_t res = len; if (should_fail_usercopy()) - return n; - if (access_ok(from, n)) { - instrument_copy_from_user_before(to, from, n); - res = raw_copy_from_user(to, from, n); - instrument_copy_from_user_after(to, from, n, res); + return len; + if (access_ok(iter_from, len)) { + to += progress; + instrument_copy_from_user_before(to, iter_from, len); + res = raw_copy_from_user(to, iter_from, len); + instrument_copy_from_user_after(to, iter_from, len, res); } return res; } +static __always_inline +size_t memcpy_to_iter(void *iter_to, size_t progress, + size_t len, void *from, void *priv2) +{ + memcpy(iter_to, from + progress, len); + return 0; +} + +static __always_inline +size_t memcpy_from_iter(void *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + memcpy(to + progress, iter_from, len); + return 0; +} + /* * fault_in_iov_iter_readable - fault in iov iterator for reading * @i: iterator @@ -312,23 +192,29 @@ size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) return 0; if (user_backed_iter(i)) might_fault(); - iterate_and_advance(i, bytes, base, len, off, - copyout(base, addr + off, len), - memcpy(base, addr + off, len) - ) - - return bytes; + return iterate_and_advance(i, bytes, (void *)addr, + copy_to_user_iter, memcpy_to_iter); } EXPORT_SYMBOL(_copy_to_iter); #ifdef CONFIG_ARCH_HAS_COPY_MC -static int copyout_mc(void __user *to, const void *from, size_t n) -{ - if (access_ok(to, n)) { - instrument_copy_to_user(to, from, n); - n = copy_mc_to_user((__force void *) to, from, n); +static __always_inline +size_t copy_to_user_iter_mc(void __user *iter_to, size_t progress, + size_t len, void *from, void *priv2) +{ + if (access_ok(iter_to, len)) { + from += progress; + instrument_copy_to_user(iter_to, from, len); + len = copy_mc_to_user(iter_to, from, len); } - return n; + return len; +} + +static __always_inline +size_t memcpy_to_iter_mc(void *iter_to, size_t progress, + size_t len, void *from, void *priv2) +{ + return copy_mc_to_kernel(iter_to, from + progress, len); } /** @@ -361,22 +247,20 @@ size_t _copy_mc_to_iter(const void *addr, size_t bytes, struct iov_iter *i) return 0; if (user_backed_iter(i)) might_fault(); - __iterate_and_advance(i, bytes, base, len, off, - copyout_mc(base, addr + off, len), - copy_mc_to_kernel(base, addr + off, len) - ) - - return bytes; + return iterate_and_advance(i, bytes, (void *)addr, + copy_to_user_iter_mc, memcpy_to_iter_mc); } EXPORT_SYMBOL_GPL(_copy_mc_to_iter); #endif /* CONFIG_ARCH_HAS_COPY_MC */ -static void *memcpy_from_iter(struct iov_iter *i, void *to, const void *from, - size_t size) +static size_t memcpy_from_iter_mc(void *iter_from, size_t progress, + size_t len, void *to, void *priv2) { - if (iov_iter_is_copy_mc(i)) - return (void *)copy_mc_to_kernel(to, from, size); - return memcpy(to, from, size); + struct iov_iter *iter = priv2; + + if (iov_iter_is_copy_mc(iter)) + return copy_mc_to_kernel(to + progress, iter_from, len); + return memcpy_from_iter(iter_from, progress, len, to, priv2); } size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i) @@ -386,30 +270,46 @@ size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i) if (user_backed_iter(i)) might_fault(); - iterate_and_advance(i, bytes, base, len, off, - copyin(addr + off, base, len), - memcpy_from_iter(i, addr + off, base, len) - ) - - return bytes; + return iterate_and_advance2(i, bytes, addr, i, + copy_from_user_iter, + memcpy_from_iter_mc); } EXPORT_SYMBOL(_copy_from_iter); +static __always_inline +size_t copy_from_user_iter_nocache(void __user *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + return __copy_from_user_inatomic_nocache(to + progress, iter_from, len); +} + size_t _copy_from_iter_nocache(void *addr, size_t bytes, struct iov_iter *i) { if (WARN_ON_ONCE(!i->data_source)) return 0; - iterate_and_advance(i, bytes, base, len, off, - __copy_from_user_inatomic_nocache(addr + off, base, len), - memcpy(addr + off, base, len) - ) - - return bytes; + return iterate_and_advance(i, bytes, addr, + copy_from_user_iter_nocache, + memcpy_from_iter); } EXPORT_SYMBOL(_copy_from_iter_nocache); #ifdef CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE +static __always_inline +size_t copy_from_user_iter_flushcache(void __user *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + return __copy_from_user_flushcache(to + progress, iter_from, len); +} + +static __always_inline +size_t memcpy_from_iter_flushcache(void *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + memcpy_flushcache(to + progress, iter_from, len); + return 0; +} + /** * _copy_from_iter_flushcache - write destination through cpu cache * @addr: destination kernel address @@ -431,12 +331,9 @@ size_t _copy_from_iter_flushcache(void *addr, size_t bytes, struct iov_iter *i) if (WARN_ON_ONCE(!i->data_source)) return 0; - iterate_and_advance(i, bytes, base, len, off, - __copy_from_user_flushcache(addr + off, base, len), - memcpy_flushcache(addr + off, base, len) - ) - - return bytes; + return iterate_and_advance(i, bytes, addr, + copy_from_user_iter_flushcache, + memcpy_from_iter_flushcache); } EXPORT_SYMBOL_GPL(_copy_from_iter_flushcache); #endif @@ -508,10 +405,9 @@ size_t copy_page_to_iter_nofault(struct page *page, unsigned offset, size_t byte void *kaddr = kmap_local_page(page); size_t n = min(bytes, (size_t)PAGE_SIZE - offset); - iterate_and_advance(i, n, base, len, off, - copyout_nofault(base, kaddr + offset + off, len), - memcpy(base, kaddr + offset + off, len) - ) + n = iterate_and_advance(i, bytes, kaddr, + copy_to_user_iter_nofault, + memcpy_to_iter); kunmap_local(kaddr); res += n; bytes -= n; @@ -554,14 +450,25 @@ size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes, } EXPORT_SYMBOL(copy_page_from_iter); -size_t iov_iter_zero(size_t bytes, struct iov_iter *i) +static __always_inline +size_t zero_to_user_iter(void __user *iter_to, size_t progress, + size_t len, void *priv, void *priv2) { - iterate_and_advance(i, bytes, base, len, count, - clear_user(base, len), - memset(base, 0, len) - ) + return clear_user(iter_to, len); +} - return bytes; +static __always_inline +size_t zero_to_iter(void *iter_to, size_t progress, + size_t len, void *priv, void *priv2) +{ + memset(iter_to, 0, len); + return 0; +} + +size_t iov_iter_zero(size_t bytes, struct iov_iter *i) +{ + return iterate_and_advance(i, bytes, NULL, + zero_to_user_iter, zero_to_iter); } EXPORT_SYMBOL(iov_iter_zero); @@ -586,10 +493,9 @@ size_t copy_page_from_iter_atomic(struct page *page, size_t offset, } p = kmap_atomic(page) + offset; - iterate_and_advance(i, n, base, len, off, - copyin(p + off, base, len), - memcpy_from_iter(i, p + off, base, len) - ) + n = iterate_and_advance2(i, n, p, i, + copy_from_user_iter, + memcpy_from_iter_mc); kunmap_atomic(p); copied += n; offset += n; @@ -1180,32 +1086,64 @@ ssize_t iov_iter_get_pages_alloc2(struct iov_iter *i, } EXPORT_SYMBOL(iov_iter_get_pages_alloc2); +static __always_inline +size_t copy_from_user_iter_csum(void __user *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + __wsum next, *csum = priv2; + + next = csum_and_copy_from_user(iter_from, to + progress, len); + *csum = csum_block_add(*csum, next, progress); + return next ? 0 : len; +} + +static __always_inline +size_t memcpy_from_iter_csum(void *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + __wsum *csum = priv2; + + *csum = csum_and_memcpy(to + progress, iter_from, len, *csum, progress); + return 0; +} + size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i) { - __wsum sum, next; - sum = *csum; if (WARN_ON_ONCE(!i->data_source)) return 0; - - iterate_and_advance(i, bytes, base, len, off, ({ - next = csum_and_copy_from_user(base, addr + off, len); - sum = csum_block_add(sum, next, off); - next ? 0 : len; - }), ({ - sum = csum_and_memcpy(addr + off, base, len, sum, off); - }) - ) - *csum = sum; - return bytes; + return iterate_and_advance2(i, bytes, addr, csum, + copy_from_user_iter_csum, + memcpy_from_iter_csum); } EXPORT_SYMBOL(csum_and_copy_from_iter); +static __always_inline +size_t copy_to_user_iter_csum(void __user *iter_to, size_t progress, + size_t len, void *from, void *priv2) +{ + __wsum next, *csum = priv2; + + next = csum_and_copy_to_user(from + progress, iter_to, len); + *csum = csum_block_add(*csum, next, progress); + return next ? 0 : len; +} + +static __always_inline +size_t memcpy_to_iter_csum(void *iter_to, size_t progress, + size_t len, void *from, void *priv2) +{ + __wsum *csum = priv2; + + *csum = csum_and_memcpy(iter_to, from + progress, len, *csum, progress); + return 0; +} + size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *_csstate, struct iov_iter *i) { struct csum_state *csstate = _csstate; - __wsum sum, next; + __wsum sum; if (WARN_ON_ONCE(i->data_source)) return 0; @@ -1219,14 +1157,10 @@ size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *_csstate, } sum = csum_shift(csstate->csum, csstate->off); - iterate_and_advance(i, bytes, base, len, off, ({ - next = csum_and_copy_to_user(addr + off, base, len); - sum = csum_block_add(sum, next, off); - next ? 0 : len; - }), ({ - sum = csum_and_memcpy(base, addr + off, len, sum, off); - }) - ) + + bytes = iterate_and_advance2(i, bytes, (void *)addr, &sum, + copy_to_user_iter_csum, + memcpy_to_iter_csum); csstate->csum = csum_shift(sum, csstate->off); csstate->off += bytes; return bytes; From patchwork Mon Sep 25 12:03:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E923CE7A96 for ; Mon, 25 Sep 2023 12:03:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66C8B6B014C; Mon, 25 Sep 2023 08:03:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F6126B0150; Mon, 25 Sep 2023 08:03:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 470156B0153; Mon, 25 Sep 2023 08:03:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2D1C36B014C for ; Mon, 25 Sep 2023 08:03:45 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 043D41CA844 for ; Mon, 25 Sep 2023 12:03:44 +0000 (UTC) X-FDA: 81274985610.13.19B2E34 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf10.hostedemail.com (Postfix) with ESMTP id DA7B4C0029 for ; Mon, 25 Sep 2023 12:03:42 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BFEkW4ct; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643422; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ENB7CtH73xnlPC5Szfr84FelPyw0n3TXhDinmckBTOE=; b=4BhuUAuZVoLHOGL86KSBKU39RcRgy/BEm7jtYsUcKoDnDzkevPhJxM1YF9bTLhJ42C7fCQ CvLdxM7lZgFYUsX6xT+L7jLr4fkJvrNdyPsZxhbPI9uM35DohqUJuG066dMQfwvnwug+3x ROswAoLJE+LP48UfqzJyRxmnR3Lcp9w= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=BFEkW4ct; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf10.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643422; a=rsa-sha256; cv=none; b=TvInITNsH6kXjmQb5LOJmL35qktAQSKKRvtqQ5f9fSR65M3sRc38yqoGmQalF4hhIZgW+1 qTvEohC07NYpzpadpNRJuf3JaWkgCWxKUpIAcwOo7j/EY8JL3CfiYZ33/R9ZDvoZmNkme5 Da3tSFRfGmwhRCWS0+E79NzH5RF3UyA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643422; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ENB7CtH73xnlPC5Szfr84FelPyw0n3TXhDinmckBTOE=; b=BFEkW4ctiMudCUG7eqS9vzak+AUlX6zdsBmIB562G2KuwUA8w2PZwznuV8WyY3rJHYR8gF DiPBJPRzEs7abTu8fBL4DhgI39QxCsavzMUVpcEALrT2wCn+IEP1ILVoZZGaMPdYQ+gLs4 aRL1bwbLHOKPJFBk4WtfVFhkdADp2qY= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-638-Tspro7EYNxWz9xyR8d5l5A-1; Mon, 25 Sep 2023 08:03:38 -0400 X-MC-Unique: Tspro7EYNxWz9xyR8d5l5A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 07FBD3C1E9D8; Mon, 25 Sep 2023 12:03:38 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id 609BB14171B6; Mon, 25 Sep 2023 12:03:36 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 08/12] iov_iter: Don't deal with iter->copy_mc in memcpy_from_iter_mc() Date: Mon, 25 Sep 2023 13:03:05 +0100 Message-ID: <20230925120309.1731676-9-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Queue-Id: DA7B4C0029 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 5o5n61dg81pydncqt7pc88ypaugwezik X-HE-Tag: 1695643422-725366 X-HE-Meta: U2FsdGVkX1+RugM/AEzcZJnLOtOEeLkRZIqnVIKjTHD7mQNCLsEJzPzh5yxGJX+v5WKzeEK5IXayVm9K8tdHYtd2YAaqTth8x7FxzkibbcTd9CE378Rz7yks9hy4bmp17B8Rem5sMgOKxk1AkLO42MVpXtbpkCwPSUUWap6BBcXvdTzi6MA/ef/vX0wZls+xp2wUT/UnT95JAQPd4+hdSyzfhnXZQlWAM2Nt/SjowoV3/+TtspTMcvEP5aapNVEBvAxSSawMxlFNf0muwT9wcJUjzWCfGxd5IkxBjuAaCf4+pmltFykmBpcxhReZfWH2F0WHv/JOtNz1STZfqkC1t6lkyeg9D7bE5yegJ8YxYHWd03Oc9/G5EwgwxHT7ongesL/R/felH69nmw0mZRNRRbvAvM66U5JeSKmW7KQnIMPO+YtRMcdw6mTlKye1GIpDBc7RhqdYSs+tswpm8udnGdduBufJihn2k6y7oNJhuOd2VqskubwfZwZjzmZZ1MAoIa36PZf404b+D9MpBtqHPHA5CHjwCZYkxeVEYLTlvzMu3QqqrYBqI83gHJP9UH2QMQFcgSs6QiGbd+ee1O3Zp6q4Kf6Ln1jo6kEwJJj1CmQzVWpJoWLphEfmZphMukeER2RI2epaRCxirhcJ3hQUk/CLUF2T32eeyKfsSSg1fdqoOmdUqpKtkyRFOcw2jzy5+kL+w1oIE9uaSC3CrqW9HpRZPR215y+89eG1ChQ+v4z2elqsnSxR29qvv9hKPbyysqihosRco0vu/Uz7JOQShVutgrNU614CZ0U1j8GhZE2HkblXQ3bl+rJyfecQWEUG+Ew7eQLJAqaHllD65gyc59WNvUlTvCkE6gLwVLYSY/c4M6n/9RAKJN9Qbcql29onE3Grp3dD/iTDKC0r5Wfz37MpDqFVNrnefK7g6LZwYmZujJ/0b17rmAOAeviPFjtQxxnTxhna/w82EcvOF8f lwhfaHGk WSwO/geiRhkVIugGN+vRf0z81WPVzbsh7rNi4ikfetqiq3spbWq0pLxMYHZ5YapThB/82hJTnk2ISOBIzWp+uG1F3hikbVeBCJ/A37LC0SnxyXFmGs7DUPUoRZ8SrrJruTuHGZKxmeM5jtoyoLbIXZRj1ZJIFMYN40PDrKR8VOIBYMqD+x0UmQixwPDES5QYeFlIfGKUXZxusMfB1BQjt6Nxx4PeWujkpL6ddojQYK/wqcU6DYbC8ahl6laGMNHoS8F79oSFWd5fGhwfBIxtO6TKGtTlsJqA+pnf3g0iwZAGzRlhkfJd/31JFISlW2WzKWK7+reWLyQ4w9AplLjcndMzXxYM3+6bXd7p6mD/snJCznxfZ7rY1zf+iP8BFeGqKjD24NCTRiNCyzQKtbOYRPUBuqdbGbCa7RQiTjEMI7+ubi+qZ5FnUiUZK7r4dT85XUn3nyUOzRd2bv12/OL7EvtUxHctkSjfwCUbK8SKTPY+/dzdhLHKoEqwTx5npI8nqUhg9m6Z0naoITLPRH/1r86aHBxwnOTbWu/GlV/vhStSXIKi9GPAxuMgJvr9dgpCs0ScOu/kcFp961Mg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: iter->copy_mc is only used with a bvec iterator and only by dump_emit_page() in fs/coredump.c so rather than handle this in memcpy_from_iter_mc() where it is checked repeatedly by _copy_from_iter() and copy_page_from_iter_atomic(), Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- lib/iov_iter.c | 39 +++++++++++++++++++++++++++------------ 1 file changed, 27 insertions(+), 12 deletions(-) diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 65374ee91ecd..943aa3cfd7b3 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -253,14 +253,33 @@ size_t _copy_mc_to_iter(const void *addr, size_t bytes, struct iov_iter *i) EXPORT_SYMBOL_GPL(_copy_mc_to_iter); #endif /* CONFIG_ARCH_HAS_COPY_MC */ -static size_t memcpy_from_iter_mc(void *iter_from, size_t progress, - size_t len, void *to, void *priv2) +static __always_inline +size_t memcpy_from_iter_mc(void *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + return copy_mc_to_kernel(to + progress, iter_from, len); +} + +static size_t __copy_from_iter_mc(void *addr, size_t bytes, struct iov_iter *i) { - struct iov_iter *iter = priv2; + size_t progress; - if (iov_iter_is_copy_mc(iter)) - return copy_mc_to_kernel(to + progress, iter_from, len); - return memcpy_from_iter(iter_from, progress, len, to, priv2); + if (unlikely(i->count < bytes)) + bytes = i->count; + if (unlikely(!bytes)) + return 0; + progress = iterate_bvec(i, bytes, addr, NULL, memcpy_from_iter_mc); + i->count -= progress; + return progress; +} + +static __always_inline +size_t __copy_from_iter(void *addr, size_t bytes, struct iov_iter *i) +{ + if (unlikely(iov_iter_is_copy_mc(i))) + return __copy_from_iter_mc(addr, bytes, i); + return iterate_and_advance(i, bytes, addr, + copy_from_user_iter, memcpy_from_iter); } size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i) @@ -270,9 +289,7 @@ size_t _copy_from_iter(void *addr, size_t bytes, struct iov_iter *i) if (user_backed_iter(i)) might_fault(); - return iterate_and_advance2(i, bytes, addr, i, - copy_from_user_iter, - memcpy_from_iter_mc); + return __copy_from_iter(addr, bytes, i); } EXPORT_SYMBOL(_copy_from_iter); @@ -493,9 +510,7 @@ size_t copy_page_from_iter_atomic(struct page *page, size_t offset, } p = kmap_atomic(page) + offset; - n = iterate_and_advance2(i, n, p, i, - copy_from_user_iter, - memcpy_from_iter_mc); + __copy_from_iter(p, n, i); kunmap_atomic(p); copied += n; offset += n; From patchwork Mon Sep 25 12:03:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3C56CE7A81 for ; Mon, 25 Sep 2023 12:03:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 386B76B0153; Mon, 25 Sep 2023 08:03:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 310B56B0158; Mon, 25 Sep 2023 08:03:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13BD66B0159; Mon, 25 Sep 2023 08:03:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id F28236B0153 for ; Mon, 25 Sep 2023 08:03:47 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C242980BE4 for ; Mon, 25 Sep 2023 12:03:47 +0000 (UTC) X-FDA: 81274985694.23.69DA77D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id DE79B40017 for ; Mon, 25 Sep 2023 12:03:45 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KIlkt9aJ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643426; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BfhdjrCXjlyEIl3POBY6DeJzxxNzVNAs8n4KMCiZdlo=; b=l+TicieR42HdoGhEJKcwpDFRD4ermFybTx1yklyGFq/+xvolwyULFa+BtZ0B7TAfeNP6NJ 8ikOgcCHSBFwBHIsNe+tVsWRPC2gg/oysm5D8RfWCX0v3WDJx+BQHQGXYWtUnhiSyk871q Quy7kgWuwF6ZHAmMYYkqSLCC5qqBwMY= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KIlkt9aJ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643426; a=rsa-sha256; cv=none; b=k0sp5ScHJ2dv4sVuqVjoOOaytuauz7qiWQt1OIc0nWzU1P93eKTJ1NbOo2/1icnd9kPyRO idl7IVLs/Y4HJShvMjjTWktjbYBIGmtlCDzwnVUUvtxRv1Juogd5w59U9n2WFD3zmSNsWH WHaiDgv3RyyyhxHTTZFta00fL6Mf0kA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BfhdjrCXjlyEIl3POBY6DeJzxxNzVNAs8n4KMCiZdlo=; b=KIlkt9aJI24AVzoYdlpQbukf8iT6N3dBaaZ4yX6NWMyDFFlrJ+Q5gAVWhm7fHqZS5DLWCm VpSvkP4ZraoXWroSJSbQEfyIyrkWrZ8OyAM0iSsueShCnFASK/BGKvtpa3HxbS8fXZjz6b 8V57Z3flPBdhI5TKYD71eylSJBQagow= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-616-zhyKhm9iPvG_uiimJMIbdw-1; Mon, 25 Sep 2023 08:03:41 -0400 X-MC-Unique: zhyKhm9iPvG_uiimJMIbdw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D00E3185A78E; Mon, 25 Sep 2023 12:03:40 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id B187EC154CB; Mon, 25 Sep 2023 12:03:38 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH v7 09/12] iov_iter, net: Move csum_and_copy_to/from_iter() to net/ Date: Mon, 25 Sep 2023 13:03:06 +0100 Message-ID: <20230925120309.1731676-10-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspamd-Queue-Id: DE79B40017 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7szf4gkf3xzmcixzqpc37bhud1anubm9 X-HE-Tag: 1695643425-278381 X-HE-Meta: U2FsdGVkX18v+OPk3bcaieDRRzesWyZ3OcN9sX4hRO9BPAdS06cYhZnjOkmscd+2fmeQN1gqPKiMy+v8545tpN9wkp/7Cvl9yWKLVzvLW5nAq3J6DztVM0dQPL+cUGYqDVmQW3e78zyQQ7/2XAYOdY9YLgCH1y6q9bJF10gF6dUiM/pVeBZr1ZN0VbXvxRd1enKiT4H/9d6ykmQ/BySMCeJySG80TUJf5/rFL0iiHEJrHPDT8IvDauc6+5E8bP+OrAKBnZtfOjNLbvqqwyHZ12u1LnPmWsJtlnAnImZ0DTb0W80UBiTDETQEVVFVM65skYXBFVSQFq0QPRfUTbGfJFcoFqCtcno+p6/wueGkOjBPBavrVM/zlYoE/0B3S8dff5DjxCF9wB8uiQCm8ETelahV8O+WdT0JW/3HieyC1u7hKzd7pEVPjSk1SClsjnPkDBpOq3t5dqTTmIRgEfsLhntbk0HGNtGOhYxZ5uzcWa3wrIfQD2cgXmxql8/i1iF6iCDpX32y6zu9gnR779xDSZGNU9R9EEIByywJV7wPsy+uWgwtpp7F8Ze8pZb+RKJsCMd1d8QownYYPCd/Al891X/MOJsKztWqaFJARai1G+ct8VgOYbQs7EJ/J97tpHwYtfnkFwl6hRzmC60ND7CMiIA2XHo3zCUrVZa5sp15/ZGg/f1hPJUgNHDVZ2bqqPb50gulDTWMQt2QwpcLNX7bRge17rWei5Q829yMYjEcu0CoDy3KvG5SLvwBMmjrKnS4mKSCQXc0DenfrZN1ohp4dYwbUaxB4dSLomTGVXn/ZfmNmWFU8fBnMx2sYP259SVPFl631iK4TR7mpBAVKRR45BkptTWdi/ITqloM2OzpX6NWxpnkzDkT3Lkp/dfOKAXpVBlQE3KHmcrknI/bM+BoSYD11m9kJAdXCkuIno4NCzTxHAFdpEpdE/H6JJOwelokZzCu8mehDues51+AHsY BsYYnaV6 bIB8yg74uZoq/dn2/KYl80VO45OI2RWpPk7vyr4gwuIPsi2mdingxXfcVmxeAOdkTV+spsqS3CmVHugYCziseM91ode2P8e2OSq3YColC0jMyf1zc20mwZ+vFUk/wMVSDYUnznK5Cd6UlbAdBwZBftOeWRf/ynsC9cBCthIyvAwRow02HW02DKB+bF8EGs9rzDGpwwMPcnU+rGxIp23ryQIiAWKdBfYqwBJxGn42DHK+0vi4Cymw8ufusgdtENLlHJ6WWemQcRqbb4rIy5Bi9Fx3q/Z/SevLInuKf+m4uC25HZ1hCxvjuoXhBgRMixZu9V0SyKpvtNrWtVd1gj02EizDcG3mxqLdBT3wMZs/tKaVjTuWV8nHK6BJ5TTftygIC0mATZigpWSpgEWvyCpFSLw1usDE+jKk8hTq4UoT16ih+4BYNojMXxmtmVxa0ataa65RZIb8rx74Umu0jKm87BZfcnvPqG5gekE3CgFAy/UP6OsF7//xqFeHXPXJ3TsimBcNIYR9IHGF6X1vuUvOu48ahbqZi4ScL0CnxMcw3fngAuaHP2UwfPOTtR6H88BXH31LByJXdAH+ZxCjyMTSrjoYfs1lTMkEFG6o0QdmIuMOsu6v+rBGtB5k28oMpNG09A4LsJilFTnqnfHoXVgNsKLx8edH6yikdvEgE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move csum_and_copy_to/from_iter() to net code now that the iteration framework can be #included. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org --- include/linux/skbuff.h | 25 ++++++++++++ include/linux/uio.h | 18 --------- lib/iov_iter.c | 89 ------------------------------------------ net/core/datagram.c | 50 +++++++++++++++++++++++- net/core/skbuff.c | 33 ++++++++++++++++ 5 files changed, 107 insertions(+), 108 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 4174c4b82d13..d0656cc11c16 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3679,6 +3679,31 @@ static inline int __must_check skb_put_padto(struct sk_buff *skb, unsigned int l return __skb_put_padto(skb, len, true); } +static inline __wsum csum_and_memcpy(void *to, const void *from, size_t len, + __wsum sum, size_t off) +{ + __wsum next = csum_partial_copy_nocheck(from, to, len); + return csum_block_add(sum, next, off); +} + +struct csum_state { + __wsum csum; + size_t off; +}; + +size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i); + +static __always_inline __must_check +bool csum_and_copy_from_iter_full(void *addr, size_t bytes, + __wsum *csum, struct iov_iter *i) +{ + size_t copied = csum_and_copy_from_iter(addr, bytes, csum, i); + if (likely(copied == bytes)) + return true; + iov_iter_revert(i, copied); + return false; +} + static inline int skb_add_data(struct sk_buff *skb, struct iov_iter *from, int copy) { diff --git a/include/linux/uio.h b/include/linux/uio.h index 65d9143f83c8..0a5426c97e02 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -338,24 +338,6 @@ iov_iter_npages_cap(struct iov_iter *i, int maxpages, size_t max_bytes) return npages; } -struct csum_state { - __wsum csum; - size_t off; -}; - -size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *csstate, struct iov_iter *i); -size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i); - -static __always_inline __must_check -bool csum_and_copy_from_iter_full(void *addr, size_t bytes, - __wsum *csum, struct iov_iter *i) -{ - size_t copied = csum_and_copy_from_iter(addr, bytes, csum, i); - if (likely(copied == bytes)) - return true; - iov_iter_revert(i, copied); - return false; -} size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp, struct iov_iter *i); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 943aa3cfd7b3..fef934a8745d 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -10,7 +10,6 @@ #include #include #include -#include #include #include #include @@ -179,13 +178,6 @@ void iov_iter_init(struct iov_iter *i, unsigned int direction, } EXPORT_SYMBOL(iov_iter_init); -static __wsum csum_and_memcpy(void *to, const void *from, size_t len, - __wsum sum, size_t off) -{ - __wsum next = csum_partial_copy_nocheck(from, to, len); - return csum_block_add(sum, next, off); -} - size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) { if (WARN_ON_ONCE(i->data_source)) @@ -1101,87 +1093,6 @@ ssize_t iov_iter_get_pages_alloc2(struct iov_iter *i, } EXPORT_SYMBOL(iov_iter_get_pages_alloc2); -static __always_inline -size_t copy_from_user_iter_csum(void __user *iter_from, size_t progress, - size_t len, void *to, void *priv2) -{ - __wsum next, *csum = priv2; - - next = csum_and_copy_from_user(iter_from, to + progress, len); - *csum = csum_block_add(*csum, next, progress); - return next ? 0 : len; -} - -static __always_inline -size_t memcpy_from_iter_csum(void *iter_from, size_t progress, - size_t len, void *to, void *priv2) -{ - __wsum *csum = priv2; - - *csum = csum_and_memcpy(to + progress, iter_from, len, *csum, progress); - return 0; -} - -size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, - struct iov_iter *i) -{ - if (WARN_ON_ONCE(!i->data_source)) - return 0; - return iterate_and_advance2(i, bytes, addr, csum, - copy_from_user_iter_csum, - memcpy_from_iter_csum); -} -EXPORT_SYMBOL(csum_and_copy_from_iter); - -static __always_inline -size_t copy_to_user_iter_csum(void __user *iter_to, size_t progress, - size_t len, void *from, void *priv2) -{ - __wsum next, *csum = priv2; - - next = csum_and_copy_to_user(from + progress, iter_to, len); - *csum = csum_block_add(*csum, next, progress); - return next ? 0 : len; -} - -static __always_inline -size_t memcpy_to_iter_csum(void *iter_to, size_t progress, - size_t len, void *from, void *priv2) -{ - __wsum *csum = priv2; - - *csum = csum_and_memcpy(iter_to, from + progress, len, *csum, progress); - return 0; -} - -size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *_csstate, - struct iov_iter *i) -{ - struct csum_state *csstate = _csstate; - __wsum sum; - - if (WARN_ON_ONCE(i->data_source)) - return 0; - if (unlikely(iov_iter_is_discard(i))) { - // can't use csum_memcpy() for that one - data is not copied - csstate->csum = csum_block_add(csstate->csum, - csum_partial(addr, bytes, 0), - csstate->off); - csstate->off += bytes; - return bytes; - } - - sum = csum_shift(csstate->csum, csstate->off); - - bytes = iterate_and_advance2(i, bytes, (void *)addr, &sum, - copy_to_user_iter_csum, - memcpy_to_iter_csum); - csstate->csum = csum_shift(sum, csstate->off); - csstate->off += bytes; - return bytes; -} -EXPORT_SYMBOL(csum_and_copy_to_iter); - size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp, struct iov_iter *i) { diff --git a/net/core/datagram.c b/net/core/datagram.c index 176eb5834746..37c89d0933b7 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -50,7 +50,7 @@ #include #include #include -#include +#include #include #include @@ -716,6 +716,54 @@ int zerocopy_sg_from_iter(struct sk_buff *skb, struct iov_iter *from) } EXPORT_SYMBOL(zerocopy_sg_from_iter); +static __always_inline +size_t copy_to_user_iter_csum(void __user *iter_to, size_t progress, + size_t len, void *from, void *priv2) +{ + __wsum next, *csum = priv2; + + next = csum_and_copy_to_user(from + progress, iter_to, len); + *csum = csum_block_add(*csum, next, progress); + return next ? 0 : len; +} + +static __always_inline +size_t memcpy_to_iter_csum(void *iter_to, size_t progress, + size_t len, void *from, void *priv2) +{ + __wsum *csum = priv2; + + *csum = csum_and_memcpy(iter_to, from + progress, len, *csum, progress); + return 0; +} + +static size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *_csstate, + struct iov_iter *i) +{ + struct csum_state *csstate = _csstate; + __wsum sum; + + if (WARN_ON_ONCE(i->data_source)) + return 0; + if (unlikely(iov_iter_is_discard(i))) { + // can't use csum_memcpy() for that one - data is not copied + csstate->csum = csum_block_add(csstate->csum, + csum_partial(addr, bytes, 0), + csstate->off); + csstate->off += bytes; + return bytes; + } + + sum = csum_shift(csstate->csum, csstate->off); + + bytes = iterate_and_advance2(i, bytes, (void *)addr, &sum, + copy_to_user_iter_csum, + memcpy_to_iter_csum); + csstate->csum = csum_shift(sum, csstate->off); + csstate->off += bytes; + return bytes; +} + /** * skb_copy_and_csum_datagram - Copy datagram to an iovec iterator * and update a checksum. diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 4eaf7ed0d1f4..5dbdfce2d05f 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -62,6 +62,7 @@ #include #include #include +#include #include #include @@ -6931,3 +6932,35 @@ ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter, return spliced ?: ret; } EXPORT_SYMBOL(skb_splice_from_iter); + +static __always_inline +size_t memcpy_from_iter_csum(void *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + __wsum *csum = priv2; + + *csum = csum_and_memcpy(to + progress, iter_from, len, *csum, progress); + return 0; +} + +static __always_inline +size_t copy_from_user_iter_csum(void __user *iter_from, size_t progress, + size_t len, void *to, void *priv2) +{ + __wsum next, *csum = priv2; + + next = csum_and_copy_from_user(iter_from, to + progress, len); + *csum = csum_block_add(*csum, next, progress); + return next ? 0 : len; +} + +size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, + struct iov_iter *i) +{ + if (WARN_ON_ONCE(!i->data_source)) + return 0; + return iterate_and_advance2(i, bytes, addr, csum, + copy_from_user_iter_csum, + memcpy_from_iter_csum); +} +EXPORT_SYMBOL(csum_and_copy_from_iter); From patchwork Mon Sep 25 12:03:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD7D9CE7A81 for ; Mon, 25 Sep 2023 12:03:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C39676B015B; Mon, 25 Sep 2023 08:03:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC59F6B015D; Mon, 25 Sep 2023 08:03:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3F5F6B015E; Mon, 25 Sep 2023 08:03:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 83E196B015B for ; Mon, 25 Sep 2023 08:03:52 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 56B0C40BD3 for ; Mon, 25 Sep 2023 12:03:52 +0000 (UTC) X-FDA: 81274985904.02.B1645D4 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 5C92C18001A for ; Mon, 25 Sep 2023 12:03:50 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OPR6Y3Vi; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643430; a=rsa-sha256; cv=none; b=nJf84c9ARQytFoWJpHLvJ/pReBpUnxtXFMWbTEE4C2Jxj+tJH45QPVZoiFB5waGvfZx3ON X0+Q9U07v0mWcP+j3LlUuTCfEeUv7XX1XcP7VPmLg9ZRLRSbkXZyaWYM75sL+Ik2ueeJ46 SHaMfkK2JKMURwy56jPidTrZXcjQu28= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OPR6Y3Vi; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i+29vAFZrXF9naWng0ipN/2z31gEil+IMifE0VymLF4=; b=1WjgufKXC6CVwvxcQX6qBYZJyhOIsILtwRmoyrDyfFeai6GzjvplKZusi7EKJo9vBg7k9l 6xgKfWNfGBubbRP4VJw1g6QBhuyCKSY6syEHiDcEZH9GYba1TV7sLEhmmPWbRVmJ0bMo9j a3SFFLSNQgtLGHjhce7fXaomCnt0RoA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643429; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i+29vAFZrXF9naWng0ipN/2z31gEil+IMifE0VymLF4=; b=OPR6Y3Vi1gbzs2/qTR3HPOtXjw8N39kdyyJiuPHXHSp56QiKjl0jv7P9t1Mns2JZo6QQ5y 6pfWfTZrJJis3DkTKH7oFwFRrMGRd8s66Z6+r5lj9TQ7n6LGrEzGT1MNKwNmtku2ccDO5K pZ5pk201G+zCshLFFuaJEEOLaECTWWI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-47-rBgXHwEcPtC09KsWTkJlvw-1; Mon, 25 Sep 2023 08:03:44 -0400 X-MC-Unique: rBgXHwEcPtC09KsWTkJlvw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7DCE3800045; Mon, 25 Sep 2023 12:03:43 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id 770E5C15BB8; Mon, 25 Sep 2023 12:03:41 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH v7 10/12] iov_iter, net: Fold in csum_and_memcpy() Date: Mon, 25 Sep 2023 13:03:07 +0100 Message-ID: <20230925120309.1731676-11-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5C92C18001A X-Stat-Signature: 7u5jbupwwb85drc4kcdfj3bt3h1oc5go X-HE-Tag: 1695643430-368048 X-HE-Meta: U2FsdGVkX1/2HrBkE0SBQP+MEdSAib+AvD8wVOPQDyv04YeGitx0gqx+dTndgix5eZ8bSbvGAja6EoG+U5iqQdMLDfWW/9jdxdf872G6rvatkUWu7Jd2HBstBgakVG8Rf7ZUsTrnaCkmqFCr76TwBFpkkh1esR4R7b/Ff7k2mN6kBpEf9uItEQiCQtEUsIO1M2JELiYMMw4RNMc66+1+QG4ew4o2wlafkHchrJggl0DLFUt7ccn516W5OKEW6Oe+MqCRRC50CJlTO33UBKMraTbDetQPYzRFuHHkMHQNYLCgJwTa7AidPVq4Q36QQOXzIJsNj7VDgqXFhHmJJsr4Ij5Hx0KOo++wyjcnkmWlhUcZ2cYWX4HrxywG+f4qNyZgrTQnTuBlfYk1tUbh65f7tPVsL8roE9EBZqoXrcLkxpGeE0qJeTIpl5ozo9OQZjPX9j0pf/zv47DdX3mByMgeMqdPeVr8tAuOjKtSg6VeqJ/M349la+aHGKs2F6q7UP9GzaW7FvoUWRN4S73rmYw4CgyKxK6S+sXE5EsNlmXeXhU04+kwyaexu8KqNmPKJZOwwLcfNLVN7M1MhRZILj2jlF9XvR6KA07OxAoyFtkX0oGUkKMlfuszK5RJROokezwsFLmhH2eTGEHqgCBgprpI4S4K3JIvSx+9ciYuJwFVglSd8/D5WVThX0HEeK7gNlMV3IQF2mO591KadTQ4Qw9wJY81goRdMDMxFmsze3bBpQZEYocauMuCgP0h3btFoeYNYS5IqZnVz2q0KJWwHSBmupPYXMrCi+mmkLxbNbvGT6iPMRNv6LMF260QpIaMHdbE6iegwsOoS18gkBUL8YlMbbhauQfiDlvp3gR+Ws4HL1QM9rt7kJtkUzoQGb9U2f9N8P55TTkLSP+FSeyC++ijC4MwJ7O27ZvKXcf0nbGpv3w8hrUp6QyF8MvslW9N2Bjld5h0V8Vn1GdMkE2rPHk HyuFnbtv sJ9f+B0PdJRB7pODBIpon69wYrNbNSEZCrsBTJrLWCQO98RXFNwTDzY2qYBuy8TuhCRibwX118yCE//7fz+H6tVwXUZaW5jaV/zF24H/QAgE4oZck9ETEg5+QLW+3CC8Ny489pYNkkW15msBth/zPrNC01MNQ4/GWmd/60HgP+ibpMG7GQcsYeSmK21JZ6BZKfdV/0z9gy1u/mJQwfUXsVX+r73UdDuDhOY60X3rP74551D4H0IkrG1N9cYHZweX9io64zAt2u65hganU+FVSpwOCZ/N7TEDXXFPZWqmc2GSatSAUpNN9qAM0LHFOsVCkAcAzUsdaGxqKfQF5/sVGYky+RDFJWsu+tbZWPrBNUBM/4HBhkXUnE842ABFhe7pZdtU5Y3pKZA1LBwbHw7UGzduNaIKMEMSVxPOMfWIBxzQ8fdbIvNYWLgmCggHf36jkVJO8gdO5+8SLDpimktzLPW6dQ5pE/YX9iLLn3soxnsmn1HfC+9J+/ThsOCt4p4IaaLsN4ZIM5kWA0ke8zD5gv1SrTuVoHZ8emqwoFN4aEjmUJQwNFgwOjuLZwJW7zvZ9j/olWq+X+Nc9TcAokBfnTVMih+N8DxCxSmX0D6Vy2GfviVzCKP13/VWWH9u+xN+STPpdxGjtBJEdBM/EqPYFV8m3ht4ISc19unQ3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fold csum_and_memcpy() in to its callers. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org --- include/linux/skbuff.h | 7 ------- net/core/datagram.c | 3 ++- net/core/skbuff.c | 3 ++- 3 files changed, 4 insertions(+), 9 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index d0656cc11c16..c81ef5d76953 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3679,13 +3679,6 @@ static inline int __must_check skb_put_padto(struct sk_buff *skb, unsigned int l return __skb_put_padto(skb, len, true); } -static inline __wsum csum_and_memcpy(void *to, const void *from, size_t len, - __wsum sum, size_t off) -{ - __wsum next = csum_partial_copy_nocheck(from, to, len); - return csum_block_add(sum, next, off); -} - struct csum_state { __wsum csum; size_t off; diff --git a/net/core/datagram.c b/net/core/datagram.c index 37c89d0933b7..452620dd41e4 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -732,8 +732,9 @@ size_t memcpy_to_iter_csum(void *iter_to, size_t progress, size_t len, void *from, void *priv2) { __wsum *csum = priv2; + __wsum next = csum_partial_copy_nocheck(from, iter_to, len); - *csum = csum_and_memcpy(iter_to, from + progress, len, *csum, progress); + *csum = csum_block_add(*csum, next, progress); return 0; } diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 5dbdfce2d05f..3efed86321db 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -6938,8 +6938,9 @@ size_t memcpy_from_iter_csum(void *iter_from, size_t progress, size_t len, void *to, void *priv2) { __wsum *csum = priv2; + __wsum next = csum_partial_copy_nocheck(iter_from, to + progress, len); - *csum = csum_and_memcpy(to + progress, iter_from, len, *csum, progress); + *csum = csum_block_add(*csum, next, progress); return 0; } From patchwork Mon Sep 25 12:03:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397696 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E88A6CE7A95 for ; Mon, 25 Sep 2023 12:03:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2EC276B015A; Mon, 25 Sep 2023 08:03:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 273CB6B015B; Mon, 25 Sep 2023 08:03:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A11E6B015D; Mon, 25 Sep 2023 08:03:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E38FA6B015A for ; Mon, 25 Sep 2023 08:03:51 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B547280BDF for ; Mon, 25 Sep 2023 12:03:51 +0000 (UTC) X-FDA: 81274985862.14.5A3A218 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf15.hostedemail.com (Postfix) with ESMTP id E0965A003B for ; Mon, 25 Sep 2023 12:03:49 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AO4QXmlj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643429; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OEnm3txfdKWMLGG8rrdFEcQBV2WWgkp8Kq4P7DhBB1I=; b=WfjYCuOX2Z28jicqNKmGQIaN50ZYoK3TVD/TZFf/3XgBQInbPiJJZOYkkLPKEhRbn1JsW8 +KLFxzIB0QpNR9xpyFy+PMTpKGsNWz5NuFay6I3GV4ILpjvYBt+gTsD7PVeUTMrjqQM5vt 2PjqaIZ4ZOjryS7Pji3ENKmOsAxoyWc= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=AO4QXmlj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf15.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643429; a=rsa-sha256; cv=none; b=W4+RrMs56CY0W+nVtI0SH05EUtv1BXfkWxBLyqidJhs4hxgh3uocwnwlihsDLZsn4fOrKg tsX9n9CYXTfi4ZfXzBHkpHsjMbQW9kAMPo6Vuz4myUL3x++cwXK3YAt8GI7P8OG5GRLTlW xRiyXIfvmtjdzS6he8xNlekk9yL6Fv4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643429; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OEnm3txfdKWMLGG8rrdFEcQBV2WWgkp8Kq4P7DhBB1I=; b=AO4QXmljmMm1oYlOmwydO0HuSLhvXrgsEqpY+tD8m9jmdW8pbPd1L5N9qXK538hVM8j6cF enpWQ/FSKVMOzg0ZCAsLR5Y3Zcq0Cq81ABZu5hL43s/hEz2CCxgofgxuGG0eqagIpFJqu1 4ZaHnjMfjyAIQyFtt/XPMsCp5hANVbk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-436-X-q17QG2OBu5zjjUIF3oyA-1; Mon, 25 Sep 2023 08:03:47 -0400 X-MC-Unique: X-q17QG2OBu5zjjUIF3oyA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3C1CD811E93; Mon, 25 Sep 2023 12:03:46 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id 337AD40C6EA8; Mon, 25 Sep 2023 12:03:44 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH v7 11/12] iov_iter, net: Merge csum_and_copy_from_iter{,_full}() together Date: Mon, 25 Sep 2023 13:03:08 +0100 Message-ID: <20230925120309.1731676-12-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspamd-Queue-Id: E0965A003B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: cabac5sypkmu3ryykzy4mmm9rkrkpsoc X-HE-Tag: 1695643429-249903 X-HE-Meta: U2FsdGVkX1/haiIlqdbGWraufss/DN65YYJKWYjAg9ZIkFjGGz3AoOh6hDUAYvSQlnXAL1AIeij/aEQ5YDi2Ja57DLMtM9Ylvl0INMPiG80jQxZEkPigMxkYSVkm9uFzMlRmWRdJFz5WxdUDMgNUVoSKpi0HvbaXXABZeL/BP5getCcFq4sRsljgQiwtnvSgC8TO3ThZZMoEL4W6Ui+/Km2uETW88OkXWG7tnEPDpPFskc/Yuzy9OEB09SOH56EBmmmpooC6KKL1yHkGdwhhvJWjZrxrmqab11uhmFQAvtBoQqeu4IC8vzGk07wWU9+vStyocDPDyv876StJo2C1DNTCqTFfnTa8qkcUqClk5kVitxvv8CA45oEHlnWFJb2y12hCPJxmiVXL7Vbbrlyx5d5dY2mDim9u8rcEcNK7pDPokGQQjmAv3scFj6jAj9bTeRBcO7knO0RsVj+GLDvDt4HbkkIlFmj6Ixj/oS0YKJTao+mXW3n9Eb1TwFEJvdsSeqLaaxnU4SYpPi7SvfiPsbnykkTn4/5GiDYMf8AWhmYvbjhD8YJyGELV6D6NKo0sAlTy7zJVnDgyFJTavg9ZZp7/sSygSdNUrB3rjDITalZ/Ta6B/Fk0jbc4RL7Cemg0E/H1cgwgYAbmUbqBaYGlQqMNLd7uAyyWLY1toUzp0/apTQbl9HX/axhwzoQDvphPKp5qebv1RhCmFZ9+jex3uEBb5mak6UgN44pqBqKizesTHemG7Rl30GSiTiky1w6PrHs/HycVfTfah9wav1TjORaG9kXv9yX3k8b6YdDeUyK4kTnkY2s0wJOvDKOXOLQr7VLEeWnBLAj5rdbaeZ5mA9wQMjjZ72fkCnEYYbqsHoqGiZ3ithcp5xQ4BdMzebJup727/VD4RoKTe15wmSQGFd/SE3N0FJrdNcHHbidgqLBzPdKDQp7leaEaSe0nFZMfgAi4ZI5/N3/QIkXfaiq URij1viO WQZaL1yL/wibLkN/mSXqwfrYAhIDBbUs4tCF2WXY4qQUi3+S45uKuaUzJactSjAPG4qhQ5EM6nJfXpzLO5udCecUCqRbBIFFI7yl+NLRN4MszrrNTI417WjxHDx3mVuYcRtLYdwc3MTB7lIi4sg7fUnGSMR1yE4t+pU5Ylrsfc+4MutenB3dkFU/VDA9pHdASRdkzjsIUsuEeX2ydFnZ+2aOibNyqlgG4tVjaVZzJGrf/pkOejJqp1pZHAdbojLfCUAnHCT+QICmlT/r8BLUcfvsZfhCwOoqz7WiRP5ZF45ufwvPpEJPryynCks0/uljBfSg3mCibgdfEn2PyZLQU/xzTDMrftBtn3YYyLgWgCuiq9z7K59dQKWPyApuOGhuqx4wRDS+cQxKR2elBHjnw3Ce+yZyBZuBwKyDfw64U7XTWaW3iO4jH+RzuM7FM6umiD3/oILVvdP+CwcsBxe43IyLtBUsNOZvzElGove/yuaSbPhE9luPA+w0IY9GLnt/cwoHPF2RZ1hkqSS9mu/NL7s6OuRLUlkM6qkFU6t3dMY6BLmH8mx4i6389xLGbUBZ7ffF0lMDRTW73/b3sX/HFrPwPyg1BKOlX+jQv6K14VYrvlOrZGfvTSu9OkP0GFPkNRaRoTFvmrDQYN52jsjkZ58N4LT3Yhp62BYZk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move csum_and_copy_from_iter_full() out of line and then merge csum_and_copy_from_iter() into its only caller. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org --- include/linux/skbuff.h | 19 ++----------------- net/core/datagram.c | 5 +++++ net/core/skbuff.c | 20 +++++++++++++------- 3 files changed, 20 insertions(+), 24 deletions(-) diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index c81ef5d76953..be402f55f6d6 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3679,23 +3679,8 @@ static inline int __must_check skb_put_padto(struct sk_buff *skb, unsigned int l return __skb_put_padto(skb, len, true); } -struct csum_state { - __wsum csum; - size_t off; -}; - -size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i); - -static __always_inline __must_check -bool csum_and_copy_from_iter_full(void *addr, size_t bytes, - __wsum *csum, struct iov_iter *i) -{ - size_t copied = csum_and_copy_from_iter(addr, bytes, csum, i); - if (likely(copied == bytes)) - return true; - iov_iter_revert(i, copied); - return false; -} +bool csum_and_copy_from_iter_full(void *addr, size_t bytes, __wsum *csum, struct iov_iter *i) + __must_check; static inline int skb_add_data(struct sk_buff *skb, struct iov_iter *from, int copy) diff --git a/net/core/datagram.c b/net/core/datagram.c index 452620dd41e4..722311eeee18 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -738,6 +738,11 @@ size_t memcpy_to_iter_csum(void *iter_to, size_t progress, return 0; } +struct csum_state { + __wsum csum; + size_t off; +}; + static size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *_csstate, struct iov_iter *i) { diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 3efed86321db..2bfa6a7ba244 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -6955,13 +6955,19 @@ size_t copy_from_user_iter_csum(void __user *iter_from, size_t progress, return next ? 0 : len; } -size_t csum_and_copy_from_iter(void *addr, size_t bytes, __wsum *csum, - struct iov_iter *i) +bool csum_and_copy_from_iter_full(void *addr, size_t bytes, + __wsum *csum, struct iov_iter *i) { + size_t copied; + if (WARN_ON_ONCE(!i->data_source)) - return 0; - return iterate_and_advance2(i, bytes, addr, csum, - copy_from_user_iter_csum, - memcpy_from_iter_csum); + return false; + copied = iterate_and_advance2(i, bytes, addr, csum, + copy_from_user_iter_csum, + memcpy_from_iter_csum); + if (likely(copied == bytes)) + return true; + iov_iter_revert(i, copied); + return false; } -EXPORT_SYMBOL(csum_and_copy_from_iter); +EXPORT_SYMBOL(csum_and_copy_from_iter_full); From patchwork Mon Sep 25 12:03:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13397698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEEADCE7A94 for ; Mon, 25 Sep 2023 12:04:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CAE326B015E; Mon, 25 Sep 2023 08:03:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C38ED6B015F; Mon, 25 Sep 2023 08:03:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3C186B0162; Mon, 25 Sep 2023 08:03:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8C6556B015E for ; Mon, 25 Sep 2023 08:03:55 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 69A541C9B3B for ; Mon, 25 Sep 2023 12:03:55 +0000 (UTC) X-FDA: 81274986030.04.F08C31A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf11.hostedemail.com (Postfix) with ESMTP id AD3404002A for ; Mon, 25 Sep 2023 12:03:53 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DOLzLQUv; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695643433; a=rsa-sha256; cv=none; b=17ccCQOZQ6l0JKYJJOCbVhSGaLEwpear91QsXfaVtmV1qSQoNq+sAiOeHTnVWUbNVujHZ/ JQI57e+g6kZLRC4JtFHP+aNRKBkr1TLD3w4lQ8kYhnHcgDAiamA7+Juq+R4hfGeY/HnN3c sp5KGFWEHuFBZMpz2zvhrFu/KL/L/9Y= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DOLzLQUv; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695643433; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IFANTKBqjctjMhYlR9DlpaXTtUIzXQ6esPI4cVawB1c=; b=Ep1OAj4cmN1JfH0sUUCpdx30wgRRrJYvdfk0MlhXoINW7E0UBm0pa1uHNrcJlxxklBI0Z6 2MSn+cTs0RUFKGnUpkPGrieLqmLH5O1rwzEvruUHcGB03BekJQ18t0ZYldCuiOw5THtB66 QMd238xSheTGkPaQakKVkot01AW+/18= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695643433; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IFANTKBqjctjMhYlR9DlpaXTtUIzXQ6esPI4cVawB1c=; b=DOLzLQUvM/R3EmmnQc1S1DlyIw3SUOfUAWK/cZ9ORmatX/v0o4cUlvfG13iAK0m/t3Qsg4 4pCAeyYV6kE9O9w1/yiNjHD5JXxocxEfjjiN/7Yp7W000O0iR55D0vQ50OnxqNV/95+P8i XlwW3GC6vdJGs4m6K7JZSGguhyA5eK0= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-381-Yv4-_50VO1WoEpBUsV8YeQ-1; Mon, 25 Sep 2023 08:03:49 -0400 X-MC-Unique: Yv4-_50VO1WoEpBUsV8YeQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D8A8829AB3FD; Mon, 25 Sep 2023 12:03:48 +0000 (UTC) Received: from warthog.procyon.org.com (unknown [10.42.28.216]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE54A40C2064; Mon, 25 Sep 2023 12:03:46 +0000 (UTC) From: David Howells To: Jens Axboe Cc: David Howells , Al Viro , Linus Torvalds , Christoph Hellwig , Christian Brauner , David Laight , Matthew Wilcox , Jeff Layton , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH v7 12/12] iov_iter, net: Move hash_and_copy_to_iter() to net/ Date: Mon, 25 Sep 2023 13:03:09 +0100 Message-ID: <20230925120309.1731676-13-dhowells@redhat.com> In-Reply-To: <20230925120309.1731676-1-dhowells@redhat.com> References: <20230925120309.1731676-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: AD3404002A X-Stat-Signature: n8dj5ti8xkem1hf5qh6ub6muhj6u894d X-HE-Tag: 1695643433-479743 X-HE-Meta: U2FsdGVkX1/qWHW4IWKPIbfkPahGblxYdgcA9OYipmlgDp7O1n/DAafiSfu2/2+x8twrvLZ0N4UAd/STyr8ZMm5bS18QAr/imV4CX2iR+9HtmkqVpkoqcHXsWwj/Ukr3y5h9Y8XsTCvC+gaZ3ddChmqd9PBoF2Lm07oka0PwN5SwDwLE8KGRY5qT+7Rqwa1U6CIyGunZiuyjvd0D7R6Cxz26bdhJJfKQjGqqKDZE5JS/nPydXklfYvvMPYllSJNDCfpDHDFNg//ZsNWcQWgXWn6VavoMBtjQ1WZts2MakgjjRLPk5TbbMAD7bxhqrVF9Kff3NKwbmtkACqIgaUMbKheJnWznDQg8bUn3vjA+xnz6whGprJVoHS6Q0ibuJyXpOTjaWu/HpJio9FvkKJYgJKv6c9G8VZkXp3DjXuj1Jxuq9fHxfRUwbQIlUBdkN7UdvV4oCFoJcmi0PHuSWrgQMxI2xWbukIbuwmWvKLpxlv5Vlo2VwSSjb5Bs/EWVOhFA/31VG2yiZunwoPyRdXTCLj8sho1GXeap0xQ/SkE/ce1Zp1JXbkamC4NCniLCwM1tCLRCYI5IrUYMzzU3IpkG1kcDIzTBO/91FsdMKNDG3Oy+E/IvGgS7grjGnAx5sRwIYEMRqPVsYIX3KZpuP2FKI9/02EtuBg5xbjYZ79zbEIRqWOaLwcF+ERbPqZwfP9BTRRWsOSKhCKlZ63sSpOxpy4NMFx3jsqib3+3FO0KqAEZdazaSkuQ1WZa/JpZy3VAt6Sm0wtUBAhRCSpDGlFtG4cnHc6pURxG/xRjQaGcwN9HcbsV/4yYCXEX5VNy9r4NpSEy4+DzNZoFtp8I0P2FwUJ+4c6vTYUFKpPEObKtxxBoXrWGISJ9BmU6RbA9yOfoe6BawgdFvj1+yy0lxRKHKZlOac7nzzfkqZ9/rdYjP49riLRWXJkf380oTOpqkmifts692MzUeJJOr6WWswzB uFpKZy6K hNtyo4ZInQkBFzd8h7h1TczjarW6HISsr0rqsIlk0i3jnHejYnGuQfEwJpmyZsHDBVpkCTfYaXKTHbVKjg6CrZqaC/itb0DmFIhC33EDPSElDL+T9Gs3RJY+84HBX89fuNEtwwhrCOrbqEiWTRiyKMYqI1wtnmS5Hn4w+g9CqVZJyAHii5TKVQwFns48x4+K5YSXLdjeoUTa6hm6wiZzNMb6rZ1lK2USHGSYDjaj5yI0czrX8wJe8kohj1qBdjAimMJVZhHvTszwK4byPpRS+1jbQgsgmT1Vt5S6JhhA5jOFm7tWVQ7vhX16DRow8B2vXkMpiAkU+ZC6mkyJadlcpDDWG99aq4jUR4dATvmnSEtPq+rTfjpn9XNuR5EJT6BdqUckXtYVUurS84sYwZnfFea3p5gu6BfieWPUdE0mr86HZ7wGK/hcXVFjBaubfFgm5U9po47eZpB8vUAhq20O6CKcQBullG3ZPo4LJvYHkvWPM8bFk3QGYvv/6DnUy779/VZlk+PZLQyeJ6nxiulcQu6we0zH6o2JLX72PCTtbIhqquL8hyS/wzVr6qoQ1rZapxKlAj6R8fVob0LC38dyj3fk6jVS42K0wYKWUBEDOi12oDJw6JNzWeyInrw+iAKUvPnbvE6FE15ZoamFOXU2DmgkR2OoCVJh5Gq7L X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move hash_and_copy_to_iter() to be with its only caller in networking code. Signed-off-by: David Howells cc: Alexander Viro cc: Jens Axboe cc: Christoph Hellwig cc: Christian Brauner cc: Matthew Wilcox cc: Linus Torvalds cc: David Laight cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org --- include/linux/uio.h | 3 --- lib/iov_iter.c | 20 -------------------- net/core/datagram.c | 19 +++++++++++++++++++ 3 files changed, 19 insertions(+), 23 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 0a5426c97e02..b6214cbf2a43 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -338,9 +338,6 @@ iov_iter_npages_cap(struct iov_iter *i, int maxpages, size_t max_bytes) return npages; } -size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp, - struct iov_iter *i); - struct iovec *iovec_from_user(const struct iovec __user *uvector, unsigned long nr_segs, unsigned long fast_segs, struct iovec *fast_iov, bool compat); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index fef934a8745d..2547c96d56c7 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -1,5 +1,4 @@ // SPDX-License-Identifier: GPL-2.0-only -#include #include #include #include @@ -1093,25 +1092,6 @@ ssize_t iov_iter_get_pages_alloc2(struct iov_iter *i, } EXPORT_SYMBOL(iov_iter_get_pages_alloc2); -size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp, - struct iov_iter *i) -{ -#ifdef CONFIG_CRYPTO_HASH - struct ahash_request *hash = hashp; - struct scatterlist sg; - size_t copied; - - copied = copy_to_iter(addr, bytes, i); - sg_init_one(&sg, addr, copied); - ahash_request_set_crypt(hash, &sg, NULL, copied); - crypto_ahash_update(hash); - return copied; -#else - return 0; -#endif -} -EXPORT_SYMBOL(hash_and_copy_to_iter); - static int iov_npages(const struct iov_iter *i, int maxpages) { size_t skip = i->iov_offset, size = i->count; diff --git a/net/core/datagram.c b/net/core/datagram.c index 722311eeee18..103d46fa0eeb 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -61,6 +61,7 @@ #include #include #include +#include /* * Is a socket 'connection oriented' ? @@ -489,6 +490,24 @@ static int __skb_datagram_iter(const struct sk_buff *skb, int offset, return 0; } +static size_t hash_and_copy_to_iter(const void *addr, size_t bytes, void *hashp, + struct iov_iter *i) +{ +#ifdef CONFIG_CRYPTO_HASH + struct ahash_request *hash = hashp; + struct scatterlist sg; + size_t copied; + + copied = copy_to_iter(addr, bytes, i); + sg_init_one(&sg, addr, copied); + ahash_request_set_crypt(hash, &sg, NULL, copied); + crypto_ahash_update(hash); + return copied; +#else + return 0; +#endif +} + /** * skb_copy_and_hash_datagram_iter - Copy datagram to an iovec iterator * and update a hash.