From patchwork Fri May 26 14:31:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13256961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1131AC7EE23 for ; Fri, 26 May 2023 14:31:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A07416B007D; Fri, 26 May 2023 10:31:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9909B900002; Fri, 26 May 2023 10:31:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8583A6B0080; Fri, 26 May 2023 10:31:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 75A086B007D for ; Fri, 26 May 2023 10:31:38 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 350E1120F5E for ; Fri, 26 May 2023 14:31:38 +0000 (UTC) X-FDA: 80832644676.04.C8C84C5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf23.hostedemail.com (Postfix) with ESMTP id 0DD1A140042 for ; Fri, 26 May 2023 14:31:27 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XNCsjkUA; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685111488; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r2T5n0FpgtBULW8Nd93uDmoxOfiFTT4h2g2uaR7sC0M=; b=HC6XTESRpmzXGct/IXFcu7rhENimLkF2/e+n3+4czp588Besri2F/70gqSNvxbVahC0tCi 1+yHoe5m+WEqZAZDiG922Vaiab8N5HrqBgdOTEWv5u5zpNwI4skbvMyoHJHgifhndNOZcS ULzs11pZpeFyHVJacHc2Asn1WG+FmbU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XNCsjkUA; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf23.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685111488; a=rsa-sha256; cv=none; b=er+6vXO60vSfxkUaQuWgIYL4WCLF5DLRfonfBuNLXRhE7MnzPD5bDR8/NBV2I30y/9aewp qsXdgWpaCNrx5ycha4uPQcAwieiCWD9ibOjKxWDnCDdmQHgfA9iYGvN/gGmt7kd2UVABrW Np2rKmTnoGXA21pOa74e8HSGKJMGtkA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1685111487; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r2T5n0FpgtBULW8Nd93uDmoxOfiFTT4h2g2uaR7sC0M=; b=XNCsjkUA4ubLpTapHgOpuai5oP9IZyvBPhsHvRjFu8aOKZVx75ui4l/A3IvZf3IYrWSQeV 4Z1F1yNiAFdkB1oysLPkUBBF1Oo8WvxkRXITpe7Umu2DHp+BejNJU31d9/FM9ML8WrgXjm 8aNxLVSxhl0CfVcZ4crQ/XQX7NcUDBk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-Q_IdaTbKOKuzxyYKR0sFFA-1; Fri, 26 May 2023 10:31:23 -0400 X-MC-Unique: Q_IdaTbKOKuzxyYKR0sFFA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 669123855564; Fri, 26 May 2023 14:31:22 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.39.192.68]) by smtp.corp.redhat.com (Postfix) with ESMTP id 666C948205E; Fri, 26 May 2023 14:31:20 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org Cc: David Howells , Herbert Xu , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-crypto@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next 4/8] crypto: af_alg: Use extract_iter_to_sg() to create scatterlists Date: Fri, 26 May 2023 15:31:00 +0100 Message-Id: <20230526143104.882842-5-dhowells@redhat.com> In-Reply-To: <20230526143104.882842-1-dhowells@redhat.com> References: <20230526143104.882842-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0DD1A140042 X-Stat-Signature: qshew7cxqhzqzig5c1tnxcgxcf58eb3n X-Rspam-User: X-HE-Tag: 1685111487-222701 X-HE-Meta: U2FsdGVkX18rEg0MbXbOB4c18GvnAUrDcHgi08aGN9SvV44ouGySTO/DZlWRDgGbvIXTLETCmK5b6jJyKLdYfTjVDVRsOEUeblfE00JbqlXkQIeD7L8SRszAucFysN7UYilMIT5kqIS8XVkdCZgOZKmN0XlBFs2Ya1QpHx3vtuMihqNgjmtayTHifCC+cJ28kWsffsn96D+dpOUT1Veh6z5E1nfZkeUtA36B+9p4/CL/B48Ym6wY6THkvIcyvXb0+3vxtb05cdY2y5RFRBiLAMe+pzGEr7SQrTvc2Ym8FszBSKzK2UVlsru1W3FavG9qEtSddJ81lfpoct+Uvfaw4Hv9+pgQLH1BgHSb2+hlVe9di6apYBs5/l/mgaFVIuUKp4NYZcTrLF3RNulfI7Jt5wGrLzR7APd6cLDLKMTj/iMhzrCRBqfUxF8t+i8V9AqrC6rOIcDEPdgJF3oqK4zQRC+Rvgz7vrox8faj7R0P/A0PSPlrcIGD1F1chEvC/uz2uBhI1bzoXEv9uD7Fx52z8u5D2w/AzFUD/mGtujpVl1bjfaNAKFMa72RNlrx2ntK8zI6MtcIYjFiNZWchSQ43wk/BqVBIM4c9CfEsnBg9Z13feOsXt32/D8ocgo1rx+9/IdlwgWsiEQ7t9wt0b+emnx6z/NHQ1FcpXEECuTAZZniReYXenoWHMcH4nK526Ued1G5ykljf37v1QM/3uhSrTCvJ+mWT04ZzuWrFPvpr5OqPGYkHmeNNzvYHZOAV3xRPR8KY8FgvFq3+/fT1k/6ZaNpXxzlNDbcSLT8MO5EQbef+e9LJQh5N861Vb+TJtg+779LT5//ye6zUYDJl4oCaKkBBhCRFbO7WaouYKK+BVQvqQ5FNHQD8yVzq/0cIHk3Efc5XiS/FzvSy2L+TR3y708KfU+s8OsN2F6miJQpgMXbuDF6ECWvqckWfXJ45Deb6qZP0FmzfLQW9bGxsZQs PkClD1DQ ucyCGZDtp1KamVDws5IoVjHYOvf515ZfAagBJ7ZejMXJpNNpBH7Y/lfQvtNN3i0pS6gUDJeLvO0NtvxPs9c/4gbyz0vJaF56Mj3OGAEtvwQoCLGivcTc4zemePUJdWUqfs0lva/PB3gp792mCP3RDDDF+D/z2vwO07glGLs5vQQ95dGlbBO+cgmwS2dMDnkupKl3Owzf8XD0XzMa26r93a6fX9oi35v4cDH3qvGT/mlDfBdwgUfjd37Sz9lSyvjn9eKzgGL4aUWSSGvcBZyO/UXo00I3PD44tU0TiMOTFl25bHrvAw1UfahjPtrD3yhq+YiQEM9zXjWjQCu2raYPlJI1DB+5WxgOmnYVpEf0oTYhr3EqU7tRfoP8sX9Mn2w4nww3IhzE5JBmLp950iwpz+ZSqSYjRw1DAC503BBmFoK13wJOpeGgpEghVkCUF+U/yeu01fZnPbLL/Hbvv202HodmNnyiYxSaJ7RJAQSxqcfzsfVeo1STQLWLGkmH0oiaTMqnfN268YYVoezDQrxroVjHTgqg6re/wD85rklhwAPm/DeE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use extract_iter_to_sg() to decant the destination iterator into a scatterlist in af_alg_get_rsgl(). af_alg_make_sg() can then be removed. Signed-off-by: David Howells cc: Herbert Xu cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: linux-crypto@vger.kernel.org cc: netdev@vger.kernel.org --- crypto/af_alg.c | 56 +++++++++++------------------------------ crypto/algif_aead.c | 12 ++++----- crypto/algif_hash.c | 18 +++++++++---- crypto/algif_skcipher.c | 2 +- include/crypto/if_alg.h | 6 ++--- 5 files changed, 36 insertions(+), 58 deletions(-) diff --git a/crypto/af_alg.c b/crypto/af_alg.c index 7caff10df643..8a35f1364ac3 100644 --- a/crypto/af_alg.c +++ b/crypto/af_alg.c @@ -531,45 +531,11 @@ static const struct net_proto_family alg_family = { .owner = THIS_MODULE, }; -int af_alg_make_sg(struct af_alg_sgl *sgl, struct iov_iter *iter, int len) -{ - struct page **pages = sgl->pages; - size_t off; - ssize_t n; - int npages, i; - - n = iov_iter_extract_pages(iter, &pages, len, ALG_MAX_PAGES, 0, &off); - if (n < 0) - return n; - - sgl->need_unpin = iov_iter_extract_will_pin(iter); - - npages = DIV_ROUND_UP(off + n, PAGE_SIZE); - if (WARN_ON(npages == 0)) - return -EINVAL; - /* Add one extra for linking */ - sg_init_table(sgl->sg, npages + 1); - - for (i = 0, len = n; i < npages; i++) { - int plen = min_t(int, len, PAGE_SIZE - off); - - sg_set_page(sgl->sg + i, sgl->pages[i], plen, off); - - off = 0; - len -= plen; - } - sg_mark_end(sgl->sg + npages - 1); - sgl->npages = npages; - - return n; -} -EXPORT_SYMBOL_GPL(af_alg_make_sg); - static void af_alg_link_sg(struct af_alg_sgl *sgl_prev, struct af_alg_sgl *sgl_new) { - sg_unmark_end(sgl_prev->sg + sgl_prev->npages - 1); - sg_chain(sgl_prev->sg, sgl_prev->npages + 1, sgl_new->sg); + sg_unmark_end(sgl_prev->sgt.sgl + sgl_prev->sgt.nents - 1); + sg_chain(sgl_prev->sgt.sgl, sgl_prev->sgt.nents + 1, sgl_new->sgt.sgl); } void af_alg_free_sg(struct af_alg_sgl *sgl) @@ -577,8 +543,8 @@ void af_alg_free_sg(struct af_alg_sgl *sgl) int i; if (sgl->need_unpin) - for (i = 0; i < sgl->npages; i++) - unpin_user_page(sgl->pages[i]); + for (i = 0; i < sgl->sgt.nents; i++) + unpin_user_page(sg_page(&sgl->sgt.sgl[i])); } EXPORT_SYMBOL_GPL(af_alg_free_sg); @@ -1292,8 +1258,8 @@ int af_alg_get_rsgl(struct sock *sk, struct msghdr *msg, int flags, while (maxsize > len && msg_data_left(msg)) { struct af_alg_rsgl *rsgl; + ssize_t err; size_t seglen; - int err; /* limit the amount of readable buffers */ if (!af_alg_readable(sk)) @@ -1310,16 +1276,22 @@ int af_alg_get_rsgl(struct sock *sk, struct msghdr *msg, int flags, return -ENOMEM; } - rsgl->sgl.npages = 0; + rsgl->sgl.sgt.sgl = rsgl->sgl.sgl; + rsgl->sgl.sgt.nents = 0; + rsgl->sgl.sgt.orig_nents = 0; list_add_tail(&rsgl->list, &areq->rsgl_list); - /* make one iovec available as scatterlist */ - err = af_alg_make_sg(&rsgl->sgl, &msg->msg_iter, seglen); + sg_init_table(rsgl->sgl.sgt.sgl, ALG_MAX_PAGES); + err = extract_iter_to_sg(&msg->msg_iter, seglen, &rsgl->sgl.sgt, + ALG_MAX_PAGES, 0); if (err < 0) { rsgl->sg_num_bytes = 0; return err; } + sg_mark_end(rsgl->sgl.sgt.sgl + rsgl->sgl.sgt.nents - 1); + rsgl->sgl.need_unpin = iov_iter_extract_will_pin(&msg->msg_iter); + /* chain the new scatterlist with previous one */ if (areq->last_rsgl) af_alg_link_sg(&areq->last_rsgl->sgl, &rsgl->sgl); diff --git a/crypto/algif_aead.c b/crypto/algif_aead.c index 42493b4d8ce4..f6aa3856d8d5 100644 --- a/crypto/algif_aead.c +++ b/crypto/algif_aead.c @@ -210,7 +210,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, */ /* Use the RX SGL as source (and destination) for crypto op. */ - rsgl_src = areq->first_rsgl.sgl.sg; + rsgl_src = areq->first_rsgl.sgl.sgt.sgl; if (ctx->enc) { /* @@ -224,7 +224,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, * RX SGL: AAD || PT || Tag */ err = crypto_aead_copy_sgl(null_tfm, tsgl_src, - areq->first_rsgl.sgl.sg, processed); + areq->first_rsgl.sgl.sgt.sgl, processed); if (err) goto free; af_alg_pull_tsgl(sk, processed, NULL, 0); @@ -242,7 +242,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, /* Copy AAD || CT to RX SGL buffer for in-place operation. */ err = crypto_aead_copy_sgl(null_tfm, tsgl_src, - areq->first_rsgl.sgl.sg, outlen); + areq->first_rsgl.sgl.sgt.sgl, outlen); if (err) goto free; @@ -268,8 +268,8 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, /* RX SGL present */ struct af_alg_sgl *sgl_prev = &areq->last_rsgl->sgl; - sg_unmark_end(sgl_prev->sg + sgl_prev->npages - 1); - sg_chain(sgl_prev->sg, sgl_prev->npages + 1, + sg_unmark_end(sgl_prev->sgt.sgl + sgl_prev->sgt.nents - 1); + sg_chain(sgl_prev->sgt.sgl, sgl_prev->sgt.nents + 1, areq->tsgl); } else /* no RX SGL present (e.g. authentication only) */ @@ -278,7 +278,7 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, /* Initialize the crypto operation */ aead_request_set_crypt(&areq->cra_u.aead_req, rsgl_src, - areq->first_rsgl.sgl.sg, used, ctx->iv); + areq->first_rsgl.sgl.sgt.sgl, used, ctx->iv); aead_request_set_ad(&areq->cra_u.aead_req, ctx->aead_assoclen); aead_request_set_tfm(&areq->cra_u.aead_req, tfm); diff --git a/crypto/algif_hash.c b/crypto/algif_hash.c index 63af72e19fa8..16c69c4b9c62 100644 --- a/crypto/algif_hash.c +++ b/crypto/algif_hash.c @@ -91,13 +91,21 @@ static int hash_sendmsg(struct socket *sock, struct msghdr *msg, if (len > limit) len = limit; - len = af_alg_make_sg(&ctx->sgl, &msg->msg_iter, len); + ctx->sgl.sgt.sgl = ctx->sgl.sgl; + ctx->sgl.sgt.nents = 0; + ctx->sgl.sgt.orig_nents = 0; + + len = extract_iter_to_sg(&msg->msg_iter, len, &ctx->sgl.sgt, + ALG_MAX_PAGES, 0); if (len < 0) { err = copied ? 0 : len; goto unlock; } + sg_mark_end(ctx->sgl.sgt.sgl + ctx->sgl.sgt.nents); + + ctx->sgl.need_unpin = iov_iter_extract_will_pin(&msg->msg_iter); - ahash_request_set_crypt(&ctx->req, ctx->sgl.sg, NULL, len); + ahash_request_set_crypt(&ctx->req, ctx->sgl.sgt.sgl, NULL, len); err = crypto_wait_req(crypto_ahash_update(&ctx->req), &ctx->wait); @@ -141,8 +149,8 @@ static ssize_t hash_sendpage(struct socket *sock, struct page *page, flags |= MSG_MORE; lock_sock(sk); - sg_init_table(ctx->sgl.sg, 1); - sg_set_page(ctx->sgl.sg, page, size, offset); + sg_init_table(ctx->sgl.sgl, 1); + sg_set_page(ctx->sgl.sgl, page, size, offset); if (!(flags & MSG_MORE)) { err = hash_alloc_result(sk, ctx); @@ -151,7 +159,7 @@ static ssize_t hash_sendpage(struct socket *sock, struct page *page, } else if (!ctx->more) hash_free_result(sk, ctx); - ahash_request_set_crypt(&ctx->req, ctx->sgl.sg, ctx->result, size); + ahash_request_set_crypt(&ctx->req, ctx->sgl.sgl, ctx->result, size); if (!(flags & MSG_MORE)) { if (ctx->more) diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c index ee8890ee8f33..a251cd6bd5b9 100644 --- a/crypto/algif_skcipher.c +++ b/crypto/algif_skcipher.c @@ -105,7 +105,7 @@ static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg, /* Initialize the crypto operation */ skcipher_request_set_tfm(&areq->cra_u.skcipher_req, tfm); skcipher_request_set_crypt(&areq->cra_u.skcipher_req, areq->tsgl, - areq->first_rsgl.sgl.sg, len, ctx->iv); + areq->first_rsgl.sgl.sgt.sgl, len, ctx->iv); if (msg->msg_iocb && !is_sync_kiocb(msg->msg_iocb)) { /* AIO operation */ diff --git a/include/crypto/if_alg.h b/include/crypto/if_alg.h index 46494b33f5bc..34224e77f5a2 100644 --- a/include/crypto/if_alg.h +++ b/include/crypto/if_alg.h @@ -56,9 +56,8 @@ struct af_alg_type { }; struct af_alg_sgl { - struct scatterlist sg[ALG_MAX_PAGES + 1]; - struct page *pages[ALG_MAX_PAGES]; - unsigned int npages; + struct sg_table sgt; + struct scatterlist sgl[ALG_MAX_PAGES + 1]; bool need_unpin; }; @@ -164,7 +163,6 @@ int af_alg_release(struct socket *sock); void af_alg_release_parent(struct sock *sk); int af_alg_accept(struct sock *sk, struct socket *newsock, bool kern); -int af_alg_make_sg(struct af_alg_sgl *sgl, struct iov_iter *iter, int len); void af_alg_free_sg(struct af_alg_sgl *sgl); static inline struct alg_sock *alg_sk(struct sock *sk)