From patchwork Sat Jun 30 00:01:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 10497765 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4C32A60327 for ; Sat, 30 Jun 2018 00:05:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 497EB295F5 for ; Sat, 30 Jun 2018 00:05:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3D82F296A9; Sat, 30 Jun 2018 00:05:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD550295F5 for ; Sat, 30 Jun 2018 00:05:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S936942AbeF3AFJ (ORCPT ); Fri, 29 Jun 2018 20:05:09 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:44610 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932449AbeF3AFH (ORCPT ); Fri, 29 Jun 2018 20:05:07 -0400 Received: by mail-pl0-f65.google.com with SMTP id m16-v6so5168571pls.11 for ; Fri, 29 Jun 2018 17:05:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iFSp+VhgbPijon4PiNu+iVVLKBsyk6fFTVVsiPe93SY=; b=YwJEXvEUdxiyiYP8KbpwRhGvRZ378lXtAiPHAz9Y2xeUeTo983OhJP+D1t+XMfmebA cLuJ+aZhVZOmw4gU084TcWDA3+3ZyDO0g8qOfKCfuU69kBI+a0r5LLxBI6RzYSBsrEYz eNS3Lj6okDiQxUBHogLtBBB0jqe1cpkHFMBfk4v+/NouvyCFrmiKnzxQ/x0gpis6nbvs cx+ICQ76M/SCi5EH8/bSoeOPnYOzIj7EAE35QFIn54081AioYBCF06wuakbVar4TwOLi PCbRIInyKosAG7v2ZewgXHpnRadBienMMZ+73PPtlPi7STZeuOOSbn3XBUHhn5tD/ZkV J2Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iFSp+VhgbPijon4PiNu+iVVLKBsyk6fFTVVsiPe93SY=; b=Cf5cpVcz64zx7iWh3BVEVseRFXGpCY8tR/aQ3aPvJWChOBEtXYDftXlVj/CNQ0BrnN i9Ym7pQq50U27dN6pVf7TMwGrrJIFatVDnb6qIeMktW4ajdvkct54ZV3vY76Ch87PmIs rLaXp1BY3qOe20+S87cThBlFOd3NZNETra2M996CZgH20ChVb+Rw904nSK7J1cQjsqFs NkpzX4jyPlphx9we2kcG/v8iHFq92wmcDIHYvdE6X8uQxAZovwPCxm3f+OsySvSuf0iA bLAHvOJfamUMTeGbkY1c18VMkvwa3IHbwcTULPapL8jW7FftyVudWlBH0jqvUKT/3tS9 eULA== X-Gm-Message-State: APt69E2YVpPIvRXKLP0W4ex9ISKT41b9g3GSqyieP/rCtIzYdrqlcOHK pvzmtyFZTUHEVkdcyESfcg2EizRz X-Google-Smtp-Source: ADUXVKKZ/lXraRoZr8cQZFnEcq4csssrI9fkqGj6wqCL07PPJth9tGJndxbDNGZd0yHPCWbaIqkTZA== X-Received: by 2002:a17:902:a5:: with SMTP id a34-v6mr17122269pla.80.1530317107091; Fri, 29 Jun 2018 17:05:07 -0700 (PDT) Received: from ebiggers-linuxstation.kir.corp.google.com ([2620:15c:17:3:dc28:5c82:b905:e8a8]) by smtp.gmail.com with ESMTPSA id j13-v6sm16335788pgq.73.2018.06.29.17.05.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Jun 2018 17:05:06 -0700 (PDT) From: Eric Biggers To: linux-crypto@vger.kernel.org, Herbert Xu Cc: Megha Dey , Tim Chen , Eric Biggers Subject: [PATCH 4/4] crypto: x86/sha-mb - decrease priority of multibuffer algorithms Date: Fri, 29 Jun 2018 17:01:44 -0700 Message-Id: <20180630000144.62230-5-ebiggers3@gmail.com> X-Mailer: git-send-email 2.18.0.399.gad0ab374a1-goog In-Reply-To: <20180630000144.62230-1-ebiggers3@gmail.com> References: <20180630000144.62230-1-ebiggers3@gmail.com> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Eric Biggers With all the crypto modules enabled on x86, and with a CPU that supports AVX-2 but not SHA-NI instructions (e.g. Haswell, Broadwell, Skylake), the "multibuffer" implementations of SHA-1, SHA-256, and SHA-512 are the highest priority. However, these implementations only perform well when many hash requests are being submitted concurrently, filling all 8 AVX-2 lanes. Otherwise, they are incredibly slow, as they waste time waiting for more requests to arrive before proceeding to execute each request. For example, here are the speeds I see hashing 4096-byte buffers with a single thread on a Haswell-based processor: generic avx2 mb (multibuffer) ------- -------- ---------------- sha1 602 MB/s 997 MB/s 0.61 MB/s sha256 228 MB/s 412 MB/s 0.61 MB/s sha512 312 MB/s 559 MB/s 0.61 MB/s So, the multibuffer implementation is 500 to 1000 times slower than the other implementations. Note that with smaller buffers or more update()s per digest, the difference would be even greater. I believe the vast majority of people are in the boat where the multibuffer code is much slower, and only a small minority are doing the highly parallel, hashing-intensive, latency-flexible workloads (maybe IPsec on servers?) where the multibuffer code may be beneficial. Yet, people often aren't familiar with all the crypto config options and so the multibuffer code may inadvertently be built into the kernel. Also the multibuffer code apparently hasn't been very well tested, seeing as it was sometimes computing the wrong SHA-256 digest. So, let's make the multibuffer algorithms low priority. Users who want to use them can either request them explicitly by driver name, or use NETLINK_CRYPTO (crypto_user) to increase their priority at runtime. Signed-off-by: Eric Biggers --- arch/x86/crypto/sha1-mb/sha1_mb.c | 9 ++++++++- arch/x86/crypto/sha256-mb/sha256_mb.c | 9 ++++++++- arch/x86/crypto/sha512-mb/sha512_mb.c | 9 ++++++++- 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/x86/crypto/sha1-mb/sha1_mb.c b/arch/x86/crypto/sha1-mb/sha1_mb.c index e17655ffde79..4b2430274935 100644 --- a/arch/x86/crypto/sha1-mb/sha1_mb.c +++ b/arch/x86/crypto/sha1-mb/sha1_mb.c @@ -871,7 +871,14 @@ static struct ahash_alg sha1_mb_async_alg = { .base = { .cra_name = "sha1", .cra_driver_name = "sha1_mb", - .cra_priority = 200, + /* + * Low priority, since with few concurrent hash requests + * this is extremely slow due to the flush delay. Users + * whose workloads would benefit from this can request + * it explicitly by driver name, or can increase its + * priority at runtime using NETLINK_CRYPTO. + */ + .cra_priority = 50, .cra_flags = CRYPTO_ALG_TYPE_AHASH | CRYPTO_ALG_ASYNC, .cra_blocksize = SHA1_BLOCK_SIZE, .cra_type = &crypto_ahash_type, diff --git a/arch/x86/crypto/sha256-mb/sha256_mb.c b/arch/x86/crypto/sha256-mb/sha256_mb.c index 4c46ac1b6653..4c07f6c12c37 100644 --- a/arch/x86/crypto/sha256-mb/sha256_mb.c +++ b/arch/x86/crypto/sha256-mb/sha256_mb.c @@ -870,7 +870,14 @@ static struct ahash_alg sha256_mb_async_alg = { .base = { .cra_name = "sha256", .cra_driver_name = "sha256_mb", - .cra_priority = 200, + /* + * Low priority, since with few concurrent hash requests + * this is extremely slow due to the flush delay. Users + * whose workloads would benefit from this can request + * it explicitly by driver name, or can increase its + * priority at runtime using NETLINK_CRYPTO. + */ + .cra_priority = 50, .cra_flags = CRYPTO_ALG_TYPE_AHASH | CRYPTO_ALG_ASYNC, .cra_blocksize = SHA256_BLOCK_SIZE, diff --git a/arch/x86/crypto/sha512-mb/sha512_mb.c b/arch/x86/crypto/sha512-mb/sha512_mb.c index 39e2bbdc1836..6a8c31581604 100644 --- a/arch/x86/crypto/sha512-mb/sha512_mb.c +++ b/arch/x86/crypto/sha512-mb/sha512_mb.c @@ -904,7 +904,14 @@ static struct ahash_alg sha512_mb_async_alg = { .base = { .cra_name = "sha512", .cra_driver_name = "sha512_mb", - .cra_priority = 200, + /* + * Low priority, since with few concurrent hash requests + * this is extremely slow due to the flush delay. Users + * whose workloads would benefit from this can request + * it explicitly by driver name, or can increase its + * priority at runtime using NETLINK_CRYPTO. + */ + .cra_priority = 50, .cra_flags = CRYPTO_ALG_TYPE_AHASH | CRYPTO_ALG_ASYNC, .cra_blocksize = SHA512_BLOCK_SIZE,