From patchwork Tue Dec 15 23:47:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 11976023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9343EC2BBCD for ; Tue, 15 Dec 2020 23:55:11 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BC6B22283 for ; Tue, 15 Dec 2020 23:55:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BC6B22283 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=cBp77kI3RHJZFs5wSbFzEdreAiatAlXw4vQGkI0jVTE=; b=Zm9zfOBO36p+n9PGDgKGzH5QMF UuqlMpSBbeDYRLeLYXkEmHqEMfqaZ+LH9aqxiGmmNPXcjgFUhnZ6J4o3qUa6kN58GrxNbpTyZiER0 VyUD6yldxe5QYmuRtG0YQQxa3AApbOGsdiCLba4xHz1/4mFQsZ7tZZvlSajqgQ2TMXl9g3x+xGc6r 1TZ4AJ16qKvwq+xY7YSwibR9q6FAnNNc5L1OKSLwE3pr479vbYf7tUBe5iI1w0pHusYIWhPgidUG/ AtcjQr2gjxJrHMvDU8PpUup8Xezyr2Hq1eOt7QMsMhEfqDg4B2qDvs5Ti4AyfmpSIh/UsAvLJYq1K LnRhjcUw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kpK8g-00026l-9J; Tue, 15 Dec 2020 23:53:54 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kpK8d-000256-L8 for linux-arm-kernel@lists.infradead.org; Tue, 15 Dec 2020 23:53:52 +0000 From: Eric Biggers Authentication-Results: mail.kernel.org; dkim=permerror (bad message/signature format) To: linux-crypto@vger.kernel.org Subject: [PATCH 0/5] crypto: add NEON-optimized BLAKE2b Date: Tue, 15 Dec 2020 15:47:03 -0800 Message-Id: <20201215234708.105527-1-ebiggers@kernel.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201215_185351_775454_1DA89AF9 X-CRM114-Status: GOOD ( 14.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Jason A . Donenfeld" , Herbert Xu , David Sterba , Ard Biesheuvel , linux-arm-kernel@lists.infradead.org, Paul Crowley Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patchset adds a NEON implementation of BLAKE2b for 32-bit ARM. Patches 1-4 prepare for it by making some updates to the generic implementation, while patch 5 adds the actual NEON implementation. On Cortex-A7 (which these days is the most common ARM processor that doesn't have the ARMv8 Crypto Extensions), this is over twice as fast as SHA-256, and slightly faster than SHA-1. It is also almost three times as fast as the generic implementation of BLAKE2b: Algorithm Cycles per byte (on 4096-byte messages) =================== ======================================= blake2b-256-neon 14.1 sha1-neon 16.4 sha1-asm 20.8 blake2s-256-generic 26.1 sha256-neon 28.9 sha256-asm 32.1 blake2b-256-generic 39.9 This implementation isn't directly based on any other implementation, but it borrows some ideas from previous NEON code I've written as well as from chacha-neon-core.S. At least on Cortex-A7, it is faster than the other NEON implementations of BLAKE2b I'm aware of (the implementation in the BLAKE2 official repository using intrinsics, and Andrew Moon's implementation which can be found in SUPERCOP). NEON-optimized BLAKE2b is useful because there is interest in using BLAKE2b-256 for dm-verity on low-end Android devices (specifically, devices that lack the ARMv8 Crypto Extensions) to replace SHA-1. On these devices, the performance cost of upgrading to SHA-256 may be unacceptable, whereas BLAKE2b-256 would actually improve performance. Although BLAKE2b is intended for 64-bit platforms (unlike BLAKE2s which is intended for 32-bit platforms), on 32-bit ARM processors with NEON, BLAKE2b is actually faster than BLAKE2s. This is because NEON supports 64-bit operations, and because BLAKE2s's block size is too small for NEON to be helpful for it. The best I've been able to do with BLAKE2s on Cortex-A7 is 19.0 cpb with an optimized scalar implementation. (I didn't try BLAKE2sp and BLAKE3, which in theory would be faster, but they're more complex as they require running multiple hashes at once. Note that BLAKE2b already uses all the NEON bandwidth on the Cortex-A7, so I expect that any speedup from BLAKE2sp or BLAKE3 would come only from the smaller number of rounds, not from the extra parallelism.) This patchset was tested on a Raspberry Pi 2, including with CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y. Eric Biggers (5): crypto: blake2b - rename constants for consistency with blake2s crypto: blake2b - define shash_alg structs using macros crypto: blake2b - export helpers for optimized implementations crypto: blake2b - update file comment crypto: arm/blake2b - add NEON-optimized BLAKE2b implementation arch/arm/crypto/Kconfig | 10 + arch/arm/crypto/Makefile | 2 + arch/arm/crypto/blake2b-neon-core.S | 357 ++++++++++++++++++++++++++++ arch/arm/crypto/blake2b-neon-glue.c | 105 ++++++++ crypto/blake2b_generic.c | 205 +++++++--------- include/crypto/blake2b.h | 54 +++++ 6 files changed, 619 insertions(+), 114 deletions(-) create mode 100644 arch/arm/crypto/blake2b-neon-core.S create mode 100644 arch/arm/crypto/blake2b-neon-glue.c create mode 100644 include/crypto/blake2b.h base-commit: 3db1a3fa98808aa90f95ec3e0fa2fc7abf28f5c9