mbox series

[0/6] Clean up and improve ARM/arm64 CRC-T10DIF code

Message ID 20241028190207.1394367-8-ardb+git@google.com (mailing list archive)
Headers show
Series Clean up and improve ARM/arm64 CRC-T10DIF code | expand

Message

Ard Biesheuvel Oct. 28, 2024, 7:02 p.m. UTC 
From: Ard Biesheuvel <ardb@kernel.org>

I realized that the generic sequence implementing 64x64 polynomial
multiply using 8x8 PMULL instructions, which is used in the CRC-T10DIF
code to implement a fallback version for cores that lack the 64x64 PMULL
instruction, is not very efficient.

The folding coefficients that are used when processing the bulk of the
data are only 16 bits wide, and so 3/4 of the partial results of all those
8x8->16 bit multiplications do not contribute anything to the end result.

This means we can use a much faster implementation, producing a speedup
of 3.3x on Cortex-A72 without Crypto Extensions (Raspberry Pi 4).

The same logic can be ported to 32-bit ARM too, where it produces a
speedup of 6.6x compared with the generic C implementation on the same
platform.

Ard Biesheuvel (6):
  crypto: arm64/crct10dif - Remove obsolete chunking logic
  crypto: arm64/crct10dif - Use faster 16x64 bit polynomial multiply
  crypto: arm64/crct10dif - Remove remaining 64x64 PMULL fallback code
  crypto: arm/crct10dif - Use existing mov_l macro instead of __adrl
  crypto: arm/crct10dif - Macroify PMULL asm code
  crypto: arm/crct10dif - Implement plain NEON variant

 arch/arm/crypto/crct10dif-ce-core.S   | 201 ++++++++------
 arch/arm/crypto/crct10dif-ce-glue.c   |  54 +++-
 arch/arm64/crypto/crct10dif-ce-core.S | 282 +++++++-------------
 arch/arm64/crypto/crct10dif-ce-glue.c |  43 ++-
 4 files changed, 274 insertions(+), 306 deletions(-)