From patchwork Fri Apr 20 23:30:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 10353859 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 928D760365 for ; Fri, 20 Apr 2018 23:31:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82FD6288C3 for ; Fri, 20 Apr 2018 23:31:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 75C9E288D2; Fri, 20 Apr 2018 23:31:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25761288C3 for ; Fri, 20 Apr 2018 23:31:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752288AbeDTXbW (ORCPT ); Fri, 20 Apr 2018 19:31:22 -0400 Received: from mail-pl0-f42.google.com ([209.85.160.42]:35093 "EHLO mail-pl0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751876AbeDTXbV (ORCPT ); Fri, 20 Apr 2018 19:31:21 -0400 Received: by mail-pl0-f42.google.com with SMTP id w21-v6so6030192plq.2 for ; Fri, 20 Apr 2018 16:31:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=6AX/EpRpO7UGlHKwOCGYenGa20cKGfyyyvaH7fEGlgQ=; b=eO5vvzMrp3UQG9mFyrMVfrERE40mzWy+V7otp45AqnE6ssjOJ57VGXAhbMDE6Avwry EwNGDV77fdBN72uVHId4ZwCgzwic3RRVkS8VO5wiHvZbm9e30D7cAaGyvUptV0pP+8Vu PuTDKI4iOWuwKYvOS+Acwd9jpqBF28NufRCrp21u0/YYWH9E5241d8cUMkpzlxagxBJv hOUa63n4m3vEyD/mDK/sM+4wEiqDKRTqWDzI4L+jx5fuLk5mLva0lJzgH/Ea/0rAulyY sRkRFki6NqBnh4ZAgdoffhvjH4C2olUlNfYxvTMWXr7blJzQIBVwsscmoX0SwyjbPLpu U9iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=6AX/EpRpO7UGlHKwOCGYenGa20cKGfyyyvaH7fEGlgQ=; b=he2cpLbyYD7lpVQoXwhw2y1Y0LyvOMlLC16WGmQRzYPNb6vIKm0zOuvfmW6DXsigUC r6Jlj0ldgbttxpp3xeb0PYwK+C/7+SQI7pTq0/DSnNSiRNY5JykqJi4f5gKPlgpWe6Ir fGvtqqV04KEtW+N2+QmfO64tptdFhbP/9eNlS5L6mxoG2Y/DpSRH8Frxdwp/iGby7278 7+zCNrhwLlge8DPZQQ9KMeLbxULxNUTkbE6lH9OcgYSvxko7GZhIetp8Jmj0sb24yIR4 i4MuAqLKjNlmL7gD4NQ7dCy9Dlo1IdcZgjeUBuSO4aL0vSU46JJbVGUgzvwC6m9Fp2Mq qmvg== X-Gm-Message-State: ALQs6tBazL8Bu9vLMxKDLXgRrizpEPf1chU8oMy/dWIBZM+oA3qts4kO gDcERGl5xbaotw2wlVdqGjvQw3MdmJM= X-Google-Smtp-Source: AIpwx49RU6w1+ME6DD7oyPM+UILgqmq73oVXNG6HRNWnIiH2Au3YapFw+7AHnc7ZyR00otXeATo4KA== X-Received: by 2002:a17:902:265:: with SMTP id 92-v6mr11560653plc.368.1524267080775; Fri, 20 Apr 2018 16:31:20 -0700 (PDT) Received: from ebiggers-linuxstation.kir.corp.google.com ([2620:15c:17:3:dc28:5c82:b905:e8a8]) by smtp.gmail.com with ESMTPSA id p6sm14235885pfk.104.2018.04.20.16.31.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 20 Apr 2018 16:31:19 -0700 (PDT) From: Eric Biggers To: linux-fscrypt@vger.kernel.org, "Theodore Y . Ts'o" Cc: Jaegeuk Kim , Paul Crowley , Enric Balletbo i Serra , Mikulas Patocka , linux-kernel@vger.kernel.org, Eric Biggers Subject: [PATCH] fscrypt: use unbound workqueue for decryption Date: Fri, 20 Apr 2018 16:30:02 -0700 Message-Id: <20180420233002.134687-1-ebiggers@google.com> X-Mailer: git-send-email 2.17.0.484.g0c8726318c-goog Sender: linux-fscrypt-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fscrypt@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Improve fscrypt read performance by switching the decryption workqueue from bound to unbound. With the bound workqueue, when multiple bios completed on the same CPU, they were decrypted on that same CPU. But with the unbound queue, they are now decrypted in parallel on any CPU. Although fscrypt read performance can be tough to measure due to the many sources of variation, this change is most beneficial when decryption is slow, e.g. on CPUs without AES instructions. For example, I timed tarring up encrypted directories on f2fs. On x86 with AES-NI instructions disabled, the unbound workqueue improved performance by about 25-35%, using 1 to NUM_CPUs jobs with 4 or 8 CPUs available. But with AES-NI enabled, performance was unchanged to within ~2%. I also did the same test on a quad-core ARM CPU using xts-speck128-neon encryption. There performance was usually about 10% better with the unbound workqueue, bringing it closer to the unencrypted speed. The unbound workqueue may be worse in some cases due to worse locality, but I think it's still the better default. dm-crypt uses an unbound workqueue by default too, so this change makes fscrypt match. Signed-off-by: Eric Biggers --- fs/crypto/crypto.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/fs/crypto/crypto.c b/fs/crypto/crypto.c index ce654526c0fb..984e190f9b89 100644 --- a/fs/crypto/crypto.c +++ b/fs/crypto/crypto.c @@ -427,8 +427,17 @@ int fscrypt_initialize(unsigned int cop_flags) */ static int __init fscrypt_init(void) { + /* + * Use an unbound workqueue to allow bios to be decrypted in parallel + * even when they happen to complete on the same CPU. This sacrifices + * locality, but it's worthwhile since decryption is CPU-intensive. + * + * Also use a high-priority workqueue to prioritize decryption work, + * which blocks reads from completing, over regular application tasks. + */ fscrypt_read_workqueue = alloc_workqueue("fscrypt_read_queue", - WQ_HIGHPRI, 0); + WQ_UNBOUND | WQ_HIGHPRI, + num_online_cpus()); if (!fscrypt_read_workqueue) goto fail;