From patchwork Wed Aug 28 22:14:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 11120045 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 954CB184E for ; Wed, 28 Aug 2019 22:18:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 69FCC2339E for ; Wed, 28 Aug 2019 22:18:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="lZTMkDlh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727438AbfH1WS4 (ORCPT ); Wed, 28 Aug 2019 18:18:56 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:40564 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726839AbfH1WS4 (ORCPT ); Wed, 28 Aug 2019 18:18:56 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7SMHIY2115541; Wed, 28 Aug 2019 22:18:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2019-08-05; bh=q8UPrkMM/CujGokBaxFV4506SVYZkFz/veRW9ukkyvA=; b=lZTMkDlh6Dr2ZxANHoruFlDG3UO6nz6SA2jvkXZX2kksbMz1s9ypxfP2yHnA/ov4gUIm gQWGTTSm5TSZ6Ok6vL+7cQU7Cw8dib2QvT42jBC2ezuAlGr6W76WQMYgyzOD6cyC3bHZ UVfPu1+BE5jKGWozX9KbPDEvq6G8LmBN6MhIE1n+VnGOVs7V8v69r3uXmBWWKnLlDrXK z4laFGxF6GxmCTLAXad8ak0JZYkBbYOpOUyMKpVzoHOBbAYtiNCNUmZEFKTB6XLnnj7Z imZAdWncEZ8fYWV1DBoPgTIOtkzsSfTFp2vFneMw5RsTGtiTjqqvgVQU8CLJzrFLTqQo Qg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2130.oracle.com with ESMTP id 2up296g0h1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Aug 2019 22:18:32 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7SMDOiP007293; Wed, 28 Aug 2019 22:16:32 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3020.oracle.com with ESMTP id 2untetyqh2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Aug 2019 22:16:31 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x7SMGU3i020531; Wed, 28 Aug 2019 22:16:30 GMT Received: from zissou.us.oracle.com (/10.152.34.58) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Aug 2019 15:16:30 -0700 From: Daniel Jordan To: Herbert Xu , Steffen Klassert Cc: Sebastian Andrzej Siewior , Thomas Gleixner , linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Jordan Subject: [PATCH v2 1/5] padata: make flushing work with async users Date: Wed, 28 Aug 2019 18:14:21 -0400 Message-Id: <20190828221425.22701-2-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190828221425.22701-1-daniel.m.jordan@oracle.com> References: <20190828221425.22701-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908280214 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908280214 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org padata_flush_queues() is broken for an async ->parallel() function because flush_work() can't wait on it: # modprobe tcrypt alg="pcrypt(cryptd(rfc4106(gcm_base(ctr(aes-generic),ghash-generic))))" type=3 # modprobe tcrypt mode=215 sec=1 & # sleep 5; echo 7 > /sys/kernel/pcrypt/pencrypt/parallel_cpumask kernel BUG at kernel/padata.c:473! invalid opcode: 0000 [#1] SMP PTI CPU: 0 PID: 282 Comm: sh Not tainted 5.3.0-rc5-padata-base+ #3 RIP: 0010:padata_flush_queues+0xa7/0xb0 Call Trace: padata_replace+0xa1/0x110 padata_set_cpumask+0xae/0x130 store_cpumask+0x75/0xa0 padata_sysfs_store+0x20/0x30 ... Wait instead for the parallel_data (pd) object's refcount to drop to zero. Simplify by taking an initial reference on a pd when allocating an instance. That ref is dropped during flushing, which allows calling wait_for_completion() unconditionally. If the initial ref weren't used, the only other alternative I could think of is that complete() would be conditional on !PADATA_INIT or PADATA_REPLACE (in addition to zero pd->refcnt), and wait_for_completion() on nonzero pd->refcnt. But then complete() could be called without a matching wait: completer waiter --------- ------ DEC pd->refcnt // 0 pinst->flags |= PADATA_REPLACE LOAD pinst->flags // REPLACE LOAD pd->refcnt // 0 /* return without wait_for_completion() */ complete() No more flushing per-CPU work items, so no more CPU hotplug lock in __padata_stop. Fixes: 2b73b07ab8a4 ("padata: Flush the padata queues actively") Reported-by: Herbert Xu Suggested-by: Steffen Klassert Signed-off-by: Daniel Jordan Cc: Sebastian Andrzej Siewior Cc: Thomas Gleixner Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- include/linux/padata.h | 3 +++ kernel/padata.c | 32 ++++++++++++-------------------- 2 files changed, 15 insertions(+), 20 deletions(-) diff --git a/include/linux/padata.h b/include/linux/padata.h index 8da673861d99..1c73f9cc7b72 100644 --- a/include/linux/padata.h +++ b/include/linux/padata.h @@ -14,6 +14,7 @@ #include #include #include +#include #define PADATA_CPU_SERIAL 0x01 #define PADATA_CPU_PARALLEL 0x02 @@ -104,6 +105,7 @@ struct padata_cpumask { * @squeue: percpu padata queues used for serialuzation. * @reorder_objects: Number of objects waiting in the reorder queues. * @refcnt: Number of objects holding a reference on this parallel_data. + * @flushing_done: Wait for all objects to be sent out. * @max_seq_nr: Maximal used sequence number. * @cpu: Next CPU to be processed. * @cpumask: The cpumasks in use for parallel and serial workers. @@ -116,6 +118,7 @@ struct parallel_data { struct padata_serial_queue __percpu *squeue; atomic_t reorder_objects; atomic_t refcnt; + struct completion flushing_done; atomic_t seq_nr; int cpu; struct padata_cpumask cpumask; diff --git a/kernel/padata.c b/kernel/padata.c index b60cc3dcee58..958166e23123 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -301,7 +301,8 @@ static void padata_serial_worker(struct work_struct *serial_work) list_del_init(&padata->list); padata->serial(padata); - atomic_dec(&pd->refcnt); + if (atomic_dec_return(&pd->refcnt) == 0) + complete(&pd->flushing_done); } local_bh_enable(); } @@ -423,7 +424,9 @@ static struct parallel_data *padata_alloc_pd(struct padata_instance *pinst, padata_init_squeues(pd); atomic_set(&pd->seq_nr, -1); atomic_set(&pd->reorder_objects, 0); - atomic_set(&pd->refcnt, 0); + /* Initial ref dropped in padata_flush_queues. */ + atomic_set(&pd->refcnt, 1); + init_completion(&pd->flushing_done); pd->pinst = pinst; spin_lock_init(&pd->lock); pd->cpu = cpumask_first(pd->cpumask.pcpu); @@ -453,24 +456,15 @@ static void padata_free_pd(struct parallel_data *pd) /* Flush all objects out of the padata queues. */ static void padata_flush_queues(struct parallel_data *pd) { - int cpu; - struct padata_parallel_queue *pqueue; - struct padata_serial_queue *squeue; - - for_each_cpu(cpu, pd->cpumask.pcpu) { - pqueue = per_cpu_ptr(pd->pqueue, cpu); - flush_work(&pqueue->work); - } - - if (atomic_read(&pd->reorder_objects)) - padata_reorder(pd); + if (!(pd->pinst->flags & PADATA_INIT)) + return; - for_each_cpu(cpu, pd->cpumask.cbcpu) { - squeue = per_cpu_ptr(pd->squeue, cpu); - flush_work(&squeue->work); - } + if (atomic_dec_return(&pd->refcnt) == 0) + complete(&pd->flushing_done); - BUG_ON(atomic_read(&pd->refcnt) != 0); + wait_for_completion(&pd->flushing_done); + reinit_completion(&pd->flushing_done); + atomic_set(&pd->refcnt, 1); } static void __padata_start(struct padata_instance *pinst) @@ -487,9 +481,7 @@ static void __padata_stop(struct padata_instance *pinst) synchronize_rcu(); - get_online_cpus(); padata_flush_queues(pinst->pd); - put_online_cpus(); } /* Replace the internal control structure with a new one. */ From patchwork Wed Aug 28 22:14:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 11120265 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6335018EC for ; Thu, 29 Aug 2019 02:46:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 423492173E for ; Thu, 29 Aug 2019 02:46:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="VXXiIf7/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725886AbfH2CpI (ORCPT ); Wed, 28 Aug 2019 22:45:08 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:46730 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbfH2CpI (ORCPT ); Wed, 28 Aug 2019 22:45:08 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7T2ie4d110352; Thu, 29 Aug 2019 02:44:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2019-08-05; bh=vuXYwl4+3gT1SOY2ggdFx5WMHHwn7038nvwRZyo/0PA=; b=VXXiIf7/w+pYHuyjIGgXRxxNT/6DGj8fLCYs39dpD7ma9jEcbnx3komF1t/CvJWbuQSz ekV0RkK9Ltqm1oW18CnJLXsdblJ9b/aU39g6XZGtlSnAvLKvT7mA9cPz7uYuC1XZi4U2 W7bRwwbE7VZ0xYXyxzm50kojF6OQr5WEKDLuUcC52eRwPt6I4NoO5p0zPWbViU0HngB5 EH6Q9/Y4PPmRn+IOIeblKtvqobA0LtXSn7Ov9Ac9LRouel/4tw0RMVXzsbQZ/Jgv1LOU T1Tbm8dMMx8RqfQO9roY4l/xISn4J89z03hGXL7UGQ8BjkdRYp0etul1SpUYTqhUZa3u YQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 2up62pg0np-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 29 Aug 2019 02:44:52 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7SMDLif011581; Wed, 28 Aug 2019 22:16:32 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3030.oracle.com with ESMTP id 2unvtxxk9w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Aug 2019 22:16:31 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x7SMGVLk020533; Wed, 28 Aug 2019 22:16:31 GMT Received: from zissou.us.oracle.com (/10.152.34.58) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Aug 2019 15:16:30 -0700 From: Daniel Jordan To: Herbert Xu , Steffen Klassert Cc: Sebastian Andrzej Siewior , Thomas Gleixner , linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Jordan Subject: [PATCH v2 2/5] padata: remove reorder_objects Date: Wed, 28 Aug 2019 18:14:22 -0400 Message-Id: <20190828221425.22701-3-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190828221425.22701-1-daniel.m.jordan@oracle.com> References: <20190828221425.22701-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908280214 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908290029 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org reorder_objects is unused since the rework of padata's flushing, so remove it. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Sebastian Andrzej Siewior Cc: Steffen Klassert Cc: Thomas Gleixner Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- include/linux/padata.h | 2 -- kernel/padata.c | 3 --- 2 files changed, 5 deletions(-) diff --git a/include/linux/padata.h b/include/linux/padata.h index 1c73f9cc7b72..720e970cda0a 100644 --- a/include/linux/padata.h +++ b/include/linux/padata.h @@ -103,7 +103,6 @@ struct padata_cpumask { * @pinst: padata instance. * @pqueue: percpu padata queues used for parallelization. * @squeue: percpu padata queues used for serialuzation. - * @reorder_objects: Number of objects waiting in the reorder queues. * @refcnt: Number of objects holding a reference on this parallel_data. * @flushing_done: Wait for all objects to be sent out. * @max_seq_nr: Maximal used sequence number. @@ -116,7 +115,6 @@ struct parallel_data { struct padata_instance *pinst; struct padata_parallel_queue __percpu *pqueue; struct padata_serial_queue __percpu *squeue; - atomic_t reorder_objects; atomic_t refcnt; struct completion flushing_done; atomic_t seq_nr; diff --git a/kernel/padata.c b/kernel/padata.c index 958166e23123..2bfce01c5b85 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -179,7 +179,6 @@ static struct padata_priv *padata_get_next(struct parallel_data *pd) struct padata_priv, list); list_del_init(&padata->list); - atomic_dec(&pd->reorder_objects); pd->cpu = cpumask_next_wrap(cpu, pd->cpumask.pcpu, -1, false); @@ -323,7 +322,6 @@ void padata_do_serial(struct padata_priv *padata) spin_lock(&pqueue->reorder.lock); list_add_tail(&padata->list, &pqueue->reorder.list); - atomic_inc(&pd->reorder_objects); spin_unlock(&pqueue->reorder.lock); /* @@ -423,7 +421,6 @@ static struct parallel_data *padata_alloc_pd(struct padata_instance *pinst, padata_init_pqueues(pd); padata_init_squeues(pd); atomic_set(&pd->seq_nr, -1); - atomic_set(&pd->reorder_objects, 0); /* Initial ref dropped in padata_flush_queues. */ atomic_set(&pd->refcnt, 1); init_completion(&pd->flushing_done); From patchwork Wed Aug 28 22:14:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 11120025 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A136B14D5 for ; Wed, 28 Aug 2019 22:16:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 75ADC22DA7 for ; Wed, 28 Aug 2019 22:16:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="HYa37UCk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726882AbfH1WQ5 (ORCPT ); Wed, 28 Aug 2019 18:16:57 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:57500 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726658AbfH1WQ4 (ORCPT ); Wed, 28 Aug 2019 18:16:56 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7SMEM0p120994; Wed, 28 Aug 2019 22:16:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2019-08-05; bh=PGYZg8dmqranewG63iKkOm/uGgVLWKTGrL23VDeFueU=; b=HYa37UCk9eYYThbbG6tR/5SUpBgGbIyfEfWlVD4Hf73AKELX42EgZPDizhrLpS4SrdX/ 3+MtYmiGN8eD9aLLvTlauWPjy4+MWyhB+tFKnV1BQv+qkDjCBuDjad35UsgnTcmeAHre hnVo+WBBhwA2ieDS56cts1VLWeAyfJWSdsoT5icBrA25cak41bfesiN5bvtQilUFPS23 9bQ0W+1OD0krKvuSyRCnfuUFhn+goGqaiTWHY7CL4/Oj+Oh5/ukvb8kW0sHFjQU2o3zj Snd8LT9ws84aFPyLCJN6h6JVheSEb7ENaayp4XbC6XxUmu7w+sPGcxqYOFeV1W4S7zK4 jQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2up25500s1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Aug 2019 22:16:34 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7SMDKrE011415; Wed, 28 Aug 2019 22:16:34 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3030.oracle.com with ESMTP id 2unvtxxkb6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Aug 2019 22:16:33 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x7SMGVg8031895; Wed, 28 Aug 2019 22:16:31 GMT Received: from zissou.us.oracle.com (/10.152.34.58) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Aug 2019 15:16:31 -0700 From: Daniel Jordan To: Herbert Xu , Steffen Klassert Cc: Sebastian Andrzej Siewior , Thomas Gleixner , linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Jordan , Jonathan Corbet , linux-doc@vger.kernel.org Subject: [PATCH v2 3/5] padata: get rid of padata_remove_cpu() for real Date: Wed, 28 Aug 2019 18:14:23 -0400 Message-Id: <20190828221425.22701-4-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190828221425.22701-1-daniel.m.jordan@oracle.com> References: <20190828221425.22701-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908280214 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908280214 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org A later patch is going to address a lock ordering issue involving pinst->mutex and the CPU hotplug lock. padata_remove_cpu() needs fixing but it has no callers, so just delete it rather than maintaining unused code. The Fixes commit forgot to do it anyway. While at it remove Documentation references to other unused functions. Fixes: 815613da6a67 ("kernel/padata.c: removed unused code") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Jonathan Corbet Cc: Sebastian Andrzej Siewior Cc: Steffen Klassert Cc: Thomas Gleixner Cc: linux-crypto@vger.kernel.org Cc: linux-doc@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- Documentation/padata.txt | 18 ++++-------------- kernel/padata.c | 35 ----------------------------------- 2 files changed, 4 insertions(+), 49 deletions(-) diff --git a/Documentation/padata.txt b/Documentation/padata.txt index b103d0c82000..43ca928da713 100644 --- a/Documentation/padata.txt +++ b/Documentation/padata.txt @@ -51,27 +51,17 @@ padata cpumask contains no active CPU (flag not set). padata_stop clears the flag and blocks until the padata instance is unused. -The list of CPUs to be used can be adjusted with these functions:: +The list of CPUs to be used can be adjusted with this function:: - int padata_set_cpumasks(struct padata_instance *pinst, - cpumask_var_t pcpumask, - cpumask_var_t cbcpumask); int padata_set_cpumask(struct padata_instance *pinst, int cpumask_type, cpumask_var_t cpumask); - int padata_add_cpu(struct padata_instance *pinst, int cpu, int mask); - int padata_remove_cpu(struct padata_instance *pinst, int cpu, int mask); Changing the CPU masks are expensive operations, though, so it should not be done with great frequency. -It's possible to change both cpumasks of a padata instance with -padata_set_cpumasks by specifying the cpumasks for parallel execution (pcpumask) -and for the serial callback function (cbcpumask). padata_set_cpumask is used to -change just one of the cpumasks. Here cpumask_type is one of PADATA_CPU_SERIAL, -PADATA_CPU_PARALLEL and cpumask specifies the new cpumask to use. -To simply add or remove one CPU from a certain cpumask the functions -padata_add_cpu/padata_remove_cpu are used. cpu specifies the CPU to add or -remove and mask is one of PADATA_CPU_SERIAL, PADATA_CPU_PARALLEL. +padata_set_cpumask is used to change just one of the cpumasks. Here cpumask_type +is one of PADATA_CPU_SERIAL or PADATA_CPU_PARALLEL, and cpumask specifies the +new cpumask to use. If a user is interested in padata cpumask changes, he can register to the padata cpumask change notifier:: diff --git a/kernel/padata.c b/kernel/padata.c index 2bfce01c5b85..6adce3b203fe 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -710,41 +710,6 @@ static int __padata_remove_cpu(struct padata_instance *pinst, int cpu) return 0; } - /** - * padata_remove_cpu - remove a cpu from the one or both(serial and parallel) - * padata cpumasks. - * - * @pinst: padata instance - * @cpu: cpu to remove - * @mask: bitmask specifying from which cpumask @cpu should be removed - * The @mask may be any combination of the following flags: - * PADATA_CPU_SERIAL - serial cpumask - * PADATA_CPU_PARALLEL - parallel cpumask - */ -int padata_remove_cpu(struct padata_instance *pinst, int cpu, int mask) -{ - int err; - - if (!(mask & (PADATA_CPU_SERIAL | PADATA_CPU_PARALLEL))) - return -EINVAL; - - mutex_lock(&pinst->lock); - - get_online_cpus(); - if (mask & PADATA_CPU_SERIAL) - cpumask_clear_cpu(cpu, pinst->cpumask.cbcpu); - if (mask & PADATA_CPU_PARALLEL) - cpumask_clear_cpu(cpu, pinst->cpumask.pcpu); - - err = __padata_remove_cpu(pinst, cpu); - put_online_cpus(); - - mutex_unlock(&pinst->lock); - - return err; -} -EXPORT_SYMBOL(padata_remove_cpu); - static inline int pinst_has_cpu(struct padata_instance *pinst, int cpu) { return cpumask_test_cpu(cpu, pinst->cpumask.pcpu) || From patchwork Wed Aug 28 22:14:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 11120107 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90EC11395 for ; Thu, 29 Aug 2019 00:13:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6FBC422CED for ; Thu, 29 Aug 2019 00:13:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="sctmNZVM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726828AbfH2ANx (ORCPT ); Wed, 28 Aug 2019 20:13:53 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:48630 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726825AbfH2ANw (ORCPT ); Wed, 28 Aug 2019 20:13:52 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7T0DNVB005044; Thu, 29 Aug 2019 00:13:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2019-08-05; bh=7wG5y7dKyypT4n5VfzMGBXRCZUmNViG9gJEJUq/OhLo=; b=sctmNZVMbDTpDSLMErdcAI8t4Py/Pltzk73Oud6ZLkclBxdvkwOGuhTgeDT3au9c7tBr 7ujUB8iiDY78b9DvgG8GbPKW+DSI3hs52BrtklO2I6WR2G47pA/7d9TL/pNsEGrpcZLn DojgCf+4T23dJABggFF4eVPXRwWmEkqNoWc2MtdfdZh2bxFNvzMDZV0YiZzBjgLnT6M2 Qqvyxh+BQg+b4buhdTbRMNSPIbxjB37Z3VSxk3eFD0Wh3e+lGbhwlG/r+X+jZj1Y0Zps w62kPSVnyZVKo1iE+CLFi6ukyKDzfWBE2mHJ5WdHtJDQ2pv6whZ0S2O7dFORd/hbzuJh ww== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 2up3vmr0k6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 29 Aug 2019 00:13:35 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7SMDOC0007324; Wed, 28 Aug 2019 22:16:33 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3020.oracle.com with ESMTP id 2untetyqhw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Aug 2019 22:16:33 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x7SMGWex020546; Wed, 28 Aug 2019 22:16:32 GMT Received: from zissou.us.oracle.com (/10.152.34.58) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Aug 2019 15:16:32 -0700 From: Daniel Jordan To: Herbert Xu , Steffen Klassert Cc: Sebastian Andrzej Siewior , Thomas Gleixner , linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Jordan Subject: [PATCH v2 4/5] padata: always acquire cpu_hotplug_lock before pinst->lock Date: Wed, 28 Aug 2019 18:14:24 -0400 Message-Id: <20190828221425.22701-5-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190828221425.22701-1-daniel.m.jordan@oracle.com> References: <20190828221425.22701-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908280214 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908290001 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org lockdep complains when... # echo 0 > /sys/devices/system/cpu/cpu1/online # echo ff > /sys/kernel/pcrypt/pencrypt/parallel_cpumask ====================================================== WARNING: possible circular locking dependency detected 5.3.0-rc5-padata-base+ #6 Not tainted ------------------------------------------------------ bash/258 is trying to acquire lock: 00000000c43f7f29 (cpu_hotplug_lock.rw_sem){++++}, at: padata_set_cpumask+0x30/0x130 but task is already holding lock: 00000000676aa31d (&pinst->lock){+.+.}, at: padata_set_cpumask+0x2b/0x130 which lock already depends on the new lock. padata doesn't take cpu_hotplug_lock and pinst->lock in a consistent order. Which should be first? CPU hotplug calls into padata with cpu_hotplug_lock already held, so it should have priority. Fixes: 6751fb3c0e0c ("padata: Use get_online_cpus/put_online_cpus") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Sebastian Andrzej Siewior Cc: Steffen Klassert Cc: Thomas Gleixner Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- kernel/padata.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/padata.c b/kernel/padata.c index 6adce3b203fe..75e668fedd8d 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -603,8 +603,8 @@ int padata_set_cpumask(struct padata_instance *pinst, int cpumask_type, struct cpumask *serial_mask, *parallel_mask; int err = -EINVAL; - mutex_lock(&pinst->lock); get_online_cpus(); + mutex_lock(&pinst->lock); switch (cpumask_type) { case PADATA_CPU_PARALLEL: @@ -622,8 +622,8 @@ int padata_set_cpumask(struct padata_instance *pinst, int cpumask_type, err = __padata_set_cpumasks(pinst, parallel_mask, serial_mask); out: - put_online_cpus(); mutex_unlock(&pinst->lock); + put_online_cpus(); return err; } From patchwork Wed Aug 28 22:14:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 11120117 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 700E714D5 for ; Thu, 29 Aug 2019 00:39:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 39C2D2070B for ; Thu, 29 Aug 2019 00:39:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="jDZII5f4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725941AbfH2Ai7 (ORCPT ); Wed, 28 Aug 2019 20:38:59 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:42710 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725892AbfH2Ai7 (ORCPT ); Wed, 28 Aug 2019 20:38:59 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7T0YCx8019592; Thu, 29 Aug 2019 00:38:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2019-08-05; bh=07esnCh9Q7ADyFw64RmGaCzhnJGUf5VXTz4CoNgBoY0=; b=jDZII5f4JWrOMpu6JAChABgsFojRLGR5zWAsUAMQyvP8Vyd2dQSwNTMhsgFy1/EpsT9a n8DkDwsiGZymjRczYdF+vBp0eoR7ruEzFhS2bmVDURWJOcIQAT5tAhaQJI9n9gAuX8gj 7mEFSy3nSJ1ymXuCJ75XHD/YhQrK6HR0FKYXlzo6Sh4Zol9lmMVoMkXoAKJRdYVHrCK1 XzU2Gbu1n3F9qWfjB5tLlnHzqYplw2gWgvB+3Z0h8jy0g+lzk5Kug7T9xNejSykX/qln ewvalVip+fe4jOmhSnr7YzttspTiUYpJJhQFIh9SKMspdorNDva3TD6jyHlWpp6O8/uW tA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 2up44h817e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 29 Aug 2019 00:38:42 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x7SMEUID043190; Wed, 28 Aug 2019 22:16:33 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 2undw7xt8m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Aug 2019 22:16:33 +0000 Received: from abhmp0010.oracle.com (abhmp0010.oracle.com [141.146.116.16]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x7SMGXno031903; Wed, 28 Aug 2019 22:16:33 GMT Received: from zissou.us.oracle.com (/10.152.34.58) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 28 Aug 2019 15:16:32 -0700 From: Daniel Jordan To: Herbert Xu , Steffen Klassert Cc: Sebastian Andrzej Siewior , Thomas Gleixner , linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, Daniel Jordan Subject: [PATCH v2 5/5] padata: validate cpumask without removed CPU during offline Date: Wed, 28 Aug 2019 18:14:25 -0400 Message-Id: <20190828221425.22701-6-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190828221425.22701-1-daniel.m.jordan@oracle.com> References: <20190828221425.22701-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908280214 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9363 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908290003 Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Configuring an instance's parallel mask without any online CPUs... echo 2 > /sys/kernel/pcrypt/pencrypt/parallel_cpumask echo 0 > /sys/devices/system/cpu/cpu1/online ...makes tcrypt mode=215 crash like this: divide error: 0000 [#1] SMP PTI CPU: 4 PID: 287 Comm: modprobe Not tainted 5.3.0-rc5-padata-base+ #7 RIP: 0010:padata_do_parallel+0xfd/0x290 Call Trace: pcrypt_do_parallel+0xed/0x1e0 [pcrypt] pcrypt_aead_encrypt+0xbf/0xd0 [pcrypt] crypto_aead_encrypt+0x1f/0x30 do_mult_aead_op+0x4e/0xdf [tcrypt] test_mb_aead_speed.constprop.0.cold+0x226/0x564 [tcrypt] do_test+0x2280/0x4c16 [tcrypt] tcrypt_mod_init+0x55/0x1000 [tcrypt] ... cpumask_weight() in padata_cpu_hash() returns 0 because the mask has no CPUs. The problem is __padata_remove_cpu() checks for valid masks too early and so doesn't mark the instance PADATA_INVALID as expected, which would have made padata_do_parallel() return error before doing the division. Fix by introducing a second padata CPU hotplug state before CPUHP_BRINGUP_CPU so that __padata_remove_cpu() sees the online mask without @cpu. No need for the cpumask_clear_cpu()s since @cpu is already missing from those masks. Fixes: 33e54450683c ("padata: Handle empty padata cpumasks") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Sebastian Andrzej Siewior Cc: Steffen Klassert Cc: Thomas Gleixner Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- include/linux/cpuhotplug.h | 1 + kernel/padata.c | 21 ++++++++++++++------- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 068793a619ca..2d55cee638fc 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -59,6 +59,7 @@ enum cpuhp_state { CPUHP_IOMMU_INTEL_DEAD, CPUHP_LUSTRE_CFS_DEAD, CPUHP_AP_ARM_CACHE_B15_RAC_DEAD, + CPUHP_PADATA_DEAD, CPUHP_WORKQUEUE_PREP, CPUHP_POWER_NUMA_PREPARE, CPUHP_HRTIMERS_PREPARE, diff --git a/kernel/padata.c b/kernel/padata.c index 75e668fedd8d..5325e5978ee4 100644 --- a/kernel/padata.c +++ b/kernel/padata.c @@ -690,7 +690,7 @@ static int __padata_remove_cpu(struct padata_instance *pinst, int cpu) { struct parallel_data *pd = NULL; - if (cpumask_test_cpu(cpu, cpu_online_mask)) { + if (!cpumask_test_cpu(cpu, cpu_online_mask)) { if (!padata_validate_cpumask(pinst, pinst->cpumask.pcpu) || !padata_validate_cpumask(pinst, pinst->cpumask.cbcpu)) @@ -702,9 +702,6 @@ static int __padata_remove_cpu(struct padata_instance *pinst, int cpu) return -ENOMEM; padata_replace(pinst, pd); - - cpumask_clear_cpu(cpu, pd->cpumask.cbcpu); - cpumask_clear_cpu(cpu, pd->cpumask.pcpu); } return 0; @@ -731,7 +728,7 @@ static int padata_cpu_online(unsigned int cpu, struct hlist_node *node) return ret; } -static int padata_cpu_prep_down(unsigned int cpu, struct hlist_node *node) +static int padata_cpu_dead(unsigned int cpu, struct hlist_node *node) { struct padata_instance *pinst; int ret; @@ -752,6 +749,7 @@ static enum cpuhp_state hp_online; static void __padata_free(struct padata_instance *pinst) { #ifdef CONFIG_HOTPLUG_CPU + cpuhp_state_remove_instance_nocalls(CPUHP_PADATA_DEAD, &pinst->node); cpuhp_state_remove_instance_nocalls(hp_online, &pinst->node); #endif @@ -938,6 +936,8 @@ static struct padata_instance *padata_alloc(struct workqueue_struct *wq, #ifdef CONFIG_HOTPLUG_CPU cpuhp_state_add_instance_nocalls_cpuslocked(hp_online, &pinst->node); + cpuhp_state_add_instance_nocalls_cpuslocked(CPUHP_PADATA_DEAD, + &pinst->node); #endif return pinst; @@ -984,17 +984,24 @@ static __init int padata_driver_init(void) int ret; ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "padata:online", - padata_cpu_online, - padata_cpu_prep_down); + padata_cpu_online, NULL); if (ret < 0) return ret; hp_online = ret; + + ret = cpuhp_setup_state_multi(CPUHP_PADATA_DEAD, "padata:dead", + NULL, padata_cpu_dead); + if (ret < 0) { + cpuhp_remove_multi_state(hp_online); + return ret; + } return 0; } module_init(padata_driver_init); static __exit void padata_driver_exit(void) { + cpuhp_remove_multi_state(CPUHP_PADATA_DEAD); cpuhp_remove_multi_state(hp_online); } module_exit(padata_driver_exit);