From patchwork Thu Apr 18 05:43:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634198 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1121.securemx.jp [210.130.202.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0B255B208; Thu, 18 Apr 2024 06:19:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.129 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421149; cv=none; b=DswCBlb3LzUtu1f6H7FwfuVJpob1oDHyQ+nXS5Xrb9PsTySJD5MrsxnMv12vwwsswHJHb45Ut2wVWoV2FzeYiHCuuikqvyUH9DNr7ogdbGBTabVnJ1D39n8DbkwceT73TZ7j2JIv2xR2BAooYBwQRqWp97KORAB7NzG4qdlpMDE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713421149; c=relaxed/simple; bh=iqhSMZOztFGqdMfh57SBwjiZxFAblcAJChR5FSzrOvA=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=gZJPd4sR8Y1fP+iuyhAvY5V6RY5pZwRzA6bLpIInfcfuwdqFztSRf3yi8veqZE/kKd2D7azByYGEAQQKkTkWqzhn9wEcJI+sKU3FLs4rplDHGUNOaxR19SznQjWi/7Jx5d2gemX5Vz+ufKCNZ4TusJAtPcY7GnMw8cXNEh6Qec0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.129 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1121) id 43I5iTq72031331; Thu, 18 Apr 2024 14:44:29 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 43I5hxVQ2464871; Thu, 18 Apr 2024 14:44:00 +0900 X-Iguazu-Qid: 2rWhYYPQE6csxVxb9A X-Iguazu-QSIG: v=2; s=0; t=1713419039; q=2rWhYYPQE6csxVxb9A; m=ugGMhLoVwFKiNOp51ehxd5ki18LT8ISPHPFwj4Fq0Gg= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1122) id 43I5hwFk2851571 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:43:59 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 850ED2FE0F; Thu, 18 Apr 2024 14:43:58 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id CD9E81900001EB; Thu, 18 Apr 2024 14:31:15 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003QAAAAAA021GJ; Thu, 18 Apr 2024 14:31:15 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id C25BCA0640E18; Thu, 18 Apr 2024 14:31:15 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:43:57 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:43:57 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 01/11] md: add infra for active_aligned_reads changes Thread-Topic: [PATCH v2 01/11] md: add infra for active_aligned_reads changes Thread-Index: AdqLx9n45Jz8xtHMQ62+pkfRbuv/qg== Date: Thu, 18 Apr 2024 05:43:57 +0000 Message-ID: Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--1.684500-8.000000 x-tmase-matchedrid: qiJ73D9abaWmagT1k9kBppTQgFTHgkhZKQNhMboqZlrHhoBEyuHcbYUl JlxlTIT+ewNtMWg+aOa12HagvbwDji/7QU2czuUNA9lly13c/gE2SSdNEgI3sEz5vzLEGq8DTTC YELjM+CVvF9oQ/B54yFo4rfBbwEOR23JrSYCBAceJI2OHymiAOytc78OfBHnLGTX5zLczNzU5d9 n04fLNZlepMMW3nEo1fpJkaQdvaIEGzl9AHE05zZ4CIKY/Hg3AcmfM3DjaQLHEQdG7H66TyKsQd 9qPXhnJ/4rWvpj9UcjHPy5LhFKP1Mrhybo4rxmZmm5wpQj0cWzQvUxRY7ho0BZyUZ/veE38bbqZ gQU59nN2zGeYx7YxxEtpJMKtiIn0UFhuhyAg+DoVrg5c4Vk8DXApsmbw8EMtreWoUbwyBi2trdV Vx3oRXuqhTICYOFJ/ x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--1.684500-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: AB58909CB39702EAD9C4D99C2765CE33FA42D16BAC07815109FBD20A7CE368952000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com Prepare to smoothly change the type of active_aligned_reads from atomic_t to percpu_ref. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/raid5.c | 12 ++++++------ drivers/md/raid5.h | 20 ++++++++++++++++++++ 2 files changed, 26 insertions(+), 6 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 2bd1ce9b3922..3b04d8b526b1 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5435,7 +5435,7 @@ static void raid5_align_endio(struct bio *bi) if (!error) { bio_endio(raid_bi); - if (atomic_dec_and_test(&conf->active_aligned_reads)) + if (active_aligned_reads_dec_and_test(conf)) wake_up(&conf->wait_for_quiescent); return; } @@ -5499,7 +5499,7 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) did_inc = false; if (conf->quiesce == 0) { - atomic_inc(&conf->active_aligned_reads); + active_aligned_reads_inc(conf); did_inc = true; } /* need a memory barrier to detect the race with raid5_quiesce() */ @@ -5507,12 +5507,12 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) /* quiesce is in progress, so we need to undo io activation and wait * for it to finish */ - if (did_inc && atomic_dec_and_test(&conf->active_aligned_reads)) + if (did_inc && active_aligned_reads_dec_and_test(conf)) wake_up(&conf->wait_for_quiescent); spin_lock_irq(&conf->device_lock); wait_event_lock_irq(conf->wait_for_quiescent, conf->quiesce == 0, conf->device_lock); - atomic_inc(&conf->active_aligned_reads); + active_aligned_reads_inc(conf); spin_unlock_irq(&conf->device_lock); } @@ -6608,7 +6608,7 @@ static int retry_aligned_read(struct r5conf *conf, struct bio *raid_bio, bio_endio(raid_bio); - if (atomic_dec_and_test(&conf->active_aligned_reads)) + if (active_aligned_reads_dec_and_test(conf)) wake_up(&conf->wait_for_quiescent); return handled; } @@ -8586,7 +8586,7 @@ static void raid5_quiesce(struct mddev *mddev, int quiesce) smp_store_release(&conf->quiesce, 2); wait_event_cmd(conf->wait_for_quiescent, atomic_read(&conf->active_stripes) == 0 && - atomic_read(&conf->active_aligned_reads) == 0, + active_aligned_reads_is_zero(conf), unlock_all_device_hash_locks_irq(conf), lock_all_device_hash_locks_irq(conf)); conf->quiesce = 1; diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h index 9b5a7dc3f2a0..5bd6bb3540c5 100644 --- a/drivers/md/raid5.h +++ b/drivers/md/raid5.h @@ -696,6 +696,26 @@ struct r5conf { struct r5pending_data *next_pending_data; }; +static inline void active_aligned_reads_inc(struct r5conf *conf) +{ + atomic_inc(&conf->active_aligned_reads); +} + +static inline void active_aligned_reads_dec(struct r5conf *conf) +{ + atomic_dec(&conf->active_aligned_reads); +} + +static inline bool active_aligned_reads_is_zero(struct r5conf *conf) +{ + return atomic_read(&conf->active_aligned_reads) == 0; +} + +static inline bool active_aligned_reads_dec_and_test(struct r5conf *conf) +{ + return atomic_dec_and_test(&conf->active_aligned_reads); +} + #if PAGE_SIZE == DEFAULT_STRIPE_SIZE #define RAID5_STRIPE_SIZE(conf) STRIPE_SIZE #define RAID5_STRIPE_SHIFT(conf) STRIPE_SHIFT From patchwork Thu Apr 18 05:44:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634250 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 947C910A0E; Thu, 18 Apr 2024 07:21:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713424903; cv=none; b=dNIWynuJXFLXfvtxPamd6Kg6+i4LP4Ti3Yc24sIG/TOh0WmKRIx4+8M4Sj1UT821XNHRt7cQafIHoyRfjxL6mmNSQHYQGFMQXMpTKtcmrieEgrlrUYuXcfU6ZdG99XAxAuX1Q5Uv9RzZqjMzhORt+kpBAUfjAZs3VgQMiAOCFjU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713424903; c=relaxed/simple; bh=q8ht+jAF0alGe432i95E2EmmvgvQ5IhN3fsmHdLp8tg=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=k55zEP1TmrGrotNj6azVzHaIVVUdMndBF5tZ8/svqt7rdwriUW/OtjE+KsTEsVMAO5Fp81SqyxnP4pxcHnlKO2110b8BV2Erc6SbE8vsEd0aroDxnBWMyik1Vu4vdSxP9MNYJEpblrjKB3T7eIrUGzKjV1E+4iMBrDG1G+aX31E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 43I5iX8O2019030; Thu, 18 Apr 2024 14:44:33 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 43I5i3F52465041; Thu, 18 Apr 2024 14:44:03 +0900 X-Iguazu-Qid: 2rWhclmqQjRZuUaTAY X-Iguazu-QSIG: v=2; s=0; t=1713419043; q=2rWhclmqQjRZuUaTAY; m=UGY3jf7+DzDW54BQLc4yg7NqHUSU0KnvPpq2myBHTvk= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1122) id 43I5i2vK2851714 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:02 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 640FE2FE0F; Thu, 18 Apr 2024 14:44:02 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 3A3A81947AEC50; Thu, 18 Apr 2024 14:22:04 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002QAAAAAA01FM1; Thu, 18 Apr 2024 14:22:04 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id 2E51EAA28483C; Thu, 18 Apr 2024 14:22:04 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:01 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:01 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 02/11] md: add infra for nr_pending changes Thread-Topic: [PATCH v2 02/11] md: add infra for nr_pending changes Thread-Index: AdqLx9x1MkCN7uTlRyqnTzQSCFkVuA== Date: Thu, 18 Apr 2024 05:44:01 +0000 Message-ID: <065afbfd34f44eada6575fd48114e782@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--13.824200-8.000000 x-tmase-matchedrid: 6VY+yZKIFUOmagT1k9kBppTQgFTHgkhZKQNhMboqZlrHhoBEyuHcbXkK 0FnHivpUuH0SzH4JWPa12HagvbwDji/7QU2czuUNA9lly13c/gFr2qJoNIuCjRjNpP+VAaW2cwh qloOj04o3haSYSnhyTlXcW9CR1Vd5FIetOq2+pVDfSQNpZkETVE+crEA4+nhZWm66nNAKC7PQGF lKOB0iJLlW2fEUZEj7eIXhTXDYCVBppBeolGl6TAihQ5NZCXsSsBSA1tuZVSbo8w4N7te92p/p6 742jA2BiFSRT7HBaXD/4j+p8jFGbrkRMDcY9eUDAD5jSg1rFtBNLPQl0QAltLs3Yh2IOCYzlmGO ZvTvMRc1+ahTbhS+5eZRkGfolSWMUaNA17Pg5uoOeRRGICV9PcnlJe2gk8vIOmcd2FDUExunXhF 0LKDIJa4M7Qf5HIbfgt5d7sz8fUvsR/15S/KqDguw+MVcHJpKGYkyyELuGKTfc2Xd6VJ+yjQEU0 gh8OM4V2J5gVT3U48qq3YUbX8N0L9ZdlL8eonaC24oEZ6SpSkj80Za3RRg8BzvtAQBnqcLVTxAd QZh2c/VdfCjDgTgIP41UIhe7WKxI5vZ8n+nN5M= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--13.824200-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: DF7F84FAAD8D6F5870C1D61EB32E6334F3407B7CB44EE2E25AA9CC94FFFDC6DA2000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com Prepare to smoothly change the type of nr_pending from atomic_t to percpu_ref. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md-bitmap.c | 2 +- drivers/md/md.c | 6 +++--- drivers/md/md.h | 32 +++++++++++++++++++++++++++- drivers/md/raid1.c | 20 ++++++++--------- drivers/md/raid10.c | 46 ++++++++++++++++++++-------------------- drivers/md/raid5-cache.c | 4 ++-- drivers/md/raid5.c | 20 ++++++++--------- 7 files changed, 80 insertions(+), 50 deletions(-) diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index 059afc24c08b..6d49a6e4cd2f 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -190,7 +190,7 @@ static struct md_rdev *next_active_rdev(struct md_rdev *rdev, struct mddev *mdde if (rdev->raid_disk >= 0 && !test_bit(Faulty, &rdev->flags)) { /* this is a usable devices */ - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); rcu_read_unlock(); return rdev; } diff --git a/drivers/md/md.c b/drivers/md/md.c index 00bbafcd27bb..d53ca23bacf7 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -575,7 +575,7 @@ static void submit_flushes(struct work_struct *ws) !test_bit(Faulty, &rdev->flags)) { struct bio *bi; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); rcu_read_unlock(); bi = bio_alloc_bioset(rdev->bdev, 0, REQ_OP_WRITE | REQ_PREFLUSH, @@ -1054,7 +1054,7 @@ void md_super_write(struct mddev *mddev, struct md_rdev *rdev, | REQ_PREFLUSH | REQ_FUA, GFP_NOIO, &mddev->sync_set); - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); bio->bi_iter.bi_sector = sector; __bio_add_page(bio, page, size, 0); @@ -9269,7 +9269,7 @@ static bool rdev_removeable(struct md_rdev *rdev) return false; /* There are still inflight io, don't remove this rdev. */ - if (atomic_read(&rdev->nr_pending)) + if (nr_pending_is_not_zero(rdev)) return false; /* diff --git a/drivers/md/md.h b/drivers/md/md.h index 029dd0491a36..f57a894b924b 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -211,6 +211,36 @@ enum flag_bits { Nonrot, /* non-rotational device (SSD) */ }; +static inline void nr_pending_inc(struct md_rdev *rdev) +{ + atomic_inc(&rdev->nr_pending); +} + +static inline void nr_pending_dec(struct md_rdev *rdev) +{ + atomic_dec(&rdev->nr_pending); +} + +static inline bool nr_pending_is_zero(struct md_rdev *rdev) +{ + return atomic_read(&rdev->nr_pending) == 0; +} + +static inline bool nr_pending_is_not_zero(struct md_rdev *rdev) +{ + return atomic_read(&rdev->nr_pending) != 0; +} + +static inline unsigned int nr_pending_read(struct md_rdev *rdev) +{ + return atomic_read(&rdev->nr_pending); +} + +static inline bool nr_pending_dec_and_test(struct md_rdev *rdev) +{ + return atomic_dec_and_test(&rdev->nr_pending); +} + static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, sector_t *first_bad, int *bad_sectors) { @@ -846,7 +876,7 @@ static inline bool is_rdev_broken(struct md_rdev *rdev) static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev) { int faulty = test_bit(Faulty, &rdev->flags); - if (atomic_dec_and_test(&rdev->nr_pending) && faulty) { + if (nr_pending_dec_and_test(rdev) && faulty) { set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); md_wakeup_thread(mddev->thread); } diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index be8ac24f50b6..962cebbba6cd 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -581,7 +581,7 @@ static void update_read_sectors(struct r1conf *conf, int disk, { struct raid1_info *info = &conf->mirrors[disk]; - atomic_inc(&info->rdev->nr_pending); + nr_pending_inc(info->rdev); if (info->next_seq_sect != this_sector) info->seq_start = this_sector; info->next_seq_sect = this_sector + len; @@ -784,7 +784,7 @@ static int choose_best_rdev(struct r1conf *conf, struct r1bio *r1_bio) if (ctl.readable_disks++ == 1) set_bit(R1BIO_FailFast, &r1_bio->state); - pending = atomic_read(&rdev->nr_pending); + pending = nr_pending_read(rdev); dist = abs(r1_bio->sector - conf->mirrors[disk].head_position); /* Don't change to another disk for sequential reads */ @@ -1495,7 +1495,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, write_behind = true; if (rdev && unlikely(test_bit(Blocked, &rdev->flags))) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); blocked_rdev = rdev; break; } @@ -1506,7 +1506,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, continue; } - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (test_bit(WriteErrorSeen, &rdev->flags)) { sector_t first_bad; int bad_sectors; @@ -1879,7 +1879,7 @@ static bool raid1_remove_conf(struct r1conf *conf, int disk) struct md_rdev *rdev = info->rdev; if (!rdev || test_bit(In_sync, &rdev->flags) || - atomic_read(&rdev->nr_pending)) + nr_pending_is_not_zero(rdev)) return false; /* Only remove non-faulty devices if recovery is not possible. */ @@ -1987,7 +1987,7 @@ static int raid1_remove_disk(struct mddev *mddev, struct md_rdev *rdev) struct md_rdev *repl = conf->mirrors[conf->raid_disks + number].rdev; freeze_array(conf, 0); - if (atomic_read(&repl->nr_pending)) { + if (nr_pending_is_not_zero(repl)) { /* It means that some queued IO of retry_list * hold repl. Thus, we cannot set replacement * as NULL, avoiding rdev NULL pointer @@ -2403,7 +2403,7 @@ static void fix_read_error(struct r1conf *conf, struct r1bio *r1_bio) (!test_bit(Faulty, &rdev->flags) && rdev->recovery_offset >= sect + s)) && rdev_has_badblock(rdev, sect, s) == 0) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (sync_page_io(rdev, sect, s<<9, conf->tmppage, REQ_OP_READ, false)) success = 1; @@ -2433,7 +2433,7 @@ static void fix_read_error(struct r1conf *conf, struct r1bio *r1_bio) rdev = conf->mirrors[d].rdev; if (rdev && !test_bit(Faulty, &rdev->flags)) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); r1_sync_page_io(rdev, sect, s, conf->tmppage, REQ_OP_WRITE); rdev_dec_pending(rdev, mddev); @@ -2447,7 +2447,7 @@ static void fix_read_error(struct r1conf *conf, struct r1bio *r1_bio) rdev = conf->mirrors[d].rdev; if (rdev && !test_bit(Faulty, &rdev->flags)) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (r1_sync_page_io(rdev, sect, s, conf->tmppage, REQ_OP_READ)) { atomic_add(s, &rdev->corrected_errors); @@ -2909,7 +2909,7 @@ static sector_t raid1_sync_request(struct mddev *mddev, sector_t sector_nr, } } if (rdev && bio->bi_end_io) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); bio->bi_iter.bi_sector = sector_nr + rdev->data_offset; bio_set_dev(bio, rdev->bdev); if (test_bit(FailFast, &rdev->flags)) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index a4556d2e46bf..2b1238befd8a 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -808,7 +808,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, nonrot = bdev_nonrot(rdev->bdev); has_nonrot_disk |= nonrot; - pending = atomic_read(&rdev->nr_pending); + pending = nr_pending_read(rdev); if (min_pending > pending && nonrot) { min_pending = pending; best_pending_slot = slot; @@ -849,7 +849,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, } if (slot >= 0) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); r10_bio->read_slot = slot; } else rdev = NULL; @@ -1296,12 +1296,12 @@ static void wait_blocked_dev(struct mddev *mddev, struct r10bio *r10_bio) rdev = conf->mirrors[i].rdev; rrdev = conf->mirrors[i].replacement; if (rdev && unlikely(test_bit(Blocked, &rdev->flags))) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); blocked_rdev = rdev; break; } if (rrdev && unlikely(test_bit(Blocked, &rrdev->flags))) { - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); blocked_rdev = rrdev; break; } @@ -1322,7 +1322,7 @@ static void wait_blocked_dev(struct mddev *mddev, struct r10bio *r10_bio) * Mustn't write here until the bad block * is acknowledged */ - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); set_bit(BlockedBadBlocks, &rdev->flags); blocked_rdev = rdev; break; @@ -1467,11 +1467,11 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio, } if (rdev) { r10_bio->devs[i].bio = bio; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); } if (rrdev) { r10_bio->devs[i].repl_bio = bio; - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); } } @@ -1731,11 +1731,11 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio) if (rdev) { r10_bio->devs[disk].bio = bio; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); } if (rrdev) { r10_bio->devs[disk].repl_bio = bio; - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); } } @@ -2155,7 +2155,7 @@ static int raid10_remove_disk(struct mddev *mddev, struct md_rdev *rdev) return 0; if (test_bit(In_sync, &rdev->flags) || - atomic_read(&rdev->nr_pending)) { + nr_pending_is_not_zero(rdev)) { err = -EBUSY; goto abort; } @@ -2394,7 +2394,7 @@ static void sync_request_write(struct mddev *mddev, struct r10bio *r10_bio) bio_copy_data(tbio, fbio); - atomic_inc(&conf->mirrors[d].rdev->nr_pending); + nr_pending_inc(conf->mirrors[d].rdev); atomic_inc(&r10_bio->remaining); md_sync_acct(conf->mirrors[d].rdev->bdev, bio_sectors(tbio)); @@ -2552,12 +2552,12 @@ static void recovery_request_write(struct mddev *mddev, struct r10bio *r10_bio) */ d = r10_bio->devs[1].devnum; if (wbio->bi_end_io) { - atomic_inc(&conf->mirrors[d].rdev->nr_pending); + nr_pending_inc(conf->mirrors[d].rdev); md_sync_acct(conf->mirrors[d].rdev->bdev, bio_sectors(wbio)); submit_bio_noacct(wbio); } if (wbio2) { - atomic_inc(&conf->mirrors[d].replacement->nr_pending); + nr_pending_inc(conf->mirrors[d].replacement); md_sync_acct(conf->mirrors[d].replacement->bdev, bio_sectors(wbio2)); submit_bio_noacct(wbio2); @@ -2633,7 +2633,7 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10 rdev_has_badblock(rdev, r10_bio->devs[sl].addr + sect, s) == 0) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); success = sync_page_io(rdev, r10_bio->devs[sl].addr + sect, @@ -2682,7 +2682,7 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10 !test_bit(In_sync, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (r10_sync_page_io(rdev, r10_bio->devs[sl].addr + sect, @@ -2714,7 +2714,7 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10 !test_bit(In_sync, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); switch (r10_sync_page_io(rdev, r10_bio->devs[sl].addr + sect, @@ -3342,9 +3342,9 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, continue; } if (mrdev) - atomic_inc(&mrdev->nr_pending); + nr_pending_inc(mrdev); if (mreplace) - atomic_inc(&mreplace->nr_pending); + nr_pending_inc(mreplace); r10_bio = raid10_alloc_init_r10buf(conf); r10_bio->state = 0; @@ -3413,7 +3413,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, bio->bi_iter.bi_sector = from_addr + rdev->data_offset; bio_set_dev(bio, rdev->bdev); - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); /* and we write to 'i' (if not in_sync) */ for (k=0; kcopies; k++) @@ -3601,7 +3601,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, continue; } } - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); atomic_inc(&r10_bio->remaining); bio->bi_next = biolist; biolist = bio; @@ -3617,7 +3617,7 @@ static sector_t raid10_sync_request(struct mddev *mddev, sector_t sector_nr, if (rdev == NULL || test_bit(Faulty, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); /* Need to set up for writing to the replacement */ bio = r10_bio->devs[i].repl_bio; @@ -4906,7 +4906,7 @@ static void reshape_request_write(struct mddev *mddev, struct r10bio *r10_bio) if (!rdev || test_bit(Faulty, &rdev->flags)) continue; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); md_sync_acct_bio(b, r10_bio->sectors); atomic_inc(&r10_bio->remaining); b->bi_next = NULL; @@ -4986,7 +4986,7 @@ static int handle_reshape_read_error(struct mddev *mddev, goto failed; addr = r10b->devs[slot].addr + idx * PAGE_SIZE; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); success = sync_page_io(rdev, addr, s << 9, diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c index 874874fe4fa1..1253466666e4 100644 --- a/drivers/md/raid5-cache.c +++ b/drivers/md/raid5-cache.c @@ -1892,7 +1892,7 @@ r5l_recovery_replay_one_stripe(struct r5conf *conf, /* in case device is broken */ rdev = conf->disks[disk_index].rdev; if (rdev) { - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); sync_page_io(rdev, sh->sector, PAGE_SIZE, sh->dev[disk_index].page, REQ_OP_WRITE, false); @@ -1900,7 +1900,7 @@ r5l_recovery_replay_one_stripe(struct r5conf *conf, } rrdev = conf->disks[disk_index].replacement; if (rrdev) { - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); sync_page_io(rrdev, sh->sector, PAGE_SIZE, sh->dev[disk_index].page, REQ_OP_WRITE, false); diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 3b04d8b526b1..25da83eaa2ef 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -1199,11 +1199,11 @@ static void ops_run_io(struct stripe_head *sh, struct stripe_head_state *s) if (rdev && test_bit(Faulty, &rdev->flags)) rdev = NULL; if (rdev) - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (rrdev && test_bit(Faulty, &rrdev->flags)) rrdev = NULL; if (rrdev) - atomic_inc(&rrdev->nr_pending); + nr_pending_inc(rrdev); /* We have already checked bad blocks for reads. Now * need to check for writes. We never accept write errors @@ -1231,7 +1231,7 @@ static void ops_run_io(struct stripe_head *sh, struct stripe_head_state *s) * will dec nr_pending, we must * increment it first. */ - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); md_wait_for_blocked_rdev(rdev, conf->mddev); } else { /* Acknowledged bad block - skip the write */ @@ -3628,7 +3628,7 @@ handle_failed_stripe(struct r5conf *conf, struct stripe_head *sh, if (rdev && test_bit(In_sync, &rdev->flags) && !test_bit(Faulty, &rdev->flags)) - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); else rdev = NULL; if (rdev) { @@ -4729,7 +4729,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) set_bit(BlockedBadBlocks, &rdev->flags); s->blocked_rdev = rdev; - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); } } clear_bit(R5_Insync, &dev->flags); @@ -4767,7 +4767,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) clear_bit(R5_Insync, &dev->flags); if (rdev2 && !test_bit(Faulty, &rdev2->flags)) { s->handle_bad_blocks = 1; - atomic_inc(&rdev2->nr_pending); + nr_pending_inc(rdev2); } else clear_bit(R5_WriteError, &dev->flags); } @@ -4778,7 +4778,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) if (rdev2 && !test_bit(Faulty, &rdev2->flags)) { s->handle_bad_blocks = 1; - atomic_inc(&rdev2->nr_pending); + nr_pending_inc(rdev2); } else clear_bit(R5_MadeGood, &dev->flags); } @@ -4787,7 +4787,7 @@ static void analyse_stripe(struct stripe_head *sh, struct stripe_head_state *s) if (rdev2 && !test_bit(Faulty, &rdev2->flags)) { s->handle_bad_blocks = 1; - atomic_inc(&rdev2->nr_pending); + nr_pending_inc(rdev2); } else clear_bit(R5_MadeGoodRepl, &dev->flags); } @@ -5478,7 +5478,7 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) return 0; } - atomic_inc(&rdev->nr_pending); + nr_pending_inc(rdev); if (rdev_has_badblock(rdev, sector, bio_sectors(raid_bio))) { rdev_dec_pending(rdev, mddev); @@ -8157,7 +8157,7 @@ static int raid5_remove_disk(struct mddev *mddev, struct md_rdev *rdev) clear_bit(In_sync, &rdev->flags); if (test_bit(In_sync, &rdev->flags) || - atomic_read(&rdev->nr_pending)) { + nr_pending_is_not_zero(rdev)) { err = -EBUSY; goto abort; } From patchwork Thu Apr 18 05:44:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634224 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1122.securemx.jp [210.130.202.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E031B37719; Thu, 18 Apr 2024 07:00:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713423626; cv=none; b=KxLq8gnsy9rrypkv+nnscrrBW5KTognQC05hT6bosxH0vsRai+LSqOIicZaqlqEWKj8oiZkqqNDKeo3V2PpDL2CrwpS6Srpf8dehlNurwmiCbE+P+ir1P+5bNwIus5hsD+gjp/9Rj/3KHE2HliPxGu9qDhJbU1w435J7Pk4CLCM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713423626; c=relaxed/simple; bh=mqwFrIzDju9TMIIICMRMI4d5HpzguSACJdFojcfzKpc=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=craHWy6a8yCLXeQJb9Jq/JIBKWiGG4q2USYNMqhT0Hh/CWT7UZdaXgwB3CveO7PfpwdeWmdyxm6OA0b7gF2GK4y43VpAU01n8NzLOcAM9HlJK0CmeyshTYPzSHUDF6SpJ1DkyMgIQc3wU2e+UW9Ft/Zqik//gpu21qbcD3cAW1k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1122) id 43I5iXsP2111095; Thu, 18 Apr 2024 14:44:33 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 43I5i69a1973992; Thu, 18 Apr 2024 14:44:07 +0900 X-Iguazu-Qid: 2rWhTy7tBPqzniEIkh X-Iguazu-QSIG: v=2; s=0; t=1713419046; q=2rWhTy7tBPqzniEIkh; m=AevLJNU7sdeyJIm6CUqHIA7BK0AHDkRW4BxeA0WNq2M= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 43I5i5wJ2636816 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:05 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id A77D048A81; Thu, 18 Apr 2024 14:44:05 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 7E7A51947AEC50; Thu, 18 Apr 2024 14:22:07 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002QAAAAAA01FM9; Thu, 18 Apr 2024 14:22:07 +0900 Received: from CNN1EXMB02.r1.kioxia.com (CNN1EXMB02.r1.kioxia.com [10.232.20.151]) by Switcher-Pre_Send (Postfix) with ESMTP id 7347DAA28483C; Thu, 18 Apr 2024 14:22:07 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB02.r1.kioxia.com (10.232.20.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:04 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:04 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 03/11] md: workaround for inconsistency of config state in takeover Thread-Topic: [PATCH v2 03/11] md: workaround for inconsistency of config state in takeover Thread-Index: AdqLx93+rdcA1FHTRcas4Y+n9m5suw== Date: Thu, 18 Apr 2024 05:44:04 +0000 Message-ID: <70bf226187b7466eb0a8480922ba9cc7@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--9.492700-8.000000 x-tmase-matchedrid: tkB2VEODQ1emagT1k9kBpgrcxrzwsv5uXPK9y3z82GtXGTbsQqHbkkog 3hT9f/oCJ2spf+DCeJzmn3xyPJAJoh2P280ZiGmRupDIC9422DoWUg/hm489yO+sdQufriAS4aS bGEJzu4sESV4es7KJNx8GSwQM8TG4+qIQ7qDJ9P+AwosDBeuPwZki3iIBA3o/rhnpK05RvESFFj ccLWe/lssDZ/QWaNswKDX7UR/i5drv4tNJ1H3OMBJo06V2lHTeBGvINcfHqhe52DlOQbIerbFNg mH5/5Kd585VzGMOFzA9wJeM2pSaRSAHAopEd76vz6B2CIwVCKQCUFvfbZma39F+WlOsR9awHewC W3pU/r2oTymQXYzk1g== x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--9.492700-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: AE5A978A5368A1D0ABA9418A0FFA7FD7368B0AB054A6A95739451643F4FDCDB52000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB02.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB02.r1.kioxia.com This patch depends on patch 01. An inconsistency occurs between config state in takeover and percpu_ref state. Differentiate percpu_ref state setting for takeover and other to avoid the inconsistency. Therefore, add percpu_ref state setting to match config state in takeover. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/raid5.c | 39 +++++++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 8 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 25da83eaa2ef..b7056854375d 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -7396,7 +7396,8 @@ static unsigned long raid5_cache_count(struct shrinker *shrink, return max_stripes - min_stripes; } -static struct r5conf *setup_conf(struct mddev *mddev) + +static struct r5conf *setup_conf(struct mddev *mddev, bool quiesce) { struct r5conf *conf; int raid_disk, memory, max_disks; @@ -7407,6 +7408,7 @@ static struct r5conf *setup_conf(struct mddev *mddev) int group_cnt; struct r5worker_group *new_group; int ret = -ENOMEM; + unsigned int percpu_ref_init_flags; if (mddev->new_level != 5 && mddev->new_level != 4 @@ -7477,6 +7479,10 @@ static struct r5conf *setup_conf(struct mddev *mddev) init_llist_head(&conf->released_stripes); atomic_set(&conf->active_stripes, 0); atomic_set(&conf->preread_active_stripes, 0); + if (quiesce) + percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT | PERCPU_REF_INIT_DEAD; + else + percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT; atomic_set(&conf->active_aligned_reads, 0); spin_lock_init(&conf->pending_bios_lock); conf->batch_bio_dispatch = true; @@ -7657,6 +7663,23 @@ static struct r5conf *setup_conf(struct mddev *mddev) return ERR_PTR(ret); } +static struct r5conf *setup_conf_for_run(struct mddev *mddev) +{ + return setup_conf(mddev, false); +} + +static struct r5conf *setup_conf_for_takeover(struct mddev *mddev) +{ + struct r5conf *conf; + bool quiesce = false; + + if (mddev->level == 4 || mddev->level == 5 || mddev->level == 6) { + conf = mddev->private; + quiesce = false; + } + return setup_conf(mddev, quiesce); +} + static int only_parity(int raid_disk, int algo, int raid_disks, int max_degraded) { switch (algo) { @@ -7884,7 +7907,7 @@ static int raid5_run(struct mddev *mddev) } if (mddev->private == NULL) - conf = setup_conf(mddev); + conf = setup_conf_for_run(mddev); else conf = mddev->private; @@ -8627,7 +8650,7 @@ static void *raid45_takeover_raid0(struct mddev *mddev, int level) /* make sure it will be not marked as dirty */ mddev->recovery_cp = MaxSector; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } static void *raid5_takeover_raid1(struct mddev *mddev) @@ -8655,7 +8678,7 @@ static void *raid5_takeover_raid1(struct mddev *mddev) mddev->new_layout = ALGORITHM_LEFT_SYMMETRIC; mddev->new_chunk_sectors = chunksect; - ret = setup_conf(mddev); + ret = setup_conf_for_takeover(mddev); if (!IS_ERR(ret)) mddev_clear_unsupported_flags(mddev, UNSUPPORTED_MDDEV_FLAGS); @@ -8692,7 +8715,7 @@ static void *raid5_takeover_raid6(struct mddev *mddev) mddev->new_layout = new_layout; mddev->delta_disks = -1; mddev->raid_disks -= 1; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } static int raid5_check_reshape(struct mddev *mddev) @@ -8770,7 +8793,7 @@ static void *raid5_takeover(struct mddev *mddev) if (mddev->level == 4) { mddev->new_layout = ALGORITHM_PARITY_N; mddev->new_level = 5; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } if (mddev->level == 6) return raid5_takeover_raid6(mddev); @@ -8790,7 +8813,7 @@ static void *raid4_takeover(struct mddev *mddev) mddev->layout == ALGORITHM_PARITY_N) { mddev->new_layout = 0; mddev->new_level = 4; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } return ERR_PTR(-EINVAL); } @@ -8840,7 +8863,7 @@ static void *raid6_takeover(struct mddev *mddev) mddev->new_layout = new_layout; mddev->delta_disks = 1; mddev->raid_disks += 1; - return setup_conf(mddev); + return setup_conf_for_takeover(mddev); } static int raid5_change_consistency_policy(struct mddev *mddev, const char *buf) From patchwork Thu Apr 18 05:44:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634223 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AD8F2576E; Thu, 18 Apr 2024 06:56:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713423400; cv=none; b=a9Y1e6u/jBA2tjdrMFw+U9gmU4or6UqF+iL43xQaIa4FpAfawdoi6+dNoiSA9F9j0Zryv73sYOsb55UAVxxDEvcf0Im9UuNm/TXFr2o8v2c1PB7r6YaL77gbqnMfMQN7tCnkOT7Z7TdZ1q7EG/6l+ALKXdP6z11UKIvdQbOqsy8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713423400; c=relaxed/simple; bh=a4hrlylITEtimTumffZSXVCQvjxwph/PpFXcseW5wLI=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=FGRlNnSAJERCB6YTyy7qqWLyBl6jrknRVfjcQXH38A1i1Hwo6ndu5yHMFiWKUEbAhbmTl0EVf1T1kqvSJ/z9XJ1qShUdb66vmB1W5wlJMNlDliPjOighgw6aIsILDeEI6AU3kJgFHAECI/kSLTjoV/bFauLkdcAz3IkQwofvu+M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 43I5icES2019037; Thu, 18 Apr 2024 14:44:38 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 43I5iAP12152677; Thu, 18 Apr 2024 14:44:10 +0900 X-Iguazu-Qid: 2rWhYYPQE6ctn5HIHq X-Iguazu-QSIG: v=2; s=0; t=1713419050; q=2rWhYYPQE6ctn5HIHq; m=uRZhONpQHkzEcOPMdo5WXIisYFeMFbsAQygYXrNlQrc= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 43I5i9Vd2636963 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:09 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 3968F2FE0F; Thu, 18 Apr 2024 14:44:09 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 8C2101900001EB; Thu, 18 Apr 2024 14:31:26 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003QAAAAAA021N3; Thu, 18 Apr 2024 14:31:26 +0900 Received: from CNN1EXMB01.r1.kioxia.com (CNN1EXMB01.r1.kioxia.com [10.232.20.150]) by Switcher-Pre_Send (Postfix) with ESMTP id 80EC0A0640E18; Thu, 18 Apr 2024 14:31:26 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB01.r1.kioxia.com (10.232.20.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:08 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:08 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 04/11] md: minimize execution of zero check for nr_pending Thread-Topic: [PATCH v2 04/11] md: minimize execution of zero check for nr_pending Thread-Index: AdqLx9/4oTxaoxgcTPKqUtGd+w6HlQ== Date: Thu, 18 Apr 2024 05:44:08 +0000 Message-ID: <68a3ec69e8424ba485a369870c2e9177@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--5.659600-8.000000 x-tmase-matchedrid: g7XVaO+YFMumagT1k9kBpgrcxrzwsv5uXPK9y3z82GtXGTbsQqHbkmzE TQ7/otyjd+FThkXYawcVNmOCBj98YKZY4PxfRMWETuctSpiuWyUUi4Ehat05499RlPzeVuQQB4x q6g7kDalvF9oQ/B54yGPKQgbIX4yuFdV6KRtQicSeAiCmPx4NwJwhktVkBBrQ38LauI2fxt4qtq 5d3cxkNQ97ozsDEZvUB9eDeIdUSivNgawOLAkwRBzunJpIx5qBmXpTt4Va+To= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--5.659600-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: 33DCE608A229C05FEE447A2E6D22FFCB7D0B52B69A0FD7D4C11675C897FF9A802000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB01.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB01.r1.kioxia.com This patch depends on patch 02. The Faulty flag is required for the if statement to be true. A rdev in Faulty state does not issue I/O commands. So, nr_pending is also not increased. Therefore, decrement and zero check of nr_pending can be separated and still be consistent. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.h | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/drivers/md/md.h b/drivers/md/md.h index f57a894b924b..2eaf8606af9e 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -236,11 +236,6 @@ static inline unsigned int nr_pending_read(struct md_rdev *rdev) return atomic_read(&rdev->nr_pending); } -static inline bool nr_pending_dec_and_test(struct md_rdev *rdev) -{ - return atomic_dec_and_test(&rdev->nr_pending); -} - static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, sector_t *first_bad, int *bad_sectors) { @@ -876,9 +871,12 @@ static inline bool is_rdev_broken(struct md_rdev *rdev) static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev) { int faulty = test_bit(Faulty, &rdev->flags); - if (nr_pending_dec_and_test(rdev) && faulty) { - set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); - md_wakeup_thread(mddev->thread); + nr_pending_dec(rdev); + if (faulty) { + if (nr_pending_is_zero(rdev)) { + set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); + md_wakeup_thread(mddev->thread); + } } } From patchwork Thu Apr 18 05:44:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634214 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 219372CCC2; Thu, 18 Apr 2024 06:41:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713422510; cv=none; b=Jv4hXZu1XEIPE/NbkkafOEPS6xioRJPcBr+700ZnkHI65oXECXNcQTFvkl3gU2oo5cU2vjC/lL5ADmNGxPRUvT/W8cUlCoZwD3L9imQBv/OcJKEydqI5Flt6ruIUO7xiO31Rt7+MrH05y3vIu9SM/pO6LzGJmxsoF18mrCSopbQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713422510; c=relaxed/simple; bh=SzE8IQJ48xc7908eyTNtDsHkCKQFHLy2DgGThS09xpo=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=fq9007cjwY1lIJtBLd2goHMwCKvINKac89hI1vZQ1j5aua7IOBnanpq8+NYE76TKx0xUzj9NC6T9XDTEDfA6EL51hDtyTyGDUsKophOlo86ljPtFK0dU0Qb9R+yXCkLH1WrYY/7kwRyGeL0k5iGK9dP6byzrGOuLI4Do/8Bevcw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 43I5iYZe2019031; Thu, 18 Apr 2024 14:44:34 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 43I5iE7j2465865; Thu, 18 Apr 2024 14:44:14 +0900 X-Iguazu-Qid: 2rWhYYPQE6cu19H1s9 X-Iguazu-QSIG: v=2; s=0; t=1713419053; q=2rWhYYPQE6cu19H1s9; m=afDSdK3UvNipDoLKBSLLnma5OONPJ5FrYHGDMbHQdyY= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 43I5iCX22637104 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:13 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id D4E4B48A81; Thu, 18 Apr 2024 14:44:12 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 2778B1900001EB; Thu, 18 Apr 2024 14:31:30 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003QAAAAAA02211; Thu, 18 Apr 2024 14:31:30 +0900 Received: from CNN1EXMB02.r1.kioxia.com (CNN1EXMB02.r1.kioxia.com [10.232.20.151]) by Switcher-Pre_Send (Postfix) with ESMTP id 1BFCBA0640E18; Thu, 18 Apr 2024 14:31:30 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB02.r1.kioxia.com (10.232.20.151) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:12 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:12 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 05/11] md: match the type of variables to percpu_ref Thread-Topic: [PATCH v2 05/11] md: match the type of variables to percpu_ref Thread-Index: AdqLx+FXdNJkPwyDSq2lcgBOkP0WcQ== Date: Thu, 18 Apr 2024 05:44:11 +0000 Message-ID: <1c04e1a6d79c494faad6bdc21d86d76e@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--5.106300-8.000000 x-tmase-matchedrid: g7XVaO+YFMumagT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa+K5 fhRxt1aADpxsntCj+iajrLRBxwnZS7VQ6XPWwtdyEXjPIvKd74BMkOX0UoduuRFBDiQWqOMkJwY Oi+iMxY3U5iXDpe6YeP6mFiC4o7QUdJMdJS+tDywZgmFGHqyx630tCKdnhB581kTfEkyaZdz6C0 ePs7A07QsfaqMZktsd9K4u2mW1AZ3iZawwAP+WlZ2c5QMBYjIDRSkcJMJhH2I= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--5.106300-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: 206CAC30B0DCD9F165B7D62BF7D51A0DF8E8655C7C4CA21B759AF0D5981D8EF02000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB02.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB02.r1.kioxia.com This patch depends on patch 04. The counter size of percpu_ref is different from that of atomic_t. This is the only place where the value of nr_pending is read. The others are used for zero check and type has no effect. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.h | 4 ++-- drivers/md/raid1.c | 4 ++-- drivers/md/raid10.c | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/md/md.h b/drivers/md/md.h index 2eaf8606af9e..783fec1f1445 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -231,9 +231,9 @@ static inline bool nr_pending_is_not_zero(struct md_rdev *rdev) return atomic_read(&rdev->nr_pending) != 0; } -static inline unsigned int nr_pending_read(struct md_rdev *rdev) +static inline unsigned long nr_pending_read(struct md_rdev *rdev) { - return atomic_read(&rdev->nr_pending); + return (unsigned long)atomic_read(&rdev->nr_pending); } static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 962cebbba6cd..12318fb15a88 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -751,7 +751,7 @@ static bool rdev_readable(struct md_rdev *rdev, struct r1bio *r1_bio) struct read_balance_ctl { sector_t closest_dist; int closest_dist_disk; - int min_pending; + long min_pending; int min_pending_disk; int sequential_disk; int readable_disks; @@ -771,7 +771,7 @@ static int choose_best_rdev(struct r1conf *conf, struct r1bio *r1_bio) for (disk = 0 ; disk < conf->raid_disks * 2 ; disk++) { struct md_rdev *rdev; sector_t dist; - unsigned int pending; + unsigned long pending; if (r1_bio->bios[disk] == IO_BLOCKED) continue; diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 2b1238befd8a..b91dd6c0be5a 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -733,7 +733,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, int do_balance; int best_dist_slot, best_pending_slot; bool has_nonrot_disk = false; - unsigned int min_pending; + unsigned long min_pending; struct geom *geo = &conf->geo; raid10_find_phys(conf, r10_bio); @@ -753,7 +753,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, sector_t first_bad; int bad_sectors; sector_t dev_sector; - unsigned int pending; + unsigned long pending; bool nonrot; if (r10_bio->devs[slot].bio == IO_BLOCKED) From patchwork Thu Apr 18 05:44:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634215 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13E2D2CCC2; Thu, 18 Apr 2024 06:45:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713422703; cv=none; b=EfTehW8PZSY1KZkcPgA0OLR3wUCflQ+Ji3EmK+aAIZ904n4P8+9b+fyb3bX8geYlg9u3NELXt7voarbjYLK4tda5raNcnXK4LN3+bkDkAFrB4mksN2ov9ZngBYu1vyFcUPkwiFlKdgOKyWG1cY7V/W6htHb59X+taol06UzPlNQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713422703; c=relaxed/simple; bh=L+/4oTFeLIWsXRBNEPceBnHFBU9eV7Xxg2EuFA3urAw=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=DWOMfFzA0QRpTpaMLWAnJROIqGVPF0V87o6tzdzfM+NnbacaD1WEaqvCzSgZMxsLHR/6yl7YqmI7xTSwO/MsuDOjb1u9ktPSu5ZV9rDoBsOsvNQ3KXPVTp8+BlqaUwef6wibl9wgX0JnqQSEQeVq83pjSWvCQe3zlTmB2DsLPk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 43I5iUat2019029; Thu, 18 Apr 2024 14:44:30 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 43I5iHEO1974788; Thu, 18 Apr 2024 14:44:17 +0900 X-Iguazu-Qid: 2rWhTy7tBPr0YaskhI X-Iguazu-QSIG: v=2; s=0; t=1713419056; q=2rWhTy7tBPr0YaskhI; m=/bUuy0ljH1qi2LvfcftYTdnhRM+6KW6sOhz7M0Nb/fo= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1122) id 43I5iGfT2852290 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:16 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 120DC2FE0F; Thu, 18 Apr 2024 14:44:16 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 5B6331900001EB; Thu, 18 Apr 2024 14:31:33 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003QAAAAAA02216; Thu, 18 Apr 2024 14:31:33 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id 4FF53A0640E18; Thu, 18 Apr 2024 14:31:33 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:15 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:15 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 06/11] md: avoid conflicts in active_aligned_reads operations Thread-Topic: [PATCH v2 06/11] md: avoid conflicts in active_aligned_reads operations Thread-Index: AdqLx+KRoW2QXRHySCi80jeCEydOoA== Date: Thu, 18 Apr 2024 05:44:15 +0000 Message-ID: <2732579d7512407c89a4cc9db1c63934@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--10.964700-8.000000 x-tmase-matchedrid: QprU7jbCnXWmagT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa4Ul JlxlTIT+SMhOCdNodgNTbKunqk1vfbVQ6XPWwtdyEXjPIvKd74BMkOX0UoduufLIj08N6FsEuJE UmdYPGSVzy5czsqHK1z4BLF18v1Ji23JrSYCBAcfN+qWlu2ZxaFr2hZxjCLzqMoh6scCF9jF96t 9wDfdERVvLgfDdKWEFEn91NheE+HWip6xhGipIWazGfgakLdja0NaTZUEzRZzC9DxHKrnk8luWw Soz2HVyH7iboQD+7I4VaGy/EyU2eafccBtB84hSnXdphQTSK/K/yN2q8U674odzPqBXhP/S4Mmu En4nwqo1W+3DxdRobxDflJk00ivp0FQxfjRHOzi4u3nS+3EEDn0tCKdnhB581kTfEkyaZdz6C0e Ps7A07YVH0dq7wY7up8Odl1VwpCQwXEs0RQd4EzS1UmI+/vnAHxGeptjWSXBAfvnw3P2gkZlQDd ZWm5Nd x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--10.964700-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: 1EA07DDBD178E50F0C09B170C34B3B70AA030D08F4DE65BD5931C7C15CE0C6512000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com This patch depends on patch 03. Change the type of active_aligned_reads from atomic_t to percpu_ref in percpu mode. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/raid5.c | 27 +++++++++++++++++++-------- drivers/md/raid5.h | 13 ++++--------- 2 files changed, 23 insertions(+), 17 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index b7056854375d..f267856efb48 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5435,8 +5435,7 @@ static void raid5_align_endio(struct bio *bi) if (!error) { bio_endio(raid_bi); - if (active_aligned_reads_dec_and_test(conf)) - wake_up(&conf->wait_for_quiescent); + active_aligned_reads_dec(conf); return; } @@ -5507,8 +5506,8 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio) /* quiesce is in progress, so we need to undo io activation and wait * for it to finish */ - if (did_inc && active_aligned_reads_dec_and_test(conf)) - wake_up(&conf->wait_for_quiescent); + if (did_inc) + active_aligned_reads_dec(conf); spin_lock_irq(&conf->device_lock); wait_event_lock_irq(conf->wait_for_quiescent, conf->quiesce == 0, conf->device_lock); @@ -6608,8 +6607,7 @@ static int retry_aligned_read(struct r5conf *conf, struct bio *raid_bio, bio_endio(raid_bio); - if (active_aligned_reads_dec_and_test(conf)) - wake_up(&conf->wait_for_quiescent); + active_aligned_reads_dec(conf); return handled; } @@ -7315,6 +7313,7 @@ static void free_conf(struct r5conf *conf) { int i; + percpu_ref_exit(&conf->active_aligned_reads); log_exit(conf); shrinker_free(conf->shrinker); @@ -7396,6 +7395,12 @@ static unsigned long raid5_cache_count(struct shrinker *shrink, return max_stripes - min_stripes; } +static void percpu_wakeup_handle_req_active(struct percpu_ref *r) +{ + struct r5conf *conf = container_of(r, struct r5conf, active_aligned_reads); + + wake_up(&conf->wait_for_quiescent); +} static struct r5conf *setup_conf(struct mddev *mddev, bool quiesce) { @@ -7483,7 +7488,11 @@ static struct r5conf *setup_conf(struct mddev *mddev, bool quiesce) percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT | PERCPU_REF_INIT_DEAD; else percpu_ref_init_flags = PERCPU_REF_ALLOW_REINIT; - atomic_set(&conf->active_aligned_reads, 0); + ret = percpu_ref_init(&conf->active_aligned_reads, percpu_wakeup_handle_req_active, + percpu_ref_init_flags, GFP_KERNEL); + if (ret) + goto abort; + ret = -ENOMEM; spin_lock_init(&conf->pending_bios_lock); conf->batch_bio_dispatch = true; rdev_for_each(rdev, mddev) { @@ -7675,7 +7684,7 @@ static struct r5conf *setup_conf_for_takeover(struct mddev *mddev) if (mddev->level == 4 || mddev->level == 5 || mddev->level == 6) { conf = mddev->private; - quiesce = false; + quiesce = percpu_ref_is_dying(&conf->active_aligned_reads); } return setup_conf(mddev, quiesce); } @@ -8607,6 +8616,7 @@ static void raid5_quiesce(struct mddev *mddev, int quiesce) * quiesce started and reverts to slow (locked) path. */ smp_store_release(&conf->quiesce, 2); + percpu_ref_kill(&conf->active_aligned_reads); wait_event_cmd(conf->wait_for_quiescent, atomic_read(&conf->active_stripes) == 0 && active_aligned_reads_is_zero(conf), @@ -8619,6 +8629,7 @@ static void raid5_quiesce(struct mddev *mddev, int quiesce) } else { /* re-enable writes */ lock_all_device_hash_locks_irq(conf); + percpu_ref_reinit(&conf->active_aligned_reads); conf->quiesce = 0; wake_up(&conf->wait_for_quiescent); wake_up(&conf->wait_for_overlap); diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h index 5bd6bb3540c5..c4ab418e2084 100644 --- a/drivers/md/raid5.h +++ b/drivers/md/raid5.h @@ -621,7 +621,7 @@ struct r5conf { unsigned int retry_read_offset; /* sector offset into retry_read_aligned */ struct bio *retry_read_aligned_list; /* aligned bios retry list */ atomic_t preread_active_stripes; /* stripes with scheduled io */ - atomic_t active_aligned_reads; + struct percpu_ref active_aligned_reads; atomic_t pending_full_writes; /* full write backlog */ int bypass_count; /* bypassed prereads */ int bypass_threshold; /* preread nice */ @@ -698,22 +698,17 @@ struct r5conf { static inline void active_aligned_reads_inc(struct r5conf *conf) { - atomic_inc(&conf->active_aligned_reads); + percpu_ref_get(&conf->active_aligned_reads); } static inline void active_aligned_reads_dec(struct r5conf *conf) { - atomic_dec(&conf->active_aligned_reads); + percpu_ref_put(&conf->active_aligned_reads); } static inline bool active_aligned_reads_is_zero(struct r5conf *conf) { - return atomic_read(&conf->active_aligned_reads) == 0; -} - -static inline bool active_aligned_reads_dec_and_test(struct r5conf *conf) -{ - return atomic_dec_and_test(&conf->active_aligned_reads); + return percpu_ref_is_zero(&conf->active_aligned_reads); } #if PAGE_SIZE == DEFAULT_STRIPE_SIZE From patchwork Thu Apr 18 05:44:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634226 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1121.securemx.jp [210.130.202.129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9666B38DD3; Thu, 18 Apr 2024 07:04:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.129 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713423893; cv=none; b=KRT4c/mPDpMHqFKTLm7AXOPaaCQCuDI3oL2vSsPxQkNu35y/5wui95/Q2TQU1bc9WEA38zqlKIpSBOraZnB3JH3l3EBa1iUzqaDSuuyMaO73aWuwCgUgFkAgEqNgzGMlo+UQJVgmVlkPZRWED/pflMXaVSFad4W4v9Njo2hWoZk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713423893; c=relaxed/simple; bh=uoNEKKgt7kzgTLPceRuEH7vqG26qiW6Mlgun+OQdhUc=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=hbFknlpBLr0SL5q2G/XYPRq2QD0navEpIgZJEoxFhoVl1WcXP7YHkIf3EjkRRHNPCBQECeRt3oJULpbI5/f6wHD+uVPF8cvGjaVTyUQBSATys6SCWPFNyL50jQaWdb3jndWE4Sncc6CRMbllzqVpUiKpKQ9XrHaQaLWkZODLpbk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.129 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1121) id 43I5iX9d2031332; Thu, 18 Apr 2024 14:44:33 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 43I5iKQA1974974; Thu, 18 Apr 2024 14:44:20 +0900 X-Iguazu-Qid: 2rWhclmqQjRb7VEOHb X-Iguazu-QSIG: v=2; s=0; t=1713419059; q=2rWhclmqQjRb7VEOHb; m=mBAfYaaefeel5VPM0onU99bupKNgugriUCfLeVqX7Wg= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1122) id 43I5iJUt2852443 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:19 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 093132FE0F; Thu, 18 Apr 2024 14:44:19 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id D46111947AEC50; Thu, 18 Apr 2024 14:22:20 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002QAAAAAA01G4A; Thu, 18 Apr 2024 14:22:20 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id C9198AA28483C; Thu, 18 Apr 2024 14:22:20 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:18 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:18 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 07/11] md: change the type of nr_pending from atomic_t to percpu_ref Thread-Topic: [PATCH v2 07/11] md: change the type of nr_pending from atomic_t to percpu_ref Thread-Index: AdqLx+Sa7q7ZO7b2TwapHpdnlQ/uTw== Date: Thu, 18 Apr 2024 05:44:18 +0000 Message-ID: <7da99b09cb524bdcb705be1b74fdf3e0@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--11.170200-8.000000 x-tmase-matchedrid: q6+TWfRxf4emagT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa4Ul JlxlTIT+uzW0mXOtDblgTs7PNu5YyogrKnsJ1GRgTuctSpiuWyUUi4Ehat05499RlPzeVuQQbo+ zo8QXL7dce6BqICgMNO++yHYM+maWCEeKzjvXhRNT3j6uyvUYPHO1oaQ51CuNyPRAwD/3abY9/h S2mXmf7hyQILeil6Pygimqy60vioomCg+Lak4DCfQxpA7auLwMsnK72MaPSqdKcHrMrTS0BKPFj JEFr+olSXhbxZVQ5H+OhzOa6g8KrYMsC4HjxGgcpp5TBDDn4/99nh8vjJfPPtOGGDxfqdGKNS6d yfmO4Lc= x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--11.170200-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: 9B7AE58AE751170A3157A162B8CC0E64CFF4C7F0FF471CCE1172567A353084412000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com This patch depends on patch 05. Change the type of nr_pending from atomic_t to percpu_ref. Additionally, temporarily switch from percpu mode to atomic mode. This is to reduce the amount of changes from patches 8 to 10. Patches 8 to 10 assume that nr_pending is run in percpu mode. Finally, switch from atomic mode to percpu mode in patch 11. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 13 ++++++++++++- drivers/md/md.h | 18 +++++++++--------- 2 files changed, 21 insertions(+), 10 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index d53ca23bacf7..7e6966e65d0d 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -1002,6 +1002,7 @@ void md_rdev_clear(struct md_rdev *rdev) put_page(rdev->bb_page); rdev->bb_page = NULL; } + percpu_ref_exit(&rdev->nr_pending); badblocks_exit(&rdev->badblocks); } EXPORT_SYMBOL_GPL(md_rdev_clear); @@ -3724,8 +3725,14 @@ static const struct kobj_type rdev_ktype = { .default_groups = rdev_default_groups, }; +static void percpu_wakeup_handle_req_pending(struct percpu_ref *ref) +{ +} + int md_rdev_init(struct md_rdev *rdev) { + int ret; + rdev->desc_nr = -1; rdev->saved_raid_disk = -1; rdev->raid_disk = -1; @@ -3736,7 +3743,11 @@ int md_rdev_init(struct md_rdev *rdev) rdev->last_read_error = 0; rdev->sb_loaded = 0; rdev->bb_page = NULL; - atomic_set(&rdev->nr_pending, 0); + ret = percpu_ref_init(&rdev->nr_pending, percpu_wakeup_handle_req_pending, + PERCPU_REF_ALLOW_REINIT, GFP_KERNEL); + WARN_ON(ret); + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); + nr_pending_dec(rdev); atomic_set(&rdev->read_errors, 0); atomic_set(&rdev->corrected_errors, 0); diff --git a/drivers/md/md.h b/drivers/md/md.h index 783fec1f1445..ab09e312c9bb 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -107,10 +107,10 @@ struct md_rdev { */ }; - atomic_t nr_pending; /* number of pending requests. - * only maintained for arrays that - * support hot removal - */ + struct percpu_ref nr_pending; /* number of pending requests. + * only maintained for arrays that + * support hot removal + */ atomic_t read_errors; /* number of consecutive read errors that * we have tried to ignore. */ @@ -213,27 +213,27 @@ enum flag_bits { static inline void nr_pending_inc(struct md_rdev *rdev) { - atomic_inc(&rdev->nr_pending); + percpu_ref_get(&rdev->nr_pending); } static inline void nr_pending_dec(struct md_rdev *rdev) { - atomic_dec(&rdev->nr_pending); + percpu_ref_put(&rdev->nr_pending); } static inline bool nr_pending_is_zero(struct md_rdev *rdev) { - return atomic_read(&rdev->nr_pending) == 0; + return percpu_ref_is_zero(&rdev->nr_pending); } static inline bool nr_pending_is_not_zero(struct md_rdev *rdev) { - return atomic_read(&rdev->nr_pending) != 0; + return !percpu_ref_is_zero(&rdev->nr_pending); } static inline unsigned long nr_pending_read(struct md_rdev *rdev) { - return (unsigned long)atomic_read(&rdev->nr_pending); + return atomic_long_read(&rdev->nr_pending.data->count); } static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, From patchwork Thu Apr 18 05:44:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634216 Received: from mo-csw-fb.securemx.jp (mo-csw-fb1120.securemx.jp [210.130.202.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEF146A8AD; Thu, 18 Apr 2024 06:45:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.128 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713422755; cv=none; b=sSHQeOgg6VVDmvZgFrlY46ceifncyR7y1qyKX/gYtZdjqHXSnkan7zoa454kDXCuz6jA8tWH4Ncoq6L2i3/OxFozK6xq7E25sgA+WhSiXIAARF+BihnPYaYVt93nteZoYgSHI/MNb9XAhpmj5emc3Cr9HYrRv+0eAaftjx0nNZ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713422755; c=relaxed/simple; bh=Ypi0ot5qJ8tn0ASlGHSUepHvLkwFH4KHfNbquxNLrHg=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=I1pTbvj0DoUnpz3f7k2aSP41p0nRrgJWHBMzvXOysFT3IslOYlRrfvOKtagXyXrBLU9mxmGkOg06FXvHAcFYj9JfQAg1wlEv7XRRy0vGjFN9CONlYAaywEdJdAmHySut5rW3zPtHsyJATdlXvGiSFzwCSAAMM/wn8g9d6xk/jXY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.128 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw-fb.securemx.jp (mx-mo-csw-fb1120) id 43I5ifvv2019038; Thu, 18 Apr 2024 14:44:41 +0900 Received: by mo-csw.securemx.jp (mx-mo-csw1121) id 43I5iMCF2153416; Thu, 18 Apr 2024 14:44:23 +0900 X-Iguazu-Qid: 2rWhTy7tBPr10isDrf X-Iguazu-QSIG: v=2; s=0; t=1713419062; q=2rWhTy7tBPr10isDrf; m=IDB3TLEYIEqh17n/W51ywxnqltOA5XjHLUvRZYIxeNM= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1120) id 43I5iLs33349860 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:22 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id BB40F48A81; Thu, 18 Apr 2024 14:44:21 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 0FED21900001EB; Thu, 18 Apr 2024 14:31:39 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003QAAAAAA02252; Thu, 18 Apr 2024 14:31:39 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id 0479EA0640E18; Thu, 18 Apr 2024 14:31:39 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:21 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:21 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 08/11] md: add atomic mode switching in RAID 1/10 Thread-Topic: [PATCH v2 08/11] md: add atomic mode switching in RAID 1/10 Thread-Index: AdqLx+XD0n9rpAUxRba5DThcEUS6pA== Date: Thu, 18 Apr 2024 05:44:20 +0000 Message-ID: <47c035c3e741418b80eb6b73d96e7e92@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--6.422500-8.000000 x-tmase-matchedrid: qiJ73D9abaWmagT1k9kBppTQgFTHgkhZKQNhMboqZlrGZbX3v50WrzPn 4yDW5yf3V6Ny+iZr9he3d+TOD1+jLJGZilTi8ctSTuctSpiuWyUUi4Ehat05499RlPzeVuQQse2 N9uwuoMxvu+EAUOCx0zIsEmLPbWXN3KL5FsheQW+BQTHl0mqX8bqGBW9J0YqjzAdJD7JeNMNr9d 0gL19NYGia98iXBgonCp+UfJBxGTuLb+PIurKEXancG1vVsfbQDC6fYjMn4vybKItl61J/yZkw8 KdMzN86KrauXd3MZDU1/yv/AIRJw8cld6KwOZfpg8+h0lWiZNrgXt3XbMNFbKlj5YlS/zPM x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--6.422500-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: 2D1032A6F2573EDEA24CC3A4998FBCB0AE888AE2ED3D8D9F946C69D67D587FD42000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com This patch depends on patch 07. All rdevs running in RAID 1/10 switch nr_pending to atomic mode. The value of nr_pending is read in a normal operation (choose_best_rdev()). Therefore, nr_pending must always be consistent. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.h | 14 ++++++++++++++ drivers/md/raid1.c | 7 +++++++ drivers/md/raid10.c | 4 ++++ 3 files changed, 25 insertions(+) diff --git a/drivers/md/md.h b/drivers/md/md.h index ab09e312c9bb..57b09b567ffa 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -236,6 +236,20 @@ static inline unsigned long nr_pending_read(struct md_rdev *rdev) return atomic_long_read(&rdev->nr_pending.data->count); } +static inline bool nr_pending_is_percpu_mode(struct md_rdev *rdev) +{ + unsigned long __percpu *percpu_count; + + return __ref_is_percpu(&rdev->nr_pending, &percpu_count); +} + +static inline bool nr_pending_is_atomic_mode(struct md_rdev *rdev) +{ + unsigned long __percpu *percpu_count; + + return !__ref_is_percpu(&rdev->nr_pending, &percpu_count); +} + static inline int is_badblock(struct md_rdev *rdev, sector_t s, int sectors, sector_t *first_bad, int *bad_sectors) { diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 12318fb15a88..c38ae13aadab 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -784,6 +784,7 @@ static int choose_best_rdev(struct r1conf *conf, struct r1bio *r1_bio) if (ctl.readable_disks++ == 1) set_bit(R1BIO_FailFast, &r1_bio->state); + WARN_ON_ONCE(nr_pending_is_percpu_mode(rdev)); pending = nr_pending_read(rdev); dist = abs(r1_bio->sector - conf->mirrors[disk].head_position); @@ -1930,6 +1931,7 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev) if (err) return err; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); raid1_add_conf(conf, rdev, mirror, false); /* As all devices are equivalent, we don't need a full recovery * if this was recently any drive of the array @@ -1949,6 +1951,7 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev) set_bit(Replacement, &rdev->flags); raid1_add_conf(conf, rdev, repl_slot, true); err = 0; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); conf->fullsync = 1; } @@ -3208,6 +3211,7 @@ static void raid1_free(struct mddev *mddev, void *priv); static int raid1_run(struct mddev *mddev) { struct r1conf *conf; + struct md_rdev *rdev; int i; int ret; @@ -3269,6 +3273,9 @@ static int raid1_run(struct mddev *mddev) /* * Ok, everything is just fine now */ + rdev_for_each(rdev, mddev) { + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); + } rcu_assign_pointer(mddev->thread, conf->thread); rcu_assign_pointer(conf->thread, NULL); mddev->private = conf; diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index b91dd6c0be5a..66896a1076e1 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -808,6 +808,7 @@ static struct md_rdev *read_balance(struct r10conf *conf, nonrot = bdev_nonrot(rdev->bdev); has_nonrot_disk |= nonrot; + WARN_ON_ONCE(nr_pending_is_percpu_mode(rdev)); pending = nr_pending_read(rdev); if (min_pending > pending && nonrot) { min_pending = pending; @@ -2113,6 +2114,7 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev) p->recovery_disabled = mddev->recovery_disabled - 1; rdev->raid_disk = mirror; err = 0; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); if (rdev->saved_raid_disk != mirror) conf->fullsync = 1; WRITE_ONCE(p->rdev, rdev); @@ -2127,6 +2129,7 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev) err = mddev_stack_new_rdev(mddev, rdev); if (err) return err; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); conf->fullsync = 1; WRITE_ONCE(p->replacement, rdev); } @@ -4028,6 +4031,7 @@ static int raid10_run(struct mddev *mddev) rdev_for_each(rdev, mddev) { long long diff; + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); disk_idx = rdev->raid_disk; if (disk_idx < 0) continue; From patchwork Thu Apr 18 05:44:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634179 Received: from mo-csw.securemx.jp (mo-csw1120.securemx.jp [210.130.202.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59CFE2207A; Thu, 18 Apr 2024 05:44:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713419073; cv=none; b=GrwG7kOXHJcfdtX8LzSXzOcuzzK+S99aBk6tnDEig3mB+Qaubfvf5MvYZ5XgZLhOsmOs8+gtFu5jP/oLoGh6TzeKSQ5WU/C7GjnSQNFudN3yGRymOSdzJQa667rUhuMmYuZF89KNRWFfiM7COlPnY75GZiWD2FBJh3MTilC58Nk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713419073; c=relaxed/simple; bh=QaN/FwwxLxgD/eNrmiHrOkKp2TdxHm9YHgRtIBHbloc=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=LrwgwOrcix7KrNPlAds8B3l3BcuydTV8hJKDnqfzO6OJXXQeBfdAyz/9+u8npQ7KjpKiIPuzboOAVZJAmyvxbInEqM4+NlXZbSlRWJEQoSqG2YPLSkO/0M4rA/kc8jhRLE7aX55smLT9gwBb1mUi2tzopYJspX1fM+3z+RniK2I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw.securemx.jp (mx-mo-csw1120) id 43I5iP131975285; Thu, 18 Apr 2024 14:44:25 +0900 X-Iguazu-Qid: 2rWhyQ027DO2N9iLK4 X-Iguazu-QSIG: v=2; s=0; t=1713419065; q=2rWhyQ027DO2N9iLK4; m=Mt2wR6H6xjw2q0sCmoQEI6I6joL35fOEulPFNL/GHnM= Received: from CNN1EMTA02.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1123) id 43I5iOGF2720705 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:24 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA02.test.kioxia.com (Postfix) with ESMTP id 7DD7E2F02C; Thu, 18 Apr 2024 14:44:24 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 5E6C71947AEC50; Thu, 18 Apr 2024 14:22:26 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002QAAAAAA01G6B; Thu, 18 Apr 2024 14:22:26 +0900 Received: from CNN1EXMB04.r1.kioxia.com (CNN1EXMB04.r1.kioxia.com [10.232.20.153]) by Switcher-Pre_Send (Postfix) with ESMTP id 53248AA28483C; Thu, 18 Apr 2024 14:22:26 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB04.r1.kioxia.com (10.232.20.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:23 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:23 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 09/11] md: add atomic mode switching when removing disk Thread-Topic: [PATCH v2 09/11] md: add atomic mode switching when removing disk Thread-Index: AdqLx+cUYgqWX3MjQN2rrEDmXmwOEg== Date: Thu, 18 Apr 2024 05:44:23 +0000 Message-ID: <415ab1a1454a44129e997d51822ed533@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--8.269100-8.000000 x-tmase-matchedrid: SD3TfnVbAUqmagT1k9kBppTQgFTHgkhZKQNhMboqZlrGZbX3v50Wrzos Lf1kH5Cf9KYkbDev+bE52p2V9wXr31gyUUCO8mz8gnMtC97jHVTBOVz0Jwcxl6vCrG0TnfVUg9x e4gtUJtop7xmDsiWI7eSkRJ+6d2nBY+Xr17aLI9mWsmW9lDx463Fd5+Cf9M1DywYoiKCn+q+jxY yRBa/qJUl4W8WVUOR/joczmuoPCq10cYT4S+XRd6bHdlf3vR79QLl5NGHbunXMUg19c6LynGCQd ykTnIAa x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--8.269100-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: 74E5A45F46303C720AC2985D8927258BC727B23F1A5A878ACCE478F47854666A2000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB04.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB04.r1.kioxia.com This patch depends on patch 08. To minimize the number of execution of atomic mode switching, not only adding atomic mode switching, but also remove judgment is divided into two stages. Latency is minimized because only rdev to be removed is switched to atomic mode, not all rdevs. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 7e6966e65d0d..5f785353353d 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9291,8 +9291,11 @@ static bool rdev_removeable(struct md_rdev *rdev) return false; /* Fautly rdev is not used, it's safe to remove it. */ - if (test_bit(Faulty, &rdev->flags)) - return true; + if (nr_pending_is_percpu_mode(rdev) && test_bit(Faulty, &rdev->flags)) { + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); + if (nr_pending_is_zero(rdev)) + return true; + } /* Journal disk can only be removed if it's faulty. */ if (test_bit(Journal, &rdev->flags)) @@ -9346,6 +9349,11 @@ static bool md_spares_need_change(struct mddev *mddev) { struct md_rdev *rdev; + rdev_for_each_rcu(rdev, mddev) { + if (test_bit(Faulty, &rdev->flags)) + percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); + } + rcu_read_lock(); rdev_for_each_rcu(rdev, mddev) { if (rdev_removeable(rdev) || rdev_addable(rdev)) { From patchwork Thu Apr 18 05:44:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634180 Received: from mo-csw.securemx.jp (mo-csw1122.securemx.jp [210.130.202.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41E92657D5; Thu, 18 Apr 2024 05:44:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713419076; cv=none; b=kll6DUF9Akn4nLrmctvmKSVJ/8byPFkcLyTNLeARAMcq0PCranyL5uR9OcbTVj3MCX/gwMUtlijSRStBXlMoPUObYPZQPypVUGCacRTGw5HQk7zVgQuGbfL9BJqYEry+lPlSdfnlBLw/uk+x5Dc+Dlsdnr00atsCck2kD4nT6BM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713419076; c=relaxed/simple; bh=Dr1rG6/0S25koiym8xPwRcbH4k0au0GU3wRtTBO6+cg=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=UkEVdO7Q/hF+RZKYTk1W4q7KblvyTrHfrfV478B0taGOZTNCeTdk6nJveZOwm0T/o1He2MID2t+XTWhaYx48Q5Iw6Btc7wKkVUxs2X4sCa39mppwwRq+Da4V/Sw6GxmoEwdAsxtp/yq2FsjTOxMOFQKcVDoMKEp2fTcwAESMt7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 43I5iSUj2466659; Thu, 18 Apr 2024 14:44:28 +0900 X-Iguazu-Qid: 2rWhlfeQc0hVIkrQMI X-Iguazu-QSIG: v=2; s=0; t=1713419067; q=2rWhlfeQc0hVIkrQMI; m=aL4XsiZrZfC6mQirdHAqaPmL6EJv+ITI+WP6JsVjBio= Received: from CNN1EMTA01.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1121) id 43I5iR852637678 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:27 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA01.test.kioxia.com (Postfix) with ESMTP id 0CD0F48A81; Thu, 18 Apr 2024 14:44:27 +0900 (JST) Received: from CNN1ESTR04.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 55BF71900001EB; Thu, 18 Apr 2024 14:31:44 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR04.kioxia.com with ESMTP id 0003QAAAAAA02272; Thu, 18 Apr 2024 14:31:44 +0900 Received: from CNN1EXMB01.r1.kioxia.com (CNN1EXMB01.r1.kioxia.com [10.232.20.150]) by Switcher-Pre_Send (Postfix) with ESMTP id 4A793A0640E18; Thu, 18 Apr 2024 14:31:44 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB01.r1.kioxia.com (10.232.20.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:26 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:26 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 10/11] md: add atomic mode switching when I/O completion Thread-Topic: [PATCH v2 10/11] md: add atomic mode switching when I/O completion Thread-Index: AdqLx+iHrS2hQg22QTSH7MYR+7sBqw== Date: Thu, 18 Apr 2024 05:44:26 +0000 Message-ID: <4ee16fa91cd24fb3b5cbe5134e1831f0@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--8.850000-8.000000 x-tmase-matchedrid: Sx6Z8sZ1/uymagT1k9kBppTQgFTHgkhZKQNhMboqZlrGZbX3v50WrzPn 4yDW5yf3GSNOWaKP4s2t2un08Qrrt3EWVXH5btzQTuctSpiuWyUUi4Ehat054zdlsYL2g/87GyQ B1oR95AqsIJcbcumfxGPKQgbIX4yuhcAT8+NWWH8jRwcsjqWGAh90OsLQe4vlyIKHzIGoT62k16 eVRb8rL5dhlAAXYT7M7YL9OFIRb909S7tGIPO8TQ55FEYgJX09SKtHZwIquEv+Aw16GgqpO+m57 3tusZjQTqumDQCn8tgrsSQ3gQbaojcpdZ3fQiLdgxsfzkNRlfLdB/CxWTRRu25FeHtsUoHuA4fV KyofWMJ8xdQJxSQeWCjw6eAClxWBxKzBq1ALEkpLDBwYotNgRw== x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--8.850000-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: E39F34508721F4E44A2FAE932B2C219A9E3CD167EDB334131169ADAC146489792000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB01.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB01.r1.kioxia.com This patch depends on patch 09. If percpu mode in softirq context, switch to atomic mode in delayed execution. Move from softirq context to a context where RCUs are available and switch to atomic mode. This patch completes the addition of atomic mode switching. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 16 ++++++++++++++++ drivers/md/md.h | 6 ++++++ 2 files changed, 22 insertions(+) diff --git a/drivers/md/md.c b/drivers/md/md.c index 5f785353353d..18d0f214098b 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -9606,6 +9606,7 @@ void md_check_recovery(struct mddev *mddev) return; if ( ! ( (mddev->sb_flags & ~ (1<recovery) || test_bit(MD_RECOVERY_NEEDED, &mddev->recovery) || test_bit(MD_RECOVERY_DONE, &mddev->recovery) || (mddev->external == 0 && mddev->safemode == 1) || @@ -9616,6 +9617,21 @@ void md_check_recovery(struct mddev *mddev) if (mddev_trylock(mddev)) { bool try_set_sync = mddev->safemode != 0; + struct md_rdev *fault_rdev; + bool faulty_some = false; + + rdev_for_each(fault_rdev, mddev) { + if (fault_rdev->raid_disk >= 0 && + test_bit(Faulty, &fault_rdev->flags) && + nr_pending_is_percpu_mode(fault_rdev)) { + percpu_ref_switch_to_atomic_sync(&fault_rdev->nr_pending); + faulty_some = true; + } + } + if (faulty_some) { + clear_bit(MD_RECOVERY_PERCPU, &mddev->recovery); + goto unlock; + } if (!mddev->external && mddev->safemode == 1) mddev->safemode = 0; diff --git a/drivers/md/md.h b/drivers/md/md.h index 57b09b567ffa..a3c0b5e8cf05 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -605,6 +605,7 @@ enum recovery_flags { MD_RECOVERY_FROZEN, /* User request to abort, and not restart, any action */ MD_RECOVERY_ERROR, /* sync-action interrupted because io-error */ MD_RECOVERY_WAIT, /* waiting for pers->start() to finish */ + MD_RECOVERY_PERCPU, /* nr_pending when faulty needs to be switched to atomic */ MD_RESYNCING_REMOTE, /* remote node is running resync thread */ }; @@ -887,6 +888,11 @@ static inline void rdev_dec_pending(struct md_rdev *rdev, struct mddev *mddev) int faulty = test_bit(Faulty, &rdev->flags); nr_pending_dec(rdev); if (faulty) { + if (nr_pending_is_percpu_mode(rdev)) { + set_bit(MD_RECOVERY_PERCPU, &mddev->recovery); + md_wakeup_thread(mddev->thread); + return; + } if (nr_pending_is_zero(rdev)) { set_bit(MD_RECOVERY_NEEDED, &mddev->recovery); md_wakeup_thread(mddev->thread); From patchwork Thu Apr 18 05:44:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: tada keisuke X-Patchwork-Id: 13634181 Received: from mo-csw.securemx.jp (mo-csw1122.securemx.jp [210.130.202.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E749354BFA; Thu, 18 Apr 2024 05:44:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=210.130.202.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713419083; cv=none; b=UQkKXpzfblWgX3SJMEeIpmKf0Bi45xAnbA3amllIn+SbcXJcylfio5Tz/Q1LKlrhMI/tKdkWjq+13vU9KIXK4nIUgpXEoX7YKrq4AptsdV0nI0MzFBrGW/U7R5Vfe5O7alvEA1PqddFvkLL4A16r/SJqUp++pgfQY3+v4ctIwm4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713419083; c=relaxed/simple; bh=5AKQlpoYM0KfuVMzu8IcfAJeK3YnU7fhWnIQ1LwmVZc=; h=From:To:CC:Subject:Date:Message-ID:Content-Type:MIME-Version; b=OE0pbmYtrKOD0JN+4YdeVgxz3P6IlHHD5NANpAJWX9ds+kfC0onk6PjdI4hT0IK2HzSMOjFMAvtrpoNukm+m2RgMF08R/+2Kd6wkhhHIlfLlODjean8b9JswbxJu6BKTUaSnT1A25Qk5TuEoDjKoX6ek7lyyMlrzN7spwhp4W2Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com; spf=pass smtp.mailfrom=kioxia.com; arc=none smtp.client-ip=210.130.202.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=kioxia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kioxia.com Received: by mo-csw.securemx.jp (mx-mo-csw1122) id 43I5iVL12466845; Thu, 18 Apr 2024 14:44:31 +0900 X-Iguazu-Qid: 2rWhYYPQE6cvIqaCBK X-Iguazu-QSIG: v=2; s=0; t=1713419070; q=2rWhYYPQE6cvIqaCBK; m=+A0GV4v/o1qCmYQ2W8uT41Kerdyf0z7RdGhIMFF3p7c= Received: from CNN1EMTA03.test.kioxia.com ([202.248.33.144]) by relay.securemx.jp (mx-mr1122) id 43I5iUsF2852846 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Thu, 18 Apr 2024 14:44:30 +0900 Received: from Switcher-Post_Send (gateway [10.232.20.1]) by CNN1EMTA03.test.kioxia.com (Postfix) with ESMTP id 451602FE0F; Thu, 18 Apr 2024 14:44:30 +0900 (JST) Received: from CNN1ESTR03.kioxia.com (localhost [127.0.0.1]) by Switcher-Post_Send (Postfix) with ESMTP id 1C49B1947AEC50; Thu, 18 Apr 2024 14:22:32 +0900 (JST) Received: from localhost [127.0.0.1] by CNN1ESTR03.kioxia.com with ESMTP id 0002QAAAAAA01G8B; Thu, 18 Apr 2024 14:22:32 +0900 Received: from CNN1EXMB01.r1.kioxia.com (CNN1EXMB01.r1.kioxia.com [10.232.20.150]) by Switcher-Pre_Send (Postfix) with ESMTP id 10C7AAA28483C; Thu, 18 Apr 2024 14:22:32 +0900 (JST) Received: from CNN1EXMB04.r1.kioxia.com (10.232.20.153) by CNN1EXMB01.r1.kioxia.com (10.232.20.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 18 Apr 2024 14:44:29 +0900 Received: from CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) by CNN1EXMB04.r1.kioxia.com ([10.13.100.23]) with mapi id 15.01.2507.035; Thu, 18 Apr 2024 14:44:29 +0900 From: tada keisuke To: "song@kernel.org" , "yukuai3@huawei.com" CC: "linux-raid@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: [PATCH v2 11/11] md: avoid conflicts in nr_pending operations Thread-Topic: [PATCH v2 11/11] md: avoid conflicts in nr_pending operations Thread-Index: AdqLx+onlYFlF64nSFSWBn9s8h2iIA== Date: Thu, 18 Apr 2024 05:44:29 +0000 Message-ID: <03f55e87cd7e4fae9fad86bc70d1870e@kioxia.com> Accept-Language: ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-tm-as-product-ver: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-as-result: No-10--9.404500-8.000000 x-tmase-matchedrid: y9IfzGOMTk+magT1k9kBpgrcxrzwsv5uXPK9y3z82GsPdZmyx8xAa4Ul JlxlTIT+tPwtczFrCbS12HagvbwDji/7QU2czuUNA9lly13c/gFezmeoa8MJ86G5P8xDq4BXoK5 B6+qXU7BvF9oQ/B54yN5WE7eMYlPKhrD3pNcSx1aqFx2c/3V5cbs5i+al07BJUrMN6XtW1EtvXF TtvgJmGhSu3n/oQBOZmsmH6sEd2eHyAV/ow31xqxMxKDqgAFSzuoYFb0nRiqNULhp+W/a9mWCGk K2P98wo585VzGMOFzDYh0PvnvUH8AtuKBGekqUpI/NGWt0UYPCcSjlacIPfpspOLsvV7od/l+l1 5K/AK/bhBM8s3vZTUcLAQYGtsIDO x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No x-tmase-result: 10--9.404500-8.000000 x-tmase-version: ISME-14.0.0.2080-9.0.1002-28328.001 x-tm-snts-smtp: F1F6EA9BC13B9D1ADCD72C40C43E9525B480FF568C2E6CD717CDD52AAD9A60472000:8 Precedence: bulk X-Mailing-List: linux-raid@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CrossPremisesHeadersFilteredBySendConnector: CNN1EXMB01.r1.kioxia.com X-OrganizationHeadersPreserved: CNN1EXMB01.r1.kioxia.com This patch depends on patch 10. Change nr_pending from atomic mode to percpu mode. Signed-off-by: Keisuke TADA Signed-off-by: Toshifumi OHTAKE --- drivers/md/md.c | 5 +++-- drivers/md/raid1.c | 6 ++++++ drivers/md/raid10.c | 6 ++++++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 18d0f214098b..4928d63209cb 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -3746,7 +3746,6 @@ int md_rdev_init(struct md_rdev *rdev) ret = percpu_ref_init(&rdev->nr_pending, percpu_wakeup_handle_req_pending, PERCPU_REF_ALLOW_REINIT, GFP_KERNEL); WARN_ON(ret); - percpu_ref_switch_to_atomic_sync(&rdev->nr_pending); nr_pending_dec(rdev); atomic_set(&rdev->read_errors, 0); atomic_set(&rdev->corrected_errors, 0); @@ -9280,7 +9279,7 @@ static bool rdev_removeable(struct md_rdev *rdev) return false; /* There are still inflight io, don't remove this rdev. */ - if (nr_pending_is_not_zero(rdev)) + if (nr_pending_is_atomic_mode(rdev) && nr_pending_is_not_zero(rdev)) return false; /* @@ -9384,6 +9383,8 @@ static int remove_and_add_spares(struct mddev *mddev, rdev->raid_disk = -1; removed++; } + if (mddev->pers->level != 1 && mddev->pers->level != 10) + percpu_ref_switch_to_percpu(&rdev->nr_pending); } if (removed && mddev->kobj.sd) diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index c38ae13aadab..13fcb80e1cb0 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -2006,6 +2006,7 @@ static int raid1_remove_disk(struct mddev *mddev, struct md_rdev *rdev) conf->mirrors[conf->raid_disks + number].rdev = NULL; unfreeze_array(conf); } + percpu_ref_switch_to_percpu(&rdev->nr_pending); clear_bit(WantReplacement, &rdev->flags); err = md_integrity_register(mddev); @@ -3298,6 +3299,11 @@ static int raid1_run(struct mddev *mddev) static void raid1_free(struct mddev *mddev, void *priv) { struct r1conf *conf = priv; + struct md_rdev *rdev; + + rdev_for_each(rdev, mddev) { + percpu_ref_switch_to_percpu(&rdev->nr_pending); + } mempool_exit(&conf->r1bio_pool); kfree(conf->mirrors); diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 66896a1076e1..f8ea5ec22629 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -2180,6 +2180,7 @@ static int raid10_remove_disk(struct mddev *mddev, struct md_rdev *rdev) clear_bit(Replacement, &p->replacement->flags); WRITE_ONCE(p->replacement, NULL); } + percpu_ref_switch_to_percpu(&rdev->nr_pending); clear_bit(WantReplacement, &rdev->flags); err = md_integrity_register(mddev); @@ -4168,6 +4169,11 @@ static int raid10_run(struct mddev *mddev) static void raid10_free(struct mddev *mddev, void *priv) { + struct md_rdev *rdev; + + rdev_for_each(rdev, mddev) { + percpu_ref_switch_to_percpu(&rdev->nr_pending); + } raid10_free_conf(priv); }