From patchwork Tue May 29 03:22:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongsheng Yang X-Patchwork-Id: 10434301 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A134A60327 for ; Tue, 29 May 2018 03:29:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 91925285AA for ; Tue, 29 May 2018 03:29:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86488285AF; Tue, 29 May 2018 03:29:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54E7D285AA for ; Tue, 29 May 2018 03:29:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932749AbeE2D3F (ORCPT ); Mon, 28 May 2018 23:29:05 -0400 Received: from m50210.mail.qiye.163.com ([123.125.50.210]:48960 "EHLO m50210.mail.qiye.163.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932463AbeE2D3B (ORCPT ); Mon, 28 May 2018 23:29:01 -0400 X-Greylist: delayed 356 seconds by postgrey-1.27 at vger.kernel.org; Mon, 28 May 2018 23:29:00 EDT Received: from atest-guest.localdomain (unknown [218.94.118.90]) by smtp6 (Coremail) with SMTP id RNOowEBZp0SQxwxbJuAoAA--.13S2; Tue, 29 May 2018 11:22:56 +0800 (CST) From: Dongsheng Yang To: idryomov@gmail.com, jdillama@redhat.com Cc: ceph-devel@vger.kernel.org, Dongsheng Yang Subject: [PATCH 1/2] rbd: don't queue watch delayed work when we are removing device Date: Mon, 28 May 2018 23:22:40 -0400 Message-Id: <1527564161-17328-1-git-send-email-dongsheng.yang@easystack.cn> X-Mailer: git-send-email 1.8.3.1 X-CM-TRANSID: RNOowEBZp0SQxwxbJuAoAA--.13S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxGF4kWF48Aw4fJFy5WF13XFb_yoW5KFW3pr 45Ga4UKr48Jr1jqF48Aw1UX3W3Ja1qkFykWr1xA347CF1rGw1xAr1IkFy7AF1UGr1UZrW3 Jr1rX3yft34jgaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x0JbSmhrUUUUU= X-Originating-IP: [218.94.118.90] X-CM-SenderInfo: 5grqw2pkhqwhp1dqwq5hdv52pwdfyhdfq/1tbiVBBVelf4pg5O2QAAs7 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We will cancel all watch delayed work in cancel_delayed_work_sync(&rbd_dev->watch_dwork); If we queue delayed work after this, there will be a use-after-free problem: [ 549.932085] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 [ 549.934134] PGD 0 P4D 0 [ 549.935145] Oops: 0000 [#1] SMP PTI [ 549.936283] Modules linked in: rbd(OE) libceph(OE) tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag dns_resolver ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sg cfg80211 rfkill snd_hda_codec_generic ext4 snd_hda_intel snd_hda_codec crct10dif_pclmul crc32_pclmul ghash_clmulni_intel snd_hda_core pcbc snd_hwdep snd_seq mbcache aesni_intel snd_seq_device jbd2 crypto_simd nfsd cryptd glue_helper snd_pcm snd_timer auth_rpcgss pcspkr snd virtio_balloon nfs_acl soundcore i2c_piix4 lockd grace sunrpc ip_tables xfs libcrc32c virtio_console virtio_blk ata_generic pata_acpi 8139too qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crc32c_intel virtio_pci 8139cp virtio_ring i2c_core mii virtio floppy serio_raw dm_mirror dm_region_hash [ 549.951835] dm_log dm_mod dax [last unloaded: libceph] [ 549.953490] CPU: 7 PID: 0 Comm: swapper/7 Tainted: G OE 4.17.0-rc6+ #13 [ 549.955502] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 [ 549.957246] RIP: 0010:__queue_work+0x6a/0x3b0 [ 549.958744] RSP: 0018:ffff9427df1c3e90 EFLAGS: 00010086 [ 549.960374] RAX: ffff9427deca8400 RBX: 0000000000000000 RCX: 0000000000000000 [ 549.962297] RDX: ffff9427deca8400 RSI: ffff9427df1c3e50 RDI: 0000000000000000 [ 549.964216] RBP: ffff942783e39e00 R08: ffff9427deca8400 R09: ffff9427df1c3f00 [ 549.966136] R10: 0000000000000004 R11: 0000000000000005 R12: ffff9427cfb85970 [ 549.968070] R13: 0000000000002000 R14: 000000000001eca0 R15: 0000000000000007 [ 549.969999] FS: 0000000000000000(0000) GS:ffff9427df1c0000(0000) knlGS:0000000000000000 [ 549.972069] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 549.973775] CR2: 0000000000000000 CR3: 00000004c900a005 CR4: 00000000000206e0 [ 549.975695] Call Trace: [ 549.976900] [ 549.978033] ? __queue_work+0x3b0/0x3b0 [ 549.979442] call_timer_fn+0x2d/0x130 [ 549.980824] run_timer_softirq+0x16e/0x430 [ 549.982263] ? tick_sched_timer+0x37/0x70 [ 549.983691] __do_softirq+0xd2/0x280 [ 549.985035] irq_exit+0xd5/0xe0 [ 549.986316] smp_apic_timer_interrupt+0x6c/0x130 [ 549.987835] apic_timer_interrupt+0xf/0x20 This patch forbid to queue watch_dwork when we are removing device. Signed-off-by: Dongsheng Yang --- drivers/block/rbd.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 2b4e90d..d1d8f46 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -3475,9 +3475,13 @@ static void rbd_reregister_watch(struct work_struct *work) set_bit(RBD_DEV_FLAG_BLACKLISTED, &rbd_dev->flags); wake_requests(rbd_dev, true); } else { - queue_delayed_work(rbd_dev->task_wq, - &rbd_dev->watch_dwork, - RBD_RETRY_DELAY); + spin_lock_irq(&rbd_dev->lock); + if (!test_bit(RBD_DEV_FLAG_REMOVING, &rbd_dev->flags)) { + queue_delayed_work(rbd_dev->task_wq, + &rbd_dev->watch_dwork, + RBD_RETRY_DELAY); + } + spin_unlock_irq(&rbd_dev->lock); } mutex_unlock(&rbd_dev->watch_mutex); return;