From patchwork Thu Jul 19 14:30:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiubo Li X-Patchwork-Id: 10534633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A7935600D0 for ; Thu, 19 Jul 2018 14:31:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9428C29CA1 for ; Thu, 19 Jul 2018 14:31:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 921D129B1F; Thu, 19 Jul 2018 14:31:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E973129C4C for ; Thu, 19 Jul 2018 14:31:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731110AbeGSPOf (ORCPT ); Thu, 19 Jul 2018 11:14:35 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:48988 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727441AbeGSPOf (ORCPT ); Thu, 19 Jul 2018 11:14:35 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CA15B402243B; Thu, 19 Jul 2018 14:31:07 +0000 (UTC) Received: from rhel3.localdomain (ovpn-12-194.pek2.redhat.com [10.72.12.194]) by smtp.corp.redhat.com (Postfix) with ESMTP id 967DE1C5B8; Thu, 19 Jul 2018 14:31:03 +0000 (UTC) From: xiubli@redhat.com To: linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, nab@linux-iscsi.org, martin.petersen@oracle.com, mchristi@redhat.com Cc: pkalever@redhat.com Subject: [PATCH] tcmu: fix crash for dereferencing the released udev->mb_addr memory Date: Thu, 19 Jul 2018 10:30:59 -0400 Message-Id: <1532010659-4657-1-git-send-email-xiubli@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 19 Jul 2018 14:31:07 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 19 Jul 2018 14:31:07 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'xiubli@redhat.com' RCPT:'' Sender: target-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Xiubo Li The logs are: BUG: unable to handle kernel NULL pointer dereference at 0000000000000040 IP: [] tcmu_reset_ring_store+0x149/0x240 [target_core_user] PGD 800000000e254067 PUD e255067 PMD 0 Oops: 0002 [#1] SMP [...] CPU: 0 PID: 36077 Comm: tcmu-runner Kdump: loaded Not tainted 3.10.0-924.el7.test.x86_64 #1 Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017 task: ffff922db95ab0c0 ti: ffff922d9f8d4000 task.ti: ffff922d9f8d4000 RIP: 0010:[] [] tcmu_reset_ring_store+0x149/0x240 [target_core_user] RSP: 0018:ffff922d9f8d7e30 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000001000 RCX: 00000000c0000100 RDX: ffff922d9f8d5fd8 RSI: 0000000000000000 RDI: ffff922d4b91f440 RBP: ffff922d9f8d7e70 R08: 0000000000000000 R09: 0000000000000001 R10: 0000000000000000 R11: 0000000000000000 R12: ffff922d4b91e550 R13: ffff922d4b91f3e8 R14: 0000000000000000 R15: 0000000000000000 FS: 00007f70467d7880(0000) GS:ffff922dbb600000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000000000040 CR3: 000000000a2b0000 CR4: 00000000003607f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: [] configfs_write_file+0x11f/0x160 From the crash tools: crash> bt PID: 36077 TASK: ffff922db95ab0c0 CPU: 0 COMMAND: "tcmu-runner" #0 [ffff922d9f8d7ac8] machine_kexec at ffffffffabe62d8a #1 [ffff922d9f8d7b28] __crash_kexec at ffffffffabf1bb02 #2 [ffff922d9f8d7bf8] crash_kexec at ffffffffabf1bbf0 #3 [ffff922d9f8d7c10] oops_end at ffffffffac564798 #4 [ffff922d9f8d7c38] no_context at ffffffffac552b3b #5 [ffff922d9f8d7c88] __bad_area_nosemaphore at ffffffffac552bd2 #6 [ffff922d9f8d7cd8] bad_area_nosemaphore at ffffffffac552d43 #7 [ffff922d9f8d7ce8] __do_page_fault at ffffffffac567750 #8 [ffff922d9f8d7d50] do_page_fault at ffffffffac567945 #9 [ffff922d9f8d7d80] page_fault at ffffffffac563788 [exception RIP: tcmu_reset_ring_store+329] RIP: ffffffffc072b9a9 RSP: ffff922d9f8d7e30 RFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000001000 RCX: 00000000c0000100 RDX: ffff922d9f8d5fd8 RSI: 0000000000000000 RDI: ffff922d4b91f440 RBP: ffff922d9f8d7e70 R8: 0000000000000000 R9: 0000000000000001 R10: 0000000000000000 R11: 0000000000000000 R12: ffff922d4b91e550 R13: ffff922d4b91f3e8 R14: 0000000000000000 R15: 0000000000000000 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 RIP: 00007f70458a074d RSP: 00007ffcf788dc50 RFLAGS: 00000293 RAX: 0000000000000001 RBX: 0000000000000000 RCX: ffffffffffffffff RDX: 0000000000000002 RSI: 00007ffcf788dce0 RDI: 0000000000000007 RBP: 00007ffcf788dcc0 R8: 0000000000000000 R9: 00007f7044cd10fd R10: 00007ffcf788e720 R11: 0000000000000293 R12: 0000000000407c80 R13: 00007ffcf788f170 R14: 0000000000000000 R15: 0000000000000000 ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b We can see that the IP is tcmu_reset_ring_store+329. crash> dis tcmu_reset_ring_store [...] 0xffffffffc072b955 : callq 0xffffffffac019810 0xffffffffc072b95a : jmpq 0xffffffffc072b8d8 0xffffffffc072b95f : nop [...] 0xffffffffc072b98e : jne 0xffffffffc072b988 0xffffffffc072b990 : testb $0x4,0x3efb(%rip) # 0xffffffffc072f892 0xffffffffc072b997 : jne 0xffffffffc072ba6e 0xffffffffc072b99d : movl $0x0,0xe74(%r12) 0xffffffffc072b9a9 : movl $0x0,0x40(%r14) 0xffffffffc072b9b1 : movl $0x0,0xc(%r14) 0xffffffffc072b9b9 : nopl 0x0(%rax) 0xffffffffc072b9c0 : sub $0x1000,%rbx 0xffffffffc072b9c7 : jne 0xffffffffc072b9c0 0xffffffffc072b9c9 : lea 0xf18(%r12),%rdi 0xffffffffc072b9d1 : callq 0xffffffffabea8b00 [...] And the related target_core_user.c C code for line: 0xffffffffc072b9a9 : movl $0x0,0x40(%r14) is "mb->cmd_tail = 0;" Signed-off-by: Xiubo Li --- drivers/target/target_core_user.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c index 847707a..8d7274e 100644 --- a/drivers/target/target_core_user.c +++ b/drivers/target/target_core_user.c @@ -1587,16 +1587,16 @@ static void tcmu_dev_kref_release(struct kref *kref) bool all_expired = true; int i; - vfree(udev->mb_addr); - udev->mb_addr = NULL; - spin_lock_bh(&timed_out_udevs_lock); if (!list_empty(&udev->timedout_entry)) list_del(&udev->timedout_entry); spin_unlock_bh(&timed_out_udevs_lock); - /* Upper layer should drain all requests before calling this */ mutex_lock(&udev->cmdr_lock); + vfree(udev->mb_addr); + udev->mb_addr = NULL; + + /* Upper layer should drain all requests before calling this */ idr_for_each_entry(&udev->commands, cmd, i) { if (tcmu_check_and_free_pending_cmd(cmd) != 0) all_expired = false;