From patchwork Tue Aug 27 21:05:15 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 2850359 Return-Path: X-Original-To: patchwork-ocfs2-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 7DD709F485 for ; Tue, 27 Aug 2013 21:07:02 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 5AC4B2041D for ; Tue, 27 Aug 2013 21:07:01 +0000 (UTC) Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 47A8E2041C for ; Tue, 27 Aug 2013 21:07:00 +0000 (UTC) Received: from acsinet21.oracle.com (acsinet21.oracle.com [141.146.126.237]) by aserp1040.oracle.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.1) with ESMTP id r7RL6qSq010376 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 27 Aug 2013 21:06:52 GMT Received: from oss.oracle.com (oss-external.oracle.com [137.254.96.51]) by acsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id r7RL6pm5014365 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 27 Aug 2013 21:06:52 GMT Received: from localhost ([127.0.0.1] helo=oss.oracle.com) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1VEQTT-0001sU-OD; Tue, 27 Aug 2013 14:06:51 -0700 Received: from ucsinet21.oracle.com ([156.151.31.93]) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1VEQRy-0001dv-BM for ocfs2-devel@oss.oracle.com; Tue, 27 Aug 2013 14:05:18 -0700 Received: from userp1020.oracle.com (userp1020.oracle.com [156.151.31.79]) by ucsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id r7RL5HbS020661 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Tue, 27 Aug 2013 21:05:18 GMT Received: from mail-qe0-f74.google.com (mail-qe0-f74.google.com [209.85.128.74]) by userp1020.oracle.com (Sentrion-MTA-4.3.1/Sentrion-MTA-4.3.1) with ESMTP id r7RL5Gtg016212 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=OK) for ; Tue, 27 Aug 2013 21:05:17 GMT Received: by mail-qe0-f74.google.com with SMTP id 1so560503qee.3 for ; Tue, 27 Aug 2013 14:05:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:subject:to:cc:from:date:mime-version :content-type:content-transfer-encoding:message-id; bh=AEUpbhPpHiMyFxLNYmccxv4SvyjTy/K9NQowom6zRpk=; b=F2rs3um4G+Clv+Zr/iiOWgbs9lwUHYbWFPu34U+ATaKU8q6zL6bbDgKYRG+kptQwHu SNsxr/hw+5ytKJanzmcMV9G75/RzxvXir430BLUOeQRtpXBThAhUSdlb42BUdLX8lyY0 6TlIyUxoe6hYREZ4Q+eXUDs1LlK5pJN7yvwnVhlDZU2XZmrYTmKl/bUuzqvSZA1OF4HV eEv/07f1f19QclE388twJyp4o66S0KaZDEIQfKEbVifgZfVhT+gOcVW13VtqV/jwkoL3 C/j3zb9OTbngOt3mXiBqhnB/axweqS+omKVCkUm56YEoX7YFSgBq/MbumJsOlXwHIEh9 WN0A== X-Gm-Message-State: ALoCoQmqNBXjo1qN9xEKU70puaruxv2QUfegziavHsadgwG4AiDSCtv3qvEYVbSs/WNUN5HSnuDE X-Received: by 10.236.167.138 with SMTP id i10mr9138229yhl.9.1377637516512; Tue, 27 Aug 2013 14:05:16 -0700 (PDT) Received: from corp2gmr1-2.hot.corp.google.com (corp2gmr1-2.hot.corp.google.com [172.24.189.93]) by gmr-mx.google.com with ESMTPS id s21si1407809yhc.1.1969.12.31.16.00.00 (version=TLSv1.1 cipher=AES128-SHA bits=128/128); Tue, 27 Aug 2013 14:05:16 -0700 (PDT) Received: from localhost.localdomain (akpm3.mtv.corp.google.com [172.17.131.127]) by corp2gmr1-2.hot.corp.google.com (Postfix) with ESMTP id C922D5A4276; Tue, 27 Aug 2013 14:05:15 -0700 (PDT) To: ocfs2-devel@oss.oracle.com From: akpm@linux-foundation.org Date: Tue, 27 Aug 2013 14:05:15 -0700 MIME-Version: 1.0 Message-Id: <20130827210515.C922D5A4276@corp2gmr1-2.hot.corp.google.com> X-Flow-Control-Info: class=Pass-to-MM reputation=ipRisk-All ip=209.85.128.74 ct-class=R4 ct-vol1=52 ct-vol2=6 ct-vol3=5 ct-risk=32 ct-spam1=43 ct-spam2=10 ct-bulk=67 rcpts=1 size=3813 X-Sendmail-CM-Score: 0.00% X-Sendmail-CM-Analysis: v=2.1 cv=Na/fiQz4 c=1 sm=1 tr=0 a=l12VVL2GhpU4TSYhu0SuwA==:117 a=wE6vNewnYyUA:10 a=NEiEQogP1MkA:10 a=os2CZ2fo8YAA:10 a=Z4Rwk6OoAAAA:8 a=1XWaLZrsAAAA:8 a=yPCof4ZbAAAA:8 a=r5uDilac-oYA:10 a=i0EeH86SAAAA:8 a=IXr_WNlcAAAA:8 a=iox4zFpeAAAA:8 a=fT33CZOoQcHokhLpFtMA:9 a=BxZ1OnTuqITPg3jg:21 a=eHWpBSg6plSbITVE:21 a=e4xtJxf3HDoA:10 a=hPjdaMEvmhQA:10 a=T5ZRoNnfl4MA:10 a=n9GBPR9yFnkA:10 a=jbrJJM5MRmoA:10 X-Sendmail-CT-Classification: not spam X-Sendmail-CT-RefID: str=0001.0A090203.521D148D.00DD, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 Cc: mfasheh@suse.com Subject: [Ocfs2-devel] [patch 21/22] ocfs2: fix a tiny race case when firing callbacks X-BeenThere: ocfs2-devel@oss.oracle.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ocfs2-devel-bounces@oss.oracle.com Errors-To: ocfs2-devel-bounces@oss.oracle.com X-Source-IP: acsinet21.oracle.com [141.146.126.237] X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Joseph Qi Subject: ocfs2: fix a tiny race case when firing callbacks In o2hb_shutdown_slot() and o2hb_check_slot(), since event is defined as local, it is only valid during the call stack. So the following tiny race case may happen in a multi-volumes mounted environment: o2hb-vol1 o2hb-vol2 1) o2hb_shutdown_slot allocate local event1 2) queue_node_event add event1 to global o2hb_node_events 3) o2hb_shutdown_slot allocate local event2 4) queue_node_event add event2 to global o2hb_node_events 5) o2hb_run_event_list delete event1 from o2hb_node_events 6) o2hb_run_event_list event1 empty, return 7) o2hb_shutdown_slot event1 lifecycle ends 8) o2hb_fire_callbacks event1 is already *invalid* This patch lets it wait on o2hb_callback_sem when another thread is firing callbacks. And for performance consideration, we only call o2hb_run_event_list when there is an event queued. Signed-off-by: Joyce Signed-off-by: Joseph Qi Cc: Joel Becker Cc: Mark Fasheh Signed-off-by: Andrew Morton --- fs/ocfs2/cluster/heartbeat.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff -puN fs/ocfs2/cluster/heartbeat.c~ocfs2-fix-a-tiny-race-case-when-firing-callbacks fs/ocfs2/cluster/heartbeat.c --- a/fs/ocfs2/cluster/heartbeat.c~ocfs2-fix-a-tiny-race-case-when-firing-callbacks +++ a/fs/ocfs2/cluster/heartbeat.c @@ -639,16 +639,9 @@ static void o2hb_fire_callbacks(struct o /* Will run the list in order until we process the passed event */ static void o2hb_run_event_list(struct o2hb_node_event *queued_event) { - int empty; struct o2hb_callback *hbcall; struct o2hb_node_event *event; - spin_lock(&o2hb_live_lock); - empty = list_empty(&queued_event->hn_item); - spin_unlock(&o2hb_live_lock); - if (empty) - return; - /* Holding callback sem assures we don't alter the callback * lists when doing this, and serializes ourselves with other * processes wanting callbacks. */ @@ -707,6 +700,7 @@ static void o2hb_shutdown_slot(struct o2 struct o2hb_node_event event = { .hn_item = LIST_HEAD_INIT(event.hn_item), }; struct o2nm_node *node; + int queued = 0; node = o2nm_get_node_by_num(slot->ds_node_num); if (!node) @@ -724,11 +718,13 @@ static void o2hb_shutdown_slot(struct o2 o2hb_queue_node_event(&event, O2HB_NODE_DOWN_CB, node, slot->ds_node_num); + queued = 1; } } spin_unlock(&o2hb_live_lock); - o2hb_run_event_list(&event); + if (queued) + o2hb_run_event_list(&event); o2nm_node_put(node); } @@ -788,6 +784,7 @@ static int o2hb_check_slot(struct o2hb_r unsigned int dead_ms = o2hb_dead_threshold * O2HB_REGION_TIMEOUT_MS; unsigned int slot_dead_ms; int tmp; + int queued = 0; memcpy(hb_block, slot->ds_raw_block, reg->hr_block_bytes); @@ -881,6 +878,7 @@ fire_callbacks: slot->ds_node_num); changed = 1; + queued = 1; } list_add_tail(&slot->ds_live_item, @@ -932,6 +930,7 @@ fire_callbacks: node, slot->ds_node_num); changed = 1; + queued = 1; } /* We don't clear this because the node is still @@ -947,7 +946,8 @@ fire_callbacks: out: spin_unlock(&o2hb_live_lock); - o2hb_run_event_list(&event); + if (queued) + o2hb_run_event_list(&event); if (node) o2nm_node_put(node);