[03/16] mds: don't add not issued caps when confirming cap receipt

Commit Message

Yan, Zheng Nov. 19, 2012, 2:43 a.m. UTC

From: "Yan, Zheng" <[e-mail]>

There is message ordering race in cephfs kernel client. We compose
cap messages when i_ceph_lock is hold. But when adding messages
to the output queue, the kernel releases i_ceph_lock and acquires
a mutex. So it is possible that cap messages are send out of order.
If the kernel client send a cap update, then send a cap release,
but the two messages reach MDS out of order. The update message
will re-add the released caps. This patch adds code to check if
caps were actually issued when confirming cap receipt.

Signed-off-by: Yan, Zheng <[e-mail]>
---
 src/mds/Locker.cc | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

Patch

diff --git a/src/mds/Locker.cc b/src/mds/Locker.cc
index 9d91748..c29ac34 100644
--- a/src/mds/Locker.cc
+++ b/src/mds/Locker.cc
@@ -2287,11 +2287,17 @@  void Locker::handle_client_caps(MClientCaps *m)
  
     // head inode, and cap
     MClientCaps *ack = 0;
+
+    int caps = m->get_caps();
+    if (caps & ~cap->issued()) {
+      dout(10) << " confirming not issued caps " << ccap_string(caps & ~cap->issued()) << dendl;
+      caps &= cap->issued();
+    }
     
-    cap->confirm_receipt(m->get_seq(), m->get_caps());
+    cap->confirm_receipt(m->get_seq(), caps);
     dout(10) << " follows " << follows
 	     << " retains " << ccap_string(m->get_caps())
-	     << " dirty " << ccap_string(m->get_caps())
+	     << " dirty " << ccap_string(m->get_dirty())
 	     << " on " << *in << dendl;
 
 
@@ -2424,6 +2430,10 @@  void Locker::process_request_cap_release(MDRequest *mdr, client_t client, const
     return;
   }
     
+  if (caps & ~cap->issued()) {
+    dout(10) << " confirming not issued caps " << ccap_string(caps & ~cap->issued()) << dendl;
+    caps &= cap->issued();
+  }
   cap->confirm_receipt(seq, caps);
   adjust_cap_wanted(cap, wanted, issue_seq);