From patchwork Wed Jul 11 13:16:39 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Pascal de Bruijn | Unilogic B.V." X-Patchwork-Id: 1182141 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 12B913FC8E for ; Wed, 11 Jul 2012 13:40:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757823Ab2GKNjs (ORCPT ); Wed, 11 Jul 2012 09:39:48 -0400 Received: from mail-out117.unilogicnetworks.net ([62.133.206.117]:55288 "EHLO mail.unilogicnetworks.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1757692Ab2GKNjn (ORCPT ); Wed, 11 Jul 2012 09:39:43 -0400 X-Greylist: delayed 1379 seconds by postgrey-1.27 at vger.kernel.org; Wed, 11 Jul 2012 09:39:43 EDT Received: from router (router.unilogic.nl [62.133.193.82]) by mail.unilogicnetworks.net (Postfix) with ESMTP id 0494C4A13C for ; Wed, 11 Jul 2012 15:16:43 +0200 (CEST) Received: from mail.unilogic.local (mail.unilogic.nl [10.0.11.245]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by router (Postfix) with ESMTP id AF8326E787 for ; Wed, 11 Jul 2012 15:16:38 +0200 (CEST) X-Footer: dW5pbG9naWMubmw= Received: from localhost ([127.0.0.1]) by mail.unilogic.local (using TLSv1/SSLv3 with cipher AES256-SHA (256 bits)); Wed, 11 Jul 2012 15:16:39 +0200 Subject: [PATCH] Robustify ceph-rbdnamer and adapt udev rules From: "Pascal de Bruijn | Unilogic B.V." To: ceph-devel@vger.kernel.org Cc: s.kleijkers@unilogic.nl Organization: Unilogic B.V. Date: Wed, 11 Jul 2012 15:16:39 +0200 Message-ID: <1342012599.5511.25.camel@clawhammer> Mime-Version: 1.0 X-Mailer: Evolution 2.32.2 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Below is a patch which makes the ceph-rbdnamer script more robust and fixes a problem with the rbd udev rules. On our setup we encountered a symlink which was linked to the wrong rbd: /dev/rbd/mypool/myrbd -> /dev/rbd1 While that link should have gone to /dev/rbd3 (on which a partition /dev/rbd3p1 was present). Now the old udev rule passes %n to the ceph-rbdnamer script, the problem with %n is that %n results in a value of 3 (for rbd3), but in a value of 1 (for rbd3p1), so it seems it can't be depended upon for rbdnaming. In the patch below the ceph-rbdnamer script is made more robust and it now it can be called in various ways: /usr/bin/ceph-rbdnamer /dev/rbd3 /usr/bin/ceph-rbdnamer /dev/rbd3p1 /usr/bin/ceph-rbdnamer rbd3 /usr/bin/ceph-rbdnamer rbd3p1 /usr/bin/ceph-rbdnamer 3 Even with all these different styles of calling the modified script, it should now return the same rbdname. This change "has" to be combined with calling it from udev with %k though. With that fixed, we hit the second problem. We ended up with: /dev/rbd/mypool/myrbd -> /dev/rbd3p1 So the rbdname was symlinked to the partition on the rbd instead of the rbd itself. So what probably went wrong is udev discovering the disk and running ceph-rbdnamer which resolved it to myrbd so the following symlink was created: /dev/rbd/mypool/myrbd -> /dev/rbd3 However partitions would be discovered next and ceph-rbdnamer would be run with rbd3p1 (%k) as parameter, resulting in the name myrbd too, with the previous correct symlink being overwritten with a faulty one: /dev/rbd/mypool/myrbd -> /dev/rbd3p1 The solution to the problem is in differentiating between disks and partitions in udev and handling them slightly differently. So with the patch below partitions now get their own symlinks in the following style (which is fairly consistent with other udev rules): /dev/rbd/mypool/myrbd-part1 -> /dev/rbd3p1 Please let me know any feedback you have on this patch or the approach used. Regards, Pascal de Bruijn Unilogic B.V. Signed-off-by: Pascal de Bruijn -------------------------------------------------------------------------------- --- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/src/ceph-rbdnamer b/src/ceph-rbdnamer index 49350a7..efb6804 100755 --- a/src/ceph-rbdnamer +++ b/src/ceph-rbdnamer @@ -1,8 +1,10 @@ #!/bin/sh -POOL=`cat /sys/devices/rbd/$1/pool` -IMAGE=`cat /sys/devices/rbd/$1/name` -SNAP=`cat /sys/devices/rbd/$1/current_snap` +DEV=$1 +NUM=`echo $DEV | sed 's#p.*##g' | tr -d 'a-z'` +POOL=`cat /sys/devices/rbd/$NUM/pool` +IMAGE=`cat /sys/devices/rbd/$NUM/name` +SNAP=`cat /sys/devices/rbd/$NUM/current_snap` if [ "$SNAP" = "-" ]; then echo -n "$POOL $IMAGE" else diff --git a/udev/50-rbd.rules b/udev/50-rbd.rules index 07f220f..406c31c 100644 --- a/udev/50-rbd.rules +++ b/udev/50-rbd.rules @@ -1 +1,2 @@ -KERNEL=="rbd[0-9]*", PROGRAM="/usr/bin/ceph-rbdnamer %n", SYMLINK+="rbd/%c{1}/%c{2}" +KERNEL=="rbd[0-9]*", ENV{DEVTYPE}=="disk", PROGRAM="/usr/bin/ceph-rbdnamer %k", SYMLINK+="rbd/%c{1}/%c{2}" +KERNEL=="rbd[0-9]*", ENV{DEVTYPE}=="partition", PROGRAM="/usr/bin/ceph-rbdnamer %k", SYMLINK+="rbd/%c{1}/%c{2}-part%n"