| Message ID | cdca7f92-93c0-9906-f2ed-57c3d0a97fae@jp.fujitsu.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
At Tue, 19 Sep 2017 16:41:52 +0900, Misono, Tomohiro wrote: > > "btrfs subvolume create/delete" outputs the message of "Create/Delete > subvolume ..." even when an operation fails. > Since it is confusing, let's outputs the message only when an operation succeeds. > > Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com> Current message as follows is odd as you said. ``` Create subvolume './test' ERROR: cannot create subvolume: No such file or directory ``` It's ambiguous for users to know whether creating subvolume succeeded or not. I tested this patch with injecting error on ioctl() for subvol creation/deletion. Reviewed-by: Satoru Takeuchi <satoru.takeuchi@gmail.com> Tested-by: Satoru Takeuchi <satoru.takeuchi@gmail.com> > --- > cmds-subvolume.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/cmds-subvolume.c b/cmds-subvolume.c > index 666f6e0..6d4b0fe 100644 > --- a/cmds-subvolume.c > +++ b/cmds-subvolume.c > @@ -189,7 +189,6 @@ static int cmd_subvol_create(int argc, char **argv) > if (fddst < 0) > goto out; > > - printf("Create subvolume '%s/%s'\n", dstdir, newname); > if (inherit) { > struct btrfs_ioctl_vol_args_v2 args; > > @@ -213,6 +212,7 @@ static int cmd_subvol_create(int argc, char **argv) > error("cannot create subvolume: %s", strerror(errno)); > goto out; > } > + printf("Create subvolume '%s/%s'\n", dstdir, newname); > > retval = 0; /* success */ > out: > @@ -337,9 +337,6 @@ again: > goto out; > } > > - printf("Delete subvolume (%s): '%s/%s'\n", > - commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) > - ? "commit" : "no-commit", dname, vname); > memset(&args, 0, sizeof(args)); > strncpy_null(args.name, vname); > res = ioctl(fd, BTRFS_IOC_SNAP_DESTROY, &args); > @@ -349,6 +346,9 @@ again: > ret = 1; > goto out; > } > + printf("Delete subvolume (%s): '%s/%s'\n", > + commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) > + ? "commit" : "no-commit", dname, vname); > > if (commit_mode == 1) { > res = wait_for_commit(fd); > -- > 2.9.5 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 19.09.2017 10:41, Misono, Tomohiro wrote: > "btrfs subvolume create/delete" outputs the message of "Create/Delete > subvolume ..." even when an operation fails. > Since it is confusing, let's outputs the message only when an operation succeeds. > > Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com> > --- > cmds-subvolume.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/cmds-subvolume.c b/cmds-subvolume.c > index 666f6e0..6d4b0fe 100644 > --- a/cmds-subvolume.c > +++ b/cmds-subvolume.c > @@ -189,7 +189,6 @@ static int cmd_subvol_create(int argc, char **argv) > if (fddst < 0) > goto out; > > - printf("Create subvolume '%s/%s'\n", dstdir, newname); > if (inherit) { > struct btrfs_ioctl_vol_args_v2 args; > > @@ -213,6 +212,7 @@ static int cmd_subvol_create(int argc, char **argv) > error("cannot create subvolume: %s", strerror(errno)); > goto out; > } > + printf("Create subvolume '%s/%s'\n", dstdir, newname); Please change the verb to past tense, more strongly signaling success - i.e. "Created subvolume" > > retval = 0; /* success */ > out: > @@ -337,9 +337,6 @@ again: > goto out; > } > > - printf("Delete subvolume (%s): '%s/%s'\n", > - commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) > - ? "commit" : "no-commit", dname, vname); > memset(&args, 0, sizeof(args)); > strncpy_null(args.name, vname); > res = ioctl(fd, BTRFS_IOC_SNAP_DESTROY, &args); > @@ -349,6 +346,9 @@ again: > ret = 1; > goto out; > } > + printf("Delete subvolume (%s): '%s/%s'\n", > + commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) > + ? "commit" : "no-commit", dname, vname); Ditto: "Deleted subvolume" > > if (commit_mode == 1) { > res = wait_for_commit(fd); > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 25/09/17 10:30, Nikolay Borisov wrote: > On 19.09.2017 10:41, Misono, Tomohiro wrote: >> "btrfs subvolume create/delete" outputs the message of "Create/Delete >> subvolume ..." even when an operation fails. >> Since it is confusing, let's outputs the message only when an operation succeeds. > Please change the verb to past tense, more strongly signaling success - > i.e. "Created subvolume" What about recalling some UNIX standards and returning to NOT outputting any message when operation succeeds? My scripts are full of grep -v calls after each btrfs command, and this sucks (and I don't think I'm alone in this situation). If you change the message a lot of scripts will have to be changed, at least make it worth it. -- With Best Regards, Marat Khaliili -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Sep 25, 2017 at 10:42:06AM +0300, Marat Khalili wrote: > On 25/09/17 10:30, Nikolay Borisov wrote: > >On 19.09.2017 10:41, Misono, Tomohiro wrote: > >>"btrfs subvolume create/delete" outputs the message of "Create/Delete > >>subvolume ..." even when an operation fails. > >>Since it is confusing, let's outputs the message only when an operation succeeds. > >Please change the verb to past tense, more strongly signaling success - > >i.e. "Created subvolume" > What about recalling some UNIX standards and returning to NOT > outputting any message when operation succeeds? My scripts are full > of grep -v calls after each btrfs command, and this sucks (and I > don't think I'm alone in this situation). If you change the message > a lot of scripts will have to be changed, at least make it worth it. Seconded. Make sure the return code reflects the result, and drop the printed message (or keep it if there's a --verbose flag, maybe). Hugo.
On 2017年09月25日 15:42, Marat Khalili wrote: > On 25/09/17 10:30, Nikolay Borisov wrote: >> On 19.09.2017 10:41, Misono, Tomohiro wrote: >>> "btrfs subvolume create/delete" outputs the message of "Create/Delete >>> subvolume ..." even when an operation fails. >>> Since it is confusing, let's outputs the message only when an >>> operation succeeds. >> Please change the verb to past tense, more strongly signaling success - >> i.e. "Created subvolume" > What about recalling some UNIX standards and returning to NOT outputting > any message when operation succeeds? My scripts are full of grep -v > calls after each btrfs command, and this sucks (and I don't think I'm > alone in this situation). Isn't the correct way to catch the return value instead of grepping the output? If it's some command not returning value properly, would you please report it as a bug so we can fix it. Thanks, Qu > If you change the message a lot of scripts > will have to be changed, at least make it worth it. > > -- > > With Best Regards, > Marat Khaliili > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Sep 25, 2017 at 03:46:15PM +0800, Qu Wenruo wrote: > > > On 2017年09月25日 15:42, Marat Khalili wrote: > >On 25/09/17 10:30, Nikolay Borisov wrote: > >>On 19.09.2017 10:41, Misono, Tomohiro wrote: > >>>"btrfs subvolume create/delete" outputs the message of "Create/Delete > >>>subvolume ..." even when an operation fails. > >>>Since it is confusing, let's outputs the message only when an > >>>operation succeeds. > >>Please change the verb to past tense, more strongly signaling success - > >>i.e. "Created subvolume" > >What about recalling some UNIX standards and returning to NOT > >outputting any message when operation succeeds? My scripts are > >full of grep -v calls after each btrfs command, and this sucks > >(and I don't think I'm alone in this situation). > > Isn't the correct way to catch the return value instead of grepping > the output? It is, but if, for example, you're using the command in a cron script which is expected to work, you don't want it producing output because then you get a mail every time the script runs. So you have to grep -v on the "success" output to make the successful script silent. > If it's some command not returning value properly, would you please > report it as a bug so we can fix it. It's not the return value that's problematic (although those used to be a real mess). It's the fact that a successful run of the command produces noise on stdout, which most commands don't. Hugo. > Thanks, > Qu > > >If you change the message a lot of scripts will have to be > >changed, at least make it worth it. > > > > -- > > > >With Best Regards, > >Marat Khaliili > >
On 2017年09月25日 15:52, Hugo Mills wrote: > On Mon, Sep 25, 2017 at 03:46:15PM +0800, Qu Wenruo wrote: >> >> >> On 2017年09月25日 15:42, Marat Khalili wrote: >>> On 25/09/17 10:30, Nikolay Borisov wrote: >>>> On 19.09.2017 10:41, Misono, Tomohiro wrote: >>>>> "btrfs subvolume create/delete" outputs the message of "Create/Delete >>>>> subvolume ..." even when an operation fails. >>>>> Since it is confusing, let's outputs the message only when an >>>>> operation succeeds. >>>> Please change the verb to past tense, more strongly signaling success - >>>> i.e. "Created subvolume" >>> What about recalling some UNIX standards and returning to NOT >>> outputting any message when operation succeeds? My scripts are >>> full of grep -v calls after each btrfs command, and this sucks >>> (and I don't think I'm alone in this situation). >> >> Isn't the correct way to catch the return value instead of grepping >> the output? > > It is, but if, for example, you're using the command in a cron > script which is expected to work, you don't want it producing output > because then you get a mail every time the script runs. So you have to > grep -v on the "success" output to make the successful script silent. What about redirecting stdout to /dev/null and redirecting stderr to mail if return value is not 0? As for expected-to-work case, the stdout doesn't has much meaning and return value should be good enough to judge the result. > >> If it's some command not returning value properly, would you please >> report it as a bug so we can fix it. > > It's not the return value that's problematic (although those used > to be a real mess). It's the fact that a successful run of the command > produces noise on stdout, which most commands don't. Yes, a lot of tried-and-true tools don't output anything for successful run, but also a lot of other tools do output something by default, especially for complex tools like LVM. Maybe we can introduce a global --quite option to silent some output. Thanks, Qu > > Hugo. > >> Thanks, >> Qu >> >>> If you change the message a lot of scripts will have to be >>> changed, at least make it worth it. >>> >>> -- >>> >>> With Best Regards, >>> Marat Khaliili >>> > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2017年09月25日 16:04, Qu Wenruo wrote: > > > On 2017年09月25日 15:52, Hugo Mills wrote: >> On Mon, Sep 25, 2017 at 03:46:15PM +0800, Qu Wenruo wrote: >>> >>> >>> On 2017年09月25日 15:42, Marat Khalili wrote: >>>> On 25/09/17 10:30, Nikolay Borisov wrote: >>>>> On 19.09.2017 10:41, Misono, Tomohiro wrote: >>>>>> "btrfs subvolume create/delete" outputs the message of "Create/Delete >>>>>> subvolume ..." even when an operation fails. >>>>>> Since it is confusing, let's outputs the message only when an >>>>>> operation succeeds. >>>>> Please change the verb to past tense, more strongly signaling >>>>> success - >>>>> i.e. "Created subvolume" >>>> What about recalling some UNIX standards and returning to NOT >>>> outputting any message when operation succeeds? My scripts are >>>> full of grep -v calls after each btrfs command, and this sucks >>>> (and I don't think I'm alone in this situation). >>> >>> Isn't the correct way to catch the return value instead of grepping >>> the output? >> >> It is, but if, for example, you're using the command in a cron >> script which is expected to work, you don't want it producing output >> because then you get a mail every time the script runs. So you have to >> grep -v on the "success" output to make the successful script silent. > > What about redirecting stdout to /dev/null and redirecting stderr to > mail if return value is not 0? s/if return value is not 0/if return value is 0/. The main point is, if btrfs returns 0, then nothing to worry about. (Unless there is a bug, even in that case keep an eye on stderr should be enough to catch that) Thanks, Qu > As for expected-to-work case, the stdout doesn't has much meaning and > return value should be good enough to judge the result. > >> >>> If it's some command not returning value properly, would you please >>> report it as a bug so we can fix it. >> >> It's not the return value that's problematic (although those used >> to be a real mess). It's the fact that a successful run of the command >> produces noise on stdout, which most commands don't. > > Yes, a lot of tried-and-true tools don't output anything for successful > run, but also a lot of other tools do output something by default, > especially for complex tools like LVM. > > Maybe we can introduce a global --quite option to silent some output. > > Thanks, > Qu >> >> Hugo. >>> Thanks, >>> Qu >>> >>>> If you change the message a lot of scripts will have to be >>>> changed, at least make it worth it. >>>> >>>> -- >>>> >>>> With Best Regards, >>>> Marat Khaliili >>>> >> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 25/09/17 10:52, Hugo Mills wrote: > Isn't the correct way to catch the return value instead of grepping > the output? > It is, but if, for example, you're using the command in a cron > script which is expected to work, you don't want it producing output > because then you get a mail every time the script runs. So you have to > grep -v on the "success" output to make the successful script silent. > >> If it's some command not returning value properly, would you please >> report it as a bug so we can fix it. > It's not the return value that's problematic (although those used > to be a real mess). It's the fact that a successful run of the command > produces noise on stdout, which most commands don't. Yes, exactly: cron, mail -E and just long scripts where btrfs operations are small steps here and there. (On top of this, actually catching the return value from the right command before `| grep -v` with errexit and pipefail on is so difficult that I usually end up rewriting whole mess in Python. Which would be nice result in itself if it didn't take a whole day in place of one minute for bash line.) -- With Best Regards, Marat Khalili -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 25/09/17 11:08, Qu Wenruo wrote: >> What about redirecting stdout to /dev/null and redirecting stderr to >> mail if return value is not 0? > s/if return value is not 0/if return value is 0/. > > The main point is, if btrfs returns 0, then nothing to worry about. > (Unless there is a bug, even in that case keep an eye on stderr should > be enough to catch that) Redirection to /dev/null will work. However, 1) It will always looks suspicious. grep -v with expected message is at least clear about its intent and consequences. 2) Although shorter than grep -v, it will still take space in shell scripts and force one to remember btrfs commands one has to add it after. This is already inconvenient enough to want a fix. -- With Best Regards, Marat Khalili -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Sep 25, 2017 at 04:04:03PM +0800, Qu Wenruo wrote: > > > On 2017年09月25日 15:52, Hugo Mills wrote: > >On Mon, Sep 25, 2017 at 03:46:15PM +0800, Qu Wenruo wrote: > >> > >> > >>On 2017年09月25日 15:42, Marat Khalili wrote: > >>>On 25/09/17 10:30, Nikolay Borisov wrote: > >>>>On 19.09.2017 10:41, Misono, Tomohiro wrote: > >>>>>"btrfs subvolume create/delete" outputs the message of "Create/Delete > >>>>>subvolume ..." even when an operation fails. > >>>>>Since it is confusing, let's outputs the message only when an > >>>>>operation succeeds. > >>>>Please change the verb to past tense, more strongly signaling success - > >>>>i.e. "Created subvolume" > >>>What about recalling some UNIX standards and returning to NOT > >>>outputting any message when operation succeeds? My scripts are > >>>full of grep -v calls after each btrfs command, and this sucks > >>>(and I don't think I'm alone in this situation). > >> > >>Isn't the correct way to catch the return value instead of grepping > >>the output? > > > > It is, but if, for example, you're using the command in a cron > >script which is expected to work, you don't want it producing output > >because then you get a mail every time the script runs. So you have to > >grep -v on the "success" output to make the successful script silent. > > What about redirecting stdout to /dev/null and redirecting stderr to > mail if return value is not 0? > As for expected-to-work case, the stdout doesn't has much meaning > and return value should be good enough to judge the result. > > > > >>If it's some command not returning value properly, would you please > >>report it as a bug so we can fix it. > > > > It's not the return value that's problematic (although those used > >to be a real mess). It's the fact that a successful run of the command > >produces noise on stdout, which most commands don't. > > Yes, a lot of tried-and-true tools don't output anything for > successful run, but also a lot of other tools do output something by > default, especially for complex tools like LVM. btrfs sub create and btrfs sub delete, though, aren't complex. They're about as complex as mkdir and rmdir, from a user point of view. What's more, and like mkdir/rmdir, the effects of those commands show up in the filesystem at the path given, so manual verification could be as simple as "ls -d !$" or "ls !$/..". It's really, really not necessary to have this command unconditionally print "yes, I created a directory for you" to stdout. Yes, there's ways to deal with it in shell scripts, but wouldn't life be so much better if you didn't have to? Like you don't have to filter out success reports from mkdir. > Maybe we can introduce a global --quite option to silent some output. Or drop the spurious output unless it's asked for with --verbose. Because then it makes it so much easier to compose tools together into bigger and more complex things, which is, after all, one of the fundamental things that UNIX does right. Hugo. > Thanks, > Qu > > > > Hugo. > >>Thanks, > >>Qu > >> > >>>If you change the message a lot of scripts will have to be > >>>changed, at least make it worth it. > >>> > >>> -- > >>> > >>>With Best Regards, > >>>Marat Khaliili > >>> > >
On 2017年09月25日 22:19, Hugo Mills wrote: > On Mon, Sep 25, 2017 at 04:04:03PM +0800, Qu Wenruo wrote: >> >> >> On 2017年09月25日 15:52, Hugo Mills wrote: >>> On Mon, Sep 25, 2017 at 03:46:15PM +0800, Qu Wenruo wrote: >>>> >>>> >>>> On 2017年09月25日 15:42, Marat Khalili wrote: >>>>> On 25/09/17 10:30, Nikolay Borisov wrote: >>>>>> On 19.09.2017 10:41, Misono, Tomohiro wrote: >>>>>>> "btrfs subvolume create/delete" outputs the message of "Create/Delete >>>>>>> subvolume ..." even when an operation fails. >>>>>>> Since it is confusing, let's outputs the message only when an >>>>>>> operation succeeds. >>>>>> Please change the verb to past tense, more strongly signaling success - >>>>>> i.e. "Created subvolume" >>>>> What about recalling some UNIX standards and returning to NOT >>>>> outputting any message when operation succeeds? My scripts are >>>>> full of grep -v calls after each btrfs command, and this sucks >>>>> (and I don't think I'm alone in this situation). >>>> >>>> Isn't the correct way to catch the return value instead of grepping >>>> the output? >>> >>> It is, but if, for example, you're using the command in a cron >>> script which is expected to work, you don't want it producing output >>> because then you get a mail every time the script runs. So you have to >>> grep -v on the "success" output to make the successful script silent. >> >> What about redirecting stdout to /dev/null and redirecting stderr to >> mail if return value is not 0? >> As for expected-to-work case, the stdout doesn't has much meaning >> and return value should be good enough to judge the result. >> >>> >>>> If it's some command not returning value properly, would you please >>>> report it as a bug so we can fix it. >>> >>> It's not the return value that's problematic (although those used >>> to be a real mess). It's the fact that a successful run of the command >>> produces noise on stdout, which most commands don't. >> >> Yes, a lot of tried-and-true tools don't output anything for >> successful run, but also a lot of other tools do output something by >> default, especially for complex tools like LVM. > > btrfs sub create and btrfs sub delete, though, aren't complex. > They're about as complex as mkdir and rmdir, from a user point of > view. What's more, and like mkdir/rmdir, the effects of those commands > show up in the filesystem at the path given, so manual verification > could be as simple as "ls -d !$" or "ls !$/..". It's really, really > not necessary to have this command unconditionally print "yes, I > created a directory for you" to stdout. > > Yes, there's ways to deal with it in shell scripts, but wouldn't > life be so much better if you didn't have to? Like you don't have to > filter out success reports from mkdir. > >> Maybe we can introduce a global --quite option to silent some output. > > Or drop the spurious output unless it's asked for with --verbose. > Because then it makes it so much easier to compose tools together into > bigger and more complex things, which is, after all, one of the > fundamental things that UNIX does right. I think we need at least 4 levels of message: debug, info, warn, error. And only show debug and higher for --verbose. Default to show info, and warn/error always get show (stderr). (And during the new framework, we can add support for translation) And we also need to reassign all existing message to new message level. It may be time consuming but we just need to do it since there is real user demand here. (Any in this case, anyone in the maillist can help review messages) Thanks, Qu > > Hugo. > >> Thanks, >> Qu >>> >>> Hugo. >>>> Thanks, >>>> Qu >>>> >>>>> If you change the message a lot of scripts will have to be >>>>> changed, at least make it worth it. >>>>> >>>>> -- >>>>> >>>>> With Best Regards, >>>>> Marat Khaliili >>>>> >>> > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 25/09/17 17:33, Qu Wenruo wrote:
> (Any in this case, anyone in the maillist can help review messages)
If this is a question, I can help with assigning levels to messages.
Although I think many levels are only required for complex daemons or
network tools, while btrfs utils mostly perform atomic operations which
either succeed or fail. But it's of course hard to be sure without
seeing all actual messages, probably there's some use for 4 levels.
--
With Best Regards,
Marat Khalili
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi,
I have 15 x 6 TB disks in md-raid (Because btrfs's raid6 was marked as
not-for-real-use when I first time installed this machine)
Now I have got both problem twice.
At this last time I have 233 subvolums, and millions of files (all
together)
Then filesystem went to read only with this dmesg:
[Sat Sep 23 07:25:28 2017] ------------[ cut here ]------------
[Sat Sep 23 07:25:28 2017] WARNING: CPU: 5 PID: 5431 at /build/linux-hwe-edge-CrMNv8/linux-hwe-edge-4.10.0/fs/btrfs/extent-tree.c:6947 __btrfs_free_extent.isra.61+0x2cb/0xeb0 [btrfs]
[Sat Sep 23 07:25:28 2017] BTRFS: Transaction aborted (error -28)
[Sat Sep 23 07:25:28 2017] Modules linked in: 8021q garp mrp stp llc bonding ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG xt_limit xt_tcpudp xt_multiport iptable_filter ip_tables x_tables intel_rapl sb_edac edac_core x86_pkg_temp_thermal intel_powerclamp coretemp ipmi_ssif kvm_intel kvm irqbypass intel_cstate intel_rapl_perf joydev input_leds mei_me mei lpc_ich shpchp ioatdma ipmi_si ipmi_devintf ipmi_msghandler acpi_pad acpi_power_meter mac_hid ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid0 multipath linear raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 ses enclosure ixgbe ast crct10dif_pclmul i2c_algo_bit crc32_pclmul ttm drm_kms_helper ghash_clmulni_intel
[Sat Sep 23 07:25:28 2017] dca syscopyarea pcbc sysfillrect aesni_intel aes_x86_64 hid_generic sysimgblt crypto_simd mpt3sas fb_sys_fops ptp r8169 raid_class ahci glue_helper usbhid pps_core mxm_wmi cryptd drm hid libahci mdio scsi_transport_sas mii fjes wmi
[Sat Sep 23 07:25:28 2017] CPU: 5 PID: 5431 Comm: btrfs-transacti Tainted: G W 4.10.0-26-generic #30~16.04.1-Ubuntu
[Sat Sep 23 07:25:28 2017] Hardware name: Supermicro X10DRi/X10DRI-T, BIOS 2.1 09/13/2016
[Sat Sep 23 07:25:28 2017] Call Trace:
[Sat Sep 23 07:25:28 2017] dump_stack+0x63/0x90
[Sat Sep 23 07:25:28 2017] __warn+0xcb/0xf0
[Sat Sep 23 07:25:28 2017] warn_slowpath_fmt+0x5f/0x80
[Sat Sep 23 07:25:28 2017] __btrfs_free_extent.isra.61+0x2cb/0xeb0 [btrfs]
[Sat Sep 23 07:25:28 2017] ? btrfs_merge_delayed_refs+0x64/0x640 [btrfs]
[Sat Sep 23 07:25:28 2017] ? btrfs_search_slot+0x981/0x9c0 [btrfs]
[Sat Sep 23 07:25:28 2017] __btrfs_run_delayed_refs+0xb44/0x13a0 [btrfs]
[Sat Sep 23 07:25:28 2017] ? btree_set_page_dirty+0xe/0x10 [btrfs]
[Sat Sep 23 07:25:28 2017] ? free_extent_buffer+0x4b/0xa0 [btrfs]
[Sat Sep 23 07:25:28 2017] ? igrab+0x1e/0x60
[Sat Sep 23 07:25:28 2017] btrfs_run_delayed_refs+0x7f/0x2a0 [btrfs]
[Sat Sep 23 07:25:28 2017] btrfs_write_dirty_block_groups+0x169/0x3c0 [btrfs]
[Sat Sep 23 07:25:28 2017] commit_cowonly_roots+0x221/0x2c0 [btrfs]
[Sat Sep 23 07:25:28 2017] btrfs_commit_transaction+0x542/0x950 [btrfs]
[Sat Sep 23 07:25:28 2017] transaction_kthread+0x1a6/0x1c0 [btrfs]
[Sat Sep 23 07:25:28 2017] kthread+0x109/0x140
[Sat Sep 23 07:25:28 2017] ? btrfs_cleanup_transaction+0x540/0x540 [btrfs]
[Sat Sep 23 07:25:28 2017] ? kthread_create_on_node+0x60/0x60
[Sat Sep 23 07:25:28 2017] ret_from_fork+0x2c/0x40
[Sat Sep 23 07:25:28 2017] ---[ end trace 76fd3c22e75c2b85 ]---
[Sat Sep 23 07:25:28 2017] BTRFS: error (device sdb) in __btrfs_free_extent:6947: errno=-28 No space left
[Sat Sep 23 07:25:28 2017] BTRFS: error (device sdb) in btrfs_drop_snapshot:9193: errno=-28 No space left
[Sat Sep 23 07:25:28 2017] BTRFS info (device sdb): forced readonly
After a long googling (about more complex situations) I suddenly noticed
"device sdb" WTF??? Filesystem is mounted from /dev/md3 (sdb is part of
that mdraid) so btrfs should not even know anything about that /dev/sdb.
So maybe the error is more simple that I thought. It tries to allocate
more metadata from physical drive (not from /dev/md3 as it was supposed to
do), so that "no space left on device" could be the REAL error...
But why it is doing so? Does it help if I add some extra virtualization
layer, like LVM?
# grep data /etc/fstab
/dev/md3 /data btrfs compress=zlib,space_cache,noatime 0 0
# btrfs fi df /data
Data, single: total=4.77TiB, used=4.77TiB
System, DUP: total=32.00MiB, used=592.00KiB
Metadata, DUP: total=92.00GiB, used=90.87GiB
GlobalReserve, single: total=512.00MiB, used=172.50MiB
# btrfs fi show /data
Label: none uuid: 84a31ba6-d22d-4653-b071-5525dbdfe561
Total devices 1 FS bytes used 4.85TiB
devid 2 size 70.95TiB used 4.95TiB path /dev/md3
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid10]
md3 : active raid6 sdl[17](S) sdb[16] sdq[15] sdp[14] sdm[11] sdc[1] sdo[13] sde[3] sdj[8] sdn[12] sdh[6] sdi[7] sdk[9] sdf[4] sdd[2] sdg[5]
76185088512 blocks super 1.2 level 6, 512k chunk, algorithm 2 [15/15] [UUUUUUUUUUUUUUU]
bitmap: 0/44 pages [0KB], 65536KB chunk
# btrfs --version
btrfs-progs v4.4
# uname -a
Linux backuprasia 4.10.0-26-generic #30~16.04.1-Ubuntu SMP Tue Jun 27 09:40:14 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
( This usage have been done later after some balance tries.. and mounte to another moun point)
# btrfs fi usage /data2
Overall:
Device size: 70.95TiB
Device allocated: 4.95TiB
Device unallocated: 66.01TiB
Device missing: 0.00B
Used: 4.94TiB
Free (estimated): 66.01TiB (min: 33.01TiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 512.00MiB (used: 0.00B)
Data,single: Size:4.77TiB, Used:4.76TiB
/dev/md3 4.77TiB
Metadata,DUP: Size:92.00GiB, Used:90.79GiB
/dev/md3 184.00GiB
System,DUP: Size:32.00MiB, Used:592.00KiB
/dev/md3 64.00MiB
Unallocated:
/dev/md3 66.01TiB
PS. I have noticed another bug too, but I haven't tested it with lastest
kernels after I noticed that it happens only with compression=lzo. So
maybe it is already fixed. With gzip or none compression probem does not
happens. I have email server with about 0.5 TB volume. It is using
Maildir so it contains huge amount of files. Sometimes some files goes
unreadable. After server reset problematic file could be readable again
(but not always)...
But weird thing is that unreadable file always seems to be
dovecot.index.log. I asked from dovecot author, and he said there is
nothing special with that file usage, it is just opened with O_APPEND and
some data is written to it. Problem is hard to reproduce because it seems
to be random
On 09/25/2017 06:11 PM, linux-btrfs@oh3mqu.pp.hyper.fi wrote as excerpted: > After a long googling (about more complex situations) I suddenly > noticed "device sdb" WTF??? Filesystem is mounted from /dev/md3 (sdb > is part of that mdraid) so btrfs should not even know anything about > that /dev/sdb. I would be interested in explanations regarding this too. It happened to me as well, that I was confused by /dev/sd* device paths being printed by btrfs in the logs, even though it runs on /dev/md-* (/dev/mapper/*) devices exclusively. > PS. I have noticed another bug too, but I haven't tested it with > lastest kernels after I noticed that it happens only with > compression=lzo. So maybe it is already fixed. With gzip or none > compression probem does not happens. I have email server with about > 0.5 TB volume. It is using Maildir so it contains huge amount of > files. Sometimes some files goes unreadable. After server reset > problematic file could be readable again (but not always)... > > But weird thing is that unreadable file always seems to be > dovecot.index.log. Confirm this (non-reproducible) behavior on a VPS running Debian 4.5.4-1~bpo8+1. Lukas
linux-btrfs posted on Mon, 25 Sep 2017 19:11:01 +0300 as excerpted: Three points first off: 1) Please use a name other than "linux-btrfs". That can remain the email address, but a name to go with it would be nice. (You can see how my name and email address differ, for instance.) 2) Please don't reply to existing threads with a totally different topic. Proper reply-threading clients will see it's a reply to the other thread (it's in the headers) and display it as such, even if you change the subject line. If it's a different topic, start a new thread, don't reply to an existing thread. 3) I'm not a dev, just another btrfs user and list regular. So I won't attempt to address the real technical or complicated stuff, but I can try to reply to the easy stuff at least, freeing the devs for addressing the more complicated stuff if they see a bug they can fix or might already be working on. As to the problem... > I have 15 x 6 TB disks in md-raid (Because btrfs's raid6 was marked as > not-for-real-use when I first time installed this machine) > > Now I have got both problem twice. > > At this last time I have 233 subvolums, and millions of files (all > together) Just a reminder prompted by seeing those numbers tho I'd guess you already have this covered... Sysadmin's backups rule #1: The true value of your data is defined not by any words /claiming/ value nor by what you use it for, as the machine doesn't care about that, but rather by the number of backups of it you consider it worth having. No backups means you are defining the data to be of less value than the time/trouble/resources to make the backup, so loss is never a big deal, because either you either have a backup if you considered it important to you, or you already saved what you defined as more valuable to you than that data, the time, trouble and resources you'd have otherwise put into the backup. (Similarly with the currency of those backups, only there it's the value of the data difference between your last backup and the working copy. Once the data in that difference is of more value than the time/trouble/ resources to update the backup, it'll be updated. Otherwise, the data in that delta is obviously not valuable enough to be worth that trouble, and thus not valuable enough to be terribly worried about if lost.) > Then filesystem went to read only with this dmesg: > > [Sat Sep 23 07:25:28 2017] ------------[ cut here ]------------ > [Sat Sep 23 07:25:28 2017] WARNING: CPU: 5 PID: 5431 at > /build/linux-hwe-edge-CrMNv8/linux-hwe-edge-4.10.0/fs/btrfs/extent- tree.c:6947 > __btrfs_free_extent.isra.61+0x2cb/0xeb0 [btrfs] > [Sat Sep 23 07:25:28 2017] BTRFS: Transaction aborted (error -28) > 4.10.0-26-generic #30~16.04.1-Ubuntu Note that this is the mainline btrfs development list, with btrfs still stabilizing, not yet fully stable and mature, so this list tends to be quite forward focused. We recommend and best support the latest two mainline kernels in two tracks, current and LTS. The current kernel is 4.13, so on the current track, 4.13 and 4.12 are recommended and best supported. On the LTS track, the coming 4.14 is scheduled to be an LTS, with 4.9 and 4.4 the two previous LTSs before that. So the 4.9 LTS kernel series is supported, with 4.4 currently supported but on its way out and 4.14 on the way in. And current track 4.13 and 4.12 are supported, with 4.12 on the way out. 4.10 isn't an LTS kernel, and it's old enough it's already several kernels out of current support track. So upgrading to current 4.13 or downgrading to 4.9 LTS series would be recommended. Meanwhile, your distro is in a better position to support their kernels of /whatever/ version since they know what patches they've applied and what btrfs fixes they've backported... or not. Of course we'll still try to help with 4.10, and it's not /too/ dated, but you can expect that you might get "does it still happen with a current kernel" type questions. > [Sat Sep 23 07:25:28 2017] BTRFS: error (device sdb) in > __btrfs_free_extent:6947: errno=-28 No space left [Sat Sep 23 07:25:28 > 2017] BTRFS: error (device sdb) in btrfs_drop_snapshot:9193: errno=-28 > No space left [Sat Sep 23 07:25:28 2017] BTRFS info (device sdb): forced > readonly > > > After a long googling (about more complex situations) I suddenly noticed > "device sdb" WTF??? Filesystem is mounted from /dev/md3 (sdb is part of > that mdraid) so btrfs should not even know anything about that /dev/sdb. > > So maybe the error is more simple that I thought. It tries to allocate > more metadata from physical drive (not from /dev/md3 as it was supposed > to do), so that "no space left on device" could be the REAL error... > > But why it is doing so? Does it help if I add some extra virtualization > layer, like LVM? Keep in mind that btrfs, unlike most other filesystems, can be multi- device. As such, it needs a way to track which devices are part of each filesystem, and it uses the filesystem UUID for that purpose. Meanwhile, btrfs device scan, which is auto-run by udev after a device appears, is what lets the kernel know about all those btrfs containing devices and the UUIDs associated with them. That's why btrfs is listing one of the md components as part of the filesystem -- it obviously has the same btrfs UUID as the md device that you actually created the filesystem on. That can cause btrfs to write to the wrong device in some instances, tho obviously it doesn't do it all the time or things wouldn't work as well as they do. It's thus recommended that when you're using btrfs on device layering such as mdraid or LVM, that you ensure that the lower layer devices aren't exposed to btrfs so it doesn't get confused. I believe LVM can be configured to hide the lower layer devices in at least some instances, but I'm not sure about mdraid, altho this is the first time I've seen that particular issue with it (tho it may simply be because I've not been watching for it, Chris Murphy or Hugo are likely to have more information about that as they're more active with user support than I am, and are more technically skilled too, I believe). > # btrfs --version btrfs-progs v4.4 As with the kernel version, likewise with the btrfs-progs userspace, tho in operation it's less important than the kernel, because for most normal operating commands, userspace simply calls the real code in the kernel anyway. But once things begin to go wrong, the userspace version becomes more important, because it's the userspace code that handles btrfs check, btrfs restore, etc. So while userspace 4.4 is fine for normal operations, you might want to be sure you have a current 4.12 or so available for recovery if needed, since it'll have the latest fixes and thus should give you the best chance at recovery, if it /is/ needed. > # btrfs fi usage /data2 > Overall: > Device size: 70.95TiB > Device allocated: 4.95TiB > Device unallocated: 66.01TiB > Device missing: 0.00B > Used: 4.94TiB > Free (estimated): 66.01TiB (min: 33.01TiB) > Data ratio: 1.00 > Metadata ratio: 2.00 > Global reserve: 512.00MiB (used: 0.00B) > > Data,single: Size:4.77TiB, Used:4.76TiB > /dev/md3 4.77TiB > > Metadata,DUP: Size:92.00GiB, Used:90.79GiB > /dev/md3 184.00GiB > > System,DUP: Size:32.00MiB, Used:592.00KiB > /dev/md3 64.00MiB > > Unallocated: > /dev/md3 66.01TiB This usage looks healthy. No problems here. =:^) That's the easy to see and address stuff, anyway.
On Tue, 26 Sep 2017, Duncan wrote: > consider it worth having. No backups means you are defining the data to > be of less value than the time/trouble/resources to make the backup, so > loss is never a big deal, because either you either have a backup if you I know importance of backups, and this problematic machine are backups (I still have originals if this crashes) > That's why btrfs is listing one of the md components as part of the > filesystem -- it obviously has the same btrfs UUID as the md device that > you actually created the filesystem on. # for I in c d e f g h i j k l n o p q r ; do blkid /dev/sd$I ; done ; blkid /dev/md3 /dev/sdc: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="cffdd56f-f8d6-5191-bd7b-d528018e1106" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdd: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="5d586780-d22f-a68b-769a-5c1bfe9491fb" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sde: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="5777fb6a-7156-3a70-5384-789ac8dd5656" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdf: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="f017f1c7-df26-9279-ccb9-002b8ce1edd9" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdg: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="0de626d7-f754-e53b-792b-3217f6cb63d2" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdh: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="036e8679-2c6f-4a48-601e-1379cdd0a3eb" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdi: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="f9cb6c12-dc73-92b9-363c-fafb5b0efc81" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdj: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="7f3f09d8-d09d-c263-b292-a9d5d004c7ce" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdk: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="07bde9df-8f5b-518d-78fa-3cff3a950eb4" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdl: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="1895c727-e8b2-463a-4f67-e260ee6e98ff" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdn: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="72a6d25c-a7c0-3329-5f15-8abb7c3edbdb" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdo: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="3580f74a-1605-3e5e-9396-9e929eaeb6fc" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdp: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="006883cb-76f6-2967-1a5a-225d258d72b0" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdq: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="b72b2bc3-d983-ed27-f64a-fea9b27e42bf" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/sdr: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" UUID_SUB="bd7b687d-2882-3709-abe4-03f1a1010a67" LABEL="dyn-50:3" TYPE="linux_raid_member" /dev/md3: UUID="84a31ba6-d22d-4653-b071-5525dbdfe561" UUID_SUB="36cf435b-e589-45dd-99a5-5efdb1439913" TYPE="btrfs" Those sd* devices are member of md, and seems they aren't equal with that md-device
26.09.2017 10:31, Lukas Pirl пишет: > On 09/25/2017 06:11 PM, linux-btrfs@oh3mqu.pp.hyper.fi wrote as excerpted: >> After a long googling (about more complex situations) I suddenly >> noticed "device sdb" WTF??? Filesystem is mounted from /dev/md3 (sdb >> is part of that mdraid) so btrfs should not even know anything about >> that /dev/sdb. > > I would be interested in explanations regarding this too. It happened > to me as well, that I was confused by /dev/sd* device paths being > printed by btrfs in the logs, even though it runs on /dev/md-* > (/dev/mapper/*) devices exclusively. > Could be related: https://forums.opensuse.org/showthread.php/526696-Upgrade-destroys-dmraid?p=2838207#post2838207
diff --git a/cmds-subvolume.c b/cmds-subvolume.c index 666f6e0..6d4b0fe 100644 --- a/cmds-subvolume.c +++ b/cmds-subvolume.c @@ -189,7 +189,6 @@ static int cmd_subvol_create(int argc, char **argv) if (fddst < 0) goto out; - printf("Create subvolume '%s/%s'\n", dstdir, newname); if (inherit) { struct btrfs_ioctl_vol_args_v2 args; @@ -213,6 +212,7 @@ static int cmd_subvol_create(int argc, char **argv) error("cannot create subvolume: %s", strerror(errno)); goto out; } + printf("Create subvolume '%s/%s'\n", dstdir, newname); retval = 0; /* success */ out: @@ -337,9 +337,6 @@ again: goto out; } - printf("Delete subvolume (%s): '%s/%s'\n", - commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) - ? "commit" : "no-commit", dname, vname); memset(&args, 0, sizeof(args)); strncpy_null(args.name, vname); res = ioctl(fd, BTRFS_IOC_SNAP_DESTROY, &args); @@ -349,6 +346,9 @@ again: ret = 1; goto out; } + printf("Delete subvolume (%s): '%s/%s'\n", + commit_mode == 2 || (commit_mode == 1 && cnt + 1 == argc) + ? "commit" : "no-commit", dname, vname); if (commit_mode == 1) { res = wait_for_commit(fd);
"btrfs subvolume create/delete" outputs the message of "Create/Delete subvolume ..." even when an operation fails. Since it is confusing, let's outputs the message only when an operation succeeds. Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com> --- cmds-subvolume.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)