| Message ID | 20170315221223.GA116381@samitolvanen.mtv.corp.google.com (mailing list archive) |
|---|---|
| State | Accepted, archived |
| Delegated to: | Mike Snitzer |
| Headers | show |
On 03/15/2017 11:12 PM, Sami Tolvanen wrote: > On Wed, Mar 15, 2017 at 09:22:02PM +0100, michal virgovic wrote: >> This oops keeps appearing. > > This can happen if dm-verity is set up with an invalid root hash, for > example. Please test the attached patch, which limits recursion and should > allow it to fail more gracefully. > > Sami Thanks Sami! I see these patches are already in Android, Why it is not upstream? https://android.googlesource.com/kernel/goldfish/+/21c0fe9f24b7707d2b49401f8c740c3e35c580ea%5E%21/ We tried to finally implement FEC support in veritysetup and it would be nice that upstream kernel contains all patches so we do not need to reinvent the wheel... Thanks, Milan > > --- > > dm verity fec: limit error correction recursion > > If the hash tree itself is sufficiently corrupt in addition to data blocks, > it's possible for error correction to end up in a deep recursive loop, > which eventually causes a kernel panic. This change limits the recursion to > a reasonable level during a single I/O operation. > > Fixes: a739ff3f543a ("dm verity: add support for forward error correction") > Signed-off-by: Sami Tolvanen <samitolvanen@google.com> > --- > drivers/md/dm-verity-fec.c | 12 +++++++++++- > drivers/md/dm-verity-fec.h | 4 ++++ > 2 files changed, 15 insertions(+), 1 deletion(-) > > diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c > index 0f0eb8a3d922..c3cc04d89524 100644 > --- a/drivers/md/dm-verity-fec.c > +++ b/drivers/md/dm-verity-fec.c > @@ -439,6 +439,13 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io, > if (!verity_fec_is_enabled(v)) > return -EOPNOTSUPP; > > + if (fio->level >= DM_VERITY_FEC_MAX_RECURSION) { > + DMWARN_LIMIT("%s: FEC: recursion too deep", v->data_dev->name); > + return -EIO; > + } > + > + fio->level++; > + > if (type == DM_VERITY_BLOCK_TYPE_METADATA) > block += v->data_blocks; > > @@ -470,7 +477,7 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io, > if (r < 0) { > r = fec_decode_rsb(v, io, fio, rsb, offset, true); > if (r < 0) > - return r; > + goto done; > } > > if (dest) > @@ -480,6 +487,8 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io, > r = verity_for_bv_block(v, io, iter, fec_bv_copy); > } > > +done: > + fio->level--; > return r; > } > > @@ -520,6 +529,7 @@ void verity_fec_init_io(struct dm_verity_io *io) > memset(fio->bufs, 0, sizeof(fio->bufs)); > fio->nbufs = 0; > fio->output = NULL; > + fio->level = 0; > } > > /* > diff --git a/drivers/md/dm-verity-fec.h b/drivers/md/dm-verity-fec.h > index 7fa0298b995e..bb31ce87a933 100644 > --- a/drivers/md/dm-verity-fec.h > +++ b/drivers/md/dm-verity-fec.h > @@ -27,6 +27,9 @@ > #define DM_VERITY_FEC_BUF_MAX \ > (1 << (PAGE_SHIFT - DM_VERITY_FEC_BUF_RS_BITS)) > > +/* maximum recursion level for verity_fec_decode */ > +#define DM_VERITY_FEC_MAX_RECURSION 4 > + > #define DM_VERITY_OPT_FEC_DEV "use_fec_from_device" > #define DM_VERITY_OPT_FEC_BLOCKS "fec_blocks" > #define DM_VERITY_OPT_FEC_START "fec_start" > @@ -58,6 +61,7 @@ struct dm_verity_fec_io { > unsigned nbufs; /* number of buffers allocated */ > u8 *output; /* buffer for corrected output */ > size_t output_pos; > + unsigned level; /* recursion level */ > }; > > #ifdef CONFIG_DM_VERITY_FEC > -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On 03/15/2017 11:12 PM, Sami Tolvanen wrote: > On Wed, Mar 15, 2017 at 09:22:02PM +0100, michal virgovic wrote: >> This oops keeps appearing. > > This can happen if dm-verity is set up with an invalid root hash, for > example. Please test the attached patch, which limits recursion and should > allow it to fail more gracefully. I tried it and it still does take at least two minutes and tons of logs until even udev scan finishes on a device with incorrect FEC attributes. And once it is finished, dm-verity crashes on dm-device removal... Could we please have all crucial fixes upstream? It seems that FEC is really not usable for now. Milan This is what I run (32bit VM, Linus' git + your fix), dmesg below. DEV_NAME=tst12414 DEV_DATA=/dev/sdb DEV_HASH=/dev/sdc DEV_FEC=/dev/sdd dmsetup -r create $DEV_NAME --table "0 8 verity 1 $DEV_DATA $DEV_HASH 4096 4096 1 1 sha256 0000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000 8 use_fec_from_device $DEV_FEC fec_roots 16 fec_blocks 1 fec_start 1" udevadm settle dd if=/dev/mapper/$DEV_NAME of=/dev/null bs=4k count=1 iflag=direct dmsetup remove $DEV_NAME ... Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : fec_decode_bufs: 320 callbacks suppressed Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : buffer_io_error: 66 callbacks suppressed Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:05 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:05 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:05 : device-mapper: verity: 8:16: data block 0 is corrupted Mar 17 12:35:05 : device-mapper: verity: 8:16: reached maximum errors Mar 17 12:35:09 : fec_decode_bufs: 328 callbacks suppressed Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:09 : buffer_io_error: 67 callbacks suppressed Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : fec_decode_bufs: 332 callbacks suppressed Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:14 : buffer_io_error: 68 callbacks suppressed Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : fec_decode_bufs: 332 callbacks suppressed Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:19 : buffer_io_error: 67 callbacks suppressed Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : fec_decode_bufs: 320 callbacks suppressed Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:24 : buffer_io_error: 65 callbacks suppressed Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : fec_decode_bufs: 324 callbacks suppressed Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:29 : buffer_io_error: 68 callbacks suppressed Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:30 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : fec_decode_bufs: 330 callbacks suppressed Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:34 : buffer_io_error: 68 callbacks suppressed Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:35 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:35 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:39 : fec_decode_bufs: 330 callbacks suppressed Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:39 : buffer_io_error: 67 callbacks suppressed Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:40 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:40 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:40 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : fec_decode_bufs: 330 callbacks suppressed Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:44 : buffer_io_error: 65 callbacks suppressed Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:45 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:45 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:49 : fec_decode_bufs: 320 callbacks suppressed Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:49 : buffer_io_error: 67 callbacks suppressed Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:50 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:50 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:50 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:54 : fec_decode_bufs: 326 callbacks suppressed Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:54 : buffer_io_error: 66 callbacks suppressed Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:59 : fec_decode_bufs: 328 callbacks suppressed Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:35:59 : buffer_io_error: 66 callbacks suppressed Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:00 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:00 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:00 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:04 : fec_decode_bufs: 326 callbacks suppressed Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures) Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures) Mar 17 12:36:04 : buffer_io_error: 69 callbacks suppressed Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read Mar 17 12:36:05 : ------------[ cut here ]------------ Mar 17 12:36:05 : WARNING: CPU: 0 PID: 1895 at drivers/md/dm-bufio.c:1505 dm_bufio_client_destroy+0x204/0x210 [dm_bufio] Mar 17 12:36:05 : Modules linked in: dm_verity reed_solomon dm_bufio loop rpcsec_gss_krb5 dm_mod crc32_pclmul crc32c_intel pcbc aesni_intel aes_i586 crypto_simd cryptd ata_piix Mar 17 12:36:05 : CPU: 0 PID: 1895 Comm: dmsetup Tainted: G W 4.11.0-rc2+ #163 Mar 17 12:36:05 : Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015 Mar 17 12:36:05 : Call Trace: Mar 17 12:36:05 : dump_stack+0x60/0x94 Mar 17 12:36:05 : __warn+0xc1/0xe0 Mar 17 12:36:05 : warn_slowpath_null+0x11/0x20 Mar 17 12:36:05 : dm_bufio_client_destroy+0x204/0x210 [dm_bufio] Mar 17 12:36:05 : verity_fec_dtr+0x4f/0x80 [dm_verity] Mar 17 12:36:05 : verity_dtr+0x73/0x80 [dm_verity] Mar 17 12:36:05 : dm_table_destroy+0x51/0xf0 [dm_mod] Mar 17 12:36:05 : __dm_destroy+0x132/0x220 [dm_mod] Mar 17 12:36:05 : dm_destroy+0xd/0x10 [dm_mod] Mar 17 12:36:05 : dev_remove+0xf9/0x150 [dm_mod] Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod] Mar 17 12:36:05 : ctl_ioctl+0x1f3/0x4c0 [dm_mod] Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod] Mar 17 12:36:05 : ? ctl_ioctl+0x4c0/0x4c0 [dm_mod] Mar 17 12:36:05 : dm_ctl_ioctl+0xc/0x10 [dm_mod] Mar 17 12:36:05 : do_vfs_ioctl+0x7d/0x770 Mar 17 12:36:05 : ? mntput_no_expire+0x6e/0x380 Mar 17 12:36:05 : ? SyS_semctl+0x230/0x4b0 Mar 17 12:36:05 : SyS_ioctl+0x28/0x50 Mar 17 12:36:05 : do_int80_syscall_32+0x3e/0x110 Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100 Mar 17 12:36:05 : entry_INT80_32+0x2f/0x2f Mar 17 12:36:05 : EIP: 0xb770c9f2 Mar 17 12:36:05 : EFLAGS: 00000286 CPU: 0 Mar 17 12:36:05 : EAX: ffffffda EBX: 00000003 ECX: c138fd04 EDX: 800cc910 Mar 17 12:36:05 : ESI: 00000000 EDI: b7701500 EBP: b76f5c0f ESP: bfcb9a34 Mar 17 12:36:05 : DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100 Mar 17 12:36:05 : ---[ end trace f97eb78ad904f683 ]--- Mar 17 12:36:05 : device-mapper: bufio: leaked buffer 10, hold count 2224, list 0 Mar 17 12:36:05 : ------------[ cut here ]------------ Mar 17 12:36:05 : kernel BUG at drivers/md/dm-bufio.c:1521! Mar 17 12:36:05 : invalid opcode: 0000 [#1] PREEMPT SMP Mar 17 12:36:05 : Modules linked in: dm_verity reed_solomon dm_bufio loop rpcsec_gss_krb5 dm_mod crc32_pclmul crc32c_intel pcbc aesni_intel aes_i586 crypto_simd cryptd ata_piix Mar 17 12:36:05 : CPU: 0 PID: 1895 Comm: dmsetup Tainted: G W 4.11.0-rc2+ #163 Mar 17 12:36:05 : Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015 Mar 17 12:36:05 : task: f5e201c0 task.stack: f3a34000 Mar 17 12:36:05 : EIP: dm_bufio_client_destroy+0x170/0x210 [dm_bufio] Mar 17 12:36:05 : EFLAGS: 00010297 CPU: 0 Mar 17 12:36:05 : EAX: f5ed0c0c EBX: f5ec0e00 ECX: f5ec0e5c EDX: 00000001 Mar 17 12:36:05 : ESI: f5ec0e64 EDI: 00000002 EBP: f3a35d28 ESP: f3a35d14 Mar 17 12:36:05 : DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068 Mar 17 12:36:05 : CR0: 80050033 CR2: b74ff4d0 CR3: 346c3000 CR4: 001406d0 Mar 17 12:36:05 : Call Trace: Mar 17 12:36:05 : verity_fec_dtr+0x4f/0x80 [dm_verity] Mar 17 12:36:05 : verity_dtr+0x73/0x80 [dm_verity] Mar 17 12:36:05 : dm_table_destroy+0x51/0xf0 [dm_mod] Mar 17 12:36:05 : __dm_destroy+0x132/0x220 [dm_mod] Mar 17 12:36:05 : dm_destroy+0xd/0x10 [dm_mod] Mar 17 12:36:05 : dev_remove+0xf9/0x150 [dm_mod] Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod] Mar 17 12:36:05 : ctl_ioctl+0x1f3/0x4c0 [dm_mod] Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod] Mar 17 12:36:05 : ? ctl_ioctl+0x4c0/0x4c0 [dm_mod] Mar 17 12:36:05 : dm_ctl_ioctl+0xc/0x10 [dm_mod] Mar 17 12:36:05 : do_vfs_ioctl+0x7d/0x770 Mar 17 12:36:05 : ? mntput_no_expire+0x6e/0x380 Mar 17 12:36:05 : ? SyS_semctl+0x230/0x4b0 Mar 17 12:36:05 : SyS_ioctl+0x28/0x50 Mar 17 12:36:05 : do_int80_syscall_32+0x3e/0x110 Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100 Mar 17 12:36:05 : entry_INT80_32+0x2f/0x2f Mar 17 12:36:05 : EIP: 0xb770c9f2 Mar 17 12:36:05 : EFLAGS: 00000286 CPU: 0 Mar 17 12:36:05 : EAX: ffffffda EBX: 00000003 ECX: c138fd04 EDX: 800cc910 Mar 17 12:36:05 : ESI: 00000000 EDI: b7701500 EBP: b76f5c0f ESP: bfcb9a34 Mar 17 12:36:05 : DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100 Mar 17 12:36:05 : Code: 0f 85 84 00 00 00 8b 43 68 85 c0 75 5e 8b 83 84 00 00 00 e8 43 18 02 00 89 d8 e8 bc 7e 0e c9 8d 65 f4 5b 5e 5f 5d c3 8d 74 26 00 <0f> 0b 0f 0b 0f 0b 0f 0b 84 c0 74 76 83 e9 0c 57 ff 71 28 89 4d Mar 17 12:36:05 : EIP: dm_bufio_client_destroy+0x170/0x210 [dm_bufio] SS:ESP: 0068:f3a35d14 Mar 17 12:36:05 : ---[ end trace f97eb78ad904f684 ]--- -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
2017-03-17 12:45 GMT+01:00 Milan Broz <mbroz@redhat.com>: > I tried it and it still does take at least two minutes and tons of logs until even > udev scan finishes on a device with incorrect FEC attributes. > > And once it is finished, dm-verity crashes on dm-device removal... I tried it too, but with correctly computed root-hash, salt, hash-device, fec-device (using veritysetup) and the outcome is same. Attachment file is the script i used. -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Fri, Mar 17, 2017 at 12:45:00PM +0100, Milan Broz wrote: > it still does take at least two minutes and tons of logs until even > udev scan finishes on a device with incorrect FEC attributes. If dm-verity is set up with invalid arguments so that the entire partition is corrupt, it does take a while to determine this with FEC as it attempts to correct both metadata and data blocks before giving up. It also depends on the random read performance of the underlying hardware obviously. > And once it is finished, dm-verity crashes on dm-device removal... That's interesting, I haven't seen this before. Looks like something goes wrong at the very end of dm_bufio_client_destroy. I'm not familiar enough with dm_bufio internals to immediately tell why though. > Could we please have all crucial fixes upstream? Sorry, there are no other pending fixes remaining in our tree. Sami -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On Fri, Mar 17, 2017 at 01:27:25PM +0100, michal virgovic wrote: > I tried it too, but with correctly computed root-hash, salt, > hash-device, fec-device (using veritysetup) and the outcome is same. Do you mean the panic when destroying the dm device or the issue you saw initially? If the hashes are correct, verity-fec should never use dm_bufio, except to create and destroy the client. Sami -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
On 03/17/2017 04:35 PM, Sami Tolvanen wrote: > On Fri, Mar 17, 2017 at 12:45:00PM +0100, Milan Broz wrote: >> it still does take at least two minutes and tons of logs until even >> udev scan finishes on a device with incorrect FEC attributes. > > If dm-verity is set up with invalid arguments so that the entire partition > is corrupt, it does take a while to determine this with FEC as it attempts > to correct both metadata and data blocks before giving up. It also depends > on the random read performance of the underlying hardware obviously. Sure, no problem with that. Just this testing device was the smallest possible size (4k) so almost 2 minutes for FEC repairs was quite unexpected :) >> And once it is finished, dm-verity crashes on dm-device removal... > > That's interesting, I haven't seen this before. Looks like something goes > wrong at the very end of dm_bufio_client_destroy. I'm not familiar enough > with dm_bufio internals to immediately tell why though. > >> Could we please have all crucial fixes upstream? > > Sorry, there are no other pending fixes remaining in our tree. Ok, so it is a new bug then. Sorry for being grumpy, I was just trying to setup trivial verity device to test activation code in veritysetup (this part was not in initial commit at all) and every attempt to test table format ended in kernel OOPs. We should fix all this problems because it smells like a DoS problem (someone modifies on-disk FEC data to intentionally crash kernel). Anyway, Michal (our student intern) should help with testing (and veritysetup code as well), so if you have any patches, please send it to the dm-devel list and cc him. Thanks! Milan -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c index 0f0eb8a3d922..c3cc04d89524 100644 --- a/drivers/md/dm-verity-fec.c +++ b/drivers/md/dm-verity-fec.c @@ -439,6 +439,13 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io, if (!verity_fec_is_enabled(v)) return -EOPNOTSUPP; + if (fio->level >= DM_VERITY_FEC_MAX_RECURSION) { + DMWARN_LIMIT("%s: FEC: recursion too deep", v->data_dev->name); + return -EIO; + } + + fio->level++; + if (type == DM_VERITY_BLOCK_TYPE_METADATA) block += v->data_blocks; @@ -470,7 +477,7 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io, if (r < 0) { r = fec_decode_rsb(v, io, fio, rsb, offset, true); if (r < 0) - return r; + goto done; } if (dest) @@ -480,6 +487,8 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io, r = verity_for_bv_block(v, io, iter, fec_bv_copy); } +done: + fio->level--; return r; } @@ -520,6 +529,7 @@ void verity_fec_init_io(struct dm_verity_io *io) memset(fio->bufs, 0, sizeof(fio->bufs)); fio->nbufs = 0; fio->output = NULL; + fio->level = 0; } /* diff --git a/drivers/md/dm-verity-fec.h b/drivers/md/dm-verity-fec.h index 7fa0298b995e..bb31ce87a933 100644 --- a/drivers/md/dm-verity-fec.h +++ b/drivers/md/dm-verity-fec.h @@ -27,6 +27,9 @@ #define DM_VERITY_FEC_BUF_MAX \ (1 << (PAGE_SHIFT - DM_VERITY_FEC_BUF_RS_BITS)) +/* maximum recursion level for verity_fec_decode */ +#define DM_VERITY_FEC_MAX_RECURSION 4 + #define DM_VERITY_OPT_FEC_DEV "use_fec_from_device" #define DM_VERITY_OPT_FEC_BLOCKS "fec_blocks" #define DM_VERITY_OPT_FEC_START "fec_start" @@ -58,6 +61,7 @@ struct dm_verity_fec_io { unsigned nbufs; /* number of buffers allocated */ u8 *output; /* buffer for corrected output */ size_t output_pos; + unsigned level; /* recursion level */ }; #ifdef CONFIG_DM_VERITY_FEC