| Message ID | 20190625071624.27039-1-msuchanek@suse.de (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Herbert Xu |
| Headers | show |
| Series | crypto: algapi - guard against uninitialized spawn list in crypto_remove_spawns | expand |
Hi Michal, On Tue, Jun 25, 2019 at 09:16:24AM +0200, Michal Suchanek wrote: > Reportedly on Linux 4.12 the LTP testsuite crashes at pcrypt_aead01 infrequently. > > To get it reproduce more frequently I tried > > n=0 ; while true ; do /opt/ltp/testcases/bin/pcrypt_aead01 >& /dev/null ; n=$(expr $n + 1) ; echo -ne $n\\r ; done > > but this is quite stable. However, holding ^C in the terminal where the loop is running tends to trigger the crash. > > The backtrace is: > > [ 100.615804] Unable to handle kernel paging request for data at address 0x00000000 > [ 100.615876] Faulting instruction address: 0xc000000000520e7c > [ 100.615943] Oops: Kernel access of bad area, sig: 11 [#1] > [ 100.616001] SMP NR_CPUS=2048 > [ 100.616002] NUMA > [ 100.616030] pSeries > [ 100.616054] Modules linked in: authenc pcrypt crypto_user kvm_pr kvm ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink ip_tables x_tables af_packet rtc_generic vmx_crypto ibmveth(X) gf128mul btrfs xor raid6_pq sd_mod ibmvscsi(X) scsi_transport_srp crc32c_vpmsum sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod autofs4 > [ 100.616478] Supported: Yes, External > [ 100.616509] CPU: 5 PID: 6270 Comm: pcrypt_aead01 Tainted: G 4.12.14-150.22-default #1 SLE15 > [ 100.616632] task: c000000595084d80 task.stack: c0000005be6dc000 > [ 100.616708] NIP: c000000000520e7c LR: c000000000521e3c CTR: c000000000521de0 > [ 100.616801] REGS: c0000005be6df620 TRAP: 0300 Tainted: G (4.12.14-150.22-default) > [ 100.616906] MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> > [ 100.616912] CR: 24002844 XER: 20040000 > [ 100.617003] CFAR: c000000000008860 DAR: 0000000000000000 DSISR: 40000000 SOFTE: 1 > GPR00: c0000005a331f810 c0000005be6df8a0 c00000000119aa00 c0000005a331f800 > GPR04: c0000005be6df930 c0000005be6df8c0 c0000005be6df8d0 0000000000000000 > GPR08: 7269632929290000 c0000005a331f800 0000000000000000 0000000000000000 > GPR12: c000000000521de0 c000000007a33700 00000001271a0ee0 00007fffcb9e7bb8 > GPR16: 00000001271c2d80 00000001271c2d88 00007fffcb9e7a50 00007fffcb9e7a44 > GPR20: 00007fffcb9e7a98 00007fffcb9e7a60 0000000000000010 0000000000000010 > GPR24: 0000000000000000 0000000000000000 fffffffffffff000 c0000005be6dfaf0 > GPR28: c0000005b9929d00 0000000000000c93 c0000005be6df930 c0000005be6df8e0 > [ 100.617774] NIP [c000000000520e7c] crypto_remove_spawns+0x6c/0x2e0 > [ 100.617816] LR [c000000000521e3c] crypto_unregister_instance+0x5c/0xa0 > [ 100.617881] Call Trace: > [ 100.617903] [c0000005be6df8a0] [c0000005b9929d00] 0xc0000005b9929d00 (unreliable) > [ 100.617971] [c0000005be6df910] [0000000000000000] (null) > [ 100.618021] [c0000005be6df960] [d0000000098d0894] crypto_del_alg+0xdc/0x110 [crypto_user] > [ 100.618119] [c0000005be6df990] [d0000000098d0b58] crypto_user_rcv_msg+0xe0/0x260 [crypto_user] > [ 100.618222] [c0000005be6dfa30] [c00000000086d678] netlink_rcv_skb+0x78/0x170 > [ 100.618309] [c0000005be6dfaa0] [d0000000098d0064] crypto_netlink_rcv+0x4c/0x80 [crypto_user] > [ 100.618407] [c0000005be6dfad0] [c00000000086cb98] netlink_unicast+0x208/0x2f0 > [ 100.618488] [c0000005be6dfb40] [c00000000086d170] netlink_sendmsg+0x380/0x440 > [ 100.618582] [c0000005be6dfbd0] [c0000000007e9ba4] sock_sendmsg+0x64/0x90 > [ 100.618650] [c0000005be6dfc00] [c0000000007eb94c] ___sys_sendmsg+0x2cc/0x330 > [ 100.618710] [c0000005be6dfd90] [c0000000007ed02c] __sys_sendmsg+0x5c/0xc0 > [ 100.618766] [c0000005be6dfe30] [c00000000000b188] system_call+0x3c/0x130 > [ 100.618822] Instruction dump: > [ 100.618839] e9430010 83a90020 38a10020 fbe10040 fbe10048 f8c10030 f8c10038 f8a10020 > [ 100.618902] f8a10028 38030010 7fa05040 7d475378 <e90a0000> 419e0064 60000000 60000000 > [ 100.618980] ---[ end trace 60475621348ca387 ]--- > > The code looks like this: > > 0xc000000000520e10 <+0>: c8 00 4c 3c addis r2,r12,200 > 0xc000000000520e14 <+4>: f0 9b 42 38 addi r2,r2,-25616 > 0xc000000000520e18 <+8>: a6 02 08 7c mflr r0 > 0xc000000000520e1c <+12>: 00 00 00 60 nop > 0xc000000000520e20 <+16>: 79 2b ab 7c mr. r11,r5 > 0xc000000000520e24 <+20>: f0 ff c1 fb std r30,-16(r1) > 0xc000000000520e28 <+24>: e8 ff a1 fb std r29,-24(r1) > 0xc000000000520e2c <+28>: f8 ff e1 fb std r31,-8(r1) > 0xc000000000520e30 <+32>: 91 ff 21 f8 stdu r1,-112(r1) > 0xc000000000520e34 <+36>: 78 1b 69 7c mr r9,r3 > 0xc000000000520e38 <+40>: 78 23 9e 7c mr r30,r4 > 0xc000000000520e3c <+44>: 08 00 82 41 beq 0xc000000000520e44 <crypto_remove_spawns+52> > 0xc000000000520e40 <+48>: 78 5b 69 7d mr r9,r11 > 0xc000000000520e44 <+52>: 40 00 e1 3b addi r31,r1,64 > 0xc000000000520e48 <+56>: 30 00 c1 38 addi r6,r1,48 > # 0xc000000000520e4c <+60>: 10 00 43 e9 ld r10,16(r3) > 0xc000000000520e50 <+64>: 20 00 a9 83 lwz r29,32(r9) > 0xc000000000520e54 <+68>: 20 00 a1 38 addi r5,r1,32 > 0xc000000000520e58 <+72>: 40 00 e1 fb std r31,64(r1) > 0xc000000000520e5c <+76>: 48 00 e1 fb std r31,72(r1) > 0xc000000000520e60 <+80>: 30 00 c1 f8 std r6,48(r1) > 0xc000000000520e64 <+84>: 38 00 c1 f8 std r6,56(r1) > 0xc000000000520e68 <+88>: 20 00 a1 f8 std r5,32(r1) > 0xc000000000520e6c <+92>: 28 00 a1 f8 std r5,40(r1) > 0xc000000000520e70 <+96>: 10 00 03 38 addi r0,r3,16 > & 0xc000000000520e74 <+100>: 40 50 a0 7f cmpld cr7,r0,r10 > 0xc000000000520e78 <+104>: 78 53 47 7d mr r7,r10 > * 0xc000000000520e7c <+108>: 00 00 0a e9 ld r8,0(r10) > 0xc000000000520e80 <+112>: 64 00 9e 41 beq cr7,0xc000000000520ee4 <crypto_remove_spawns+212> > > #) This looks like alg->cra_users.next is loaded to r10 > &) This looks like r10 is compared with &alg->cra_users calculated on the line > above to terminate the loop > *) This looks like *alg->cra_users.next loaded into r8 which causes the null > pointer dereference > > So the fixup needs to be applied to the first dereference of > alg->cra_users.next as well to prevent crash. > > Fixes: 9a00674213a3 ("crypto: algapi - fix NULL dereference in crypto_remove_spawns()") > > Reported-by: chetjain@in.ibm.com > Signed-off-by: Michal Suchanek <msuchanek@suse.de> > --- > I cannot really test if this fix is effective because the crash is some > heisenbug that heavily depends on timing. When the bug is not triggered it does > not really mean anything. It is also qestionable if we should be getting these > algs with uninitialized spawns. > > crypto/algapi.c | 22 +++++++++++++--------- > 1 file changed, 13 insertions(+), 9 deletions(-) > > diff --git a/crypto/algapi.c b/crypto/algapi.c > index 313a7682cef1..82125b82ffba 100644 > --- a/crypto/algapi.c > +++ b/crypto/algapi.c > @@ -151,6 +151,18 @@ void crypto_remove_spawns(struct crypto_alg *alg, struct list_head *list, > LIST_HEAD(top); > > spawns = &alg->cra_users; > + > + /* > + * We may encounter an unregistered instance here, since an instance's > + * spawns are set up prior to the instance being registered. > + * An unregistered instance will have NULL ->cra_users.next, since > + * ->cra_users isn't properly initialized until registration. But an > + * unregistered instance cannot have any users, so treat it the same as > + * ->cra_users being empty. > + */ > + if (spawns->next == NULL) > + return; > + > list_for_each_entry_safe(spawn, n, spawns, list) { > if ((spawn->alg->cra_flags ^ new_type) & spawn->mask) > continue; > @@ -177,15 +189,7 @@ void crypto_remove_spawns(struct crypto_alg *alg, struct list_head *list, > spawn->alg = NULL; > spawns = &inst->alg.cra_users; > > - /* > - * We may encounter an unregistered instance here, since > - * an instance's spawns are set up prior to the instance > - * being registered. An unregistered instance will have > - * NULL ->cra_users.next, since ->cra_users isn't > - * properly initialized until registration. But an > - * unregistered instance cannot have any users, so treat > - * it the same as ->cra_users being empty. > - */ > + /* Guard against unregistered instance */ > if (spawns->next == NULL) > break; > } > -- > 2.21.0 > The stack trace shows that crypto_remove_spawns() is being called from crypto_unregister_instance(). Therefore, the instance should already be registered and have initialized cra_users. Now, I don't claim to understand the spawn lists stuff that well, so I could have missed something; but if there *is* a bug, I'd like to see a proper explanation. Did you check whether this is actually reproducible on mainline, and not just the SUSE v4.12 based kernel? - Eric
On Tue, 25 Jun 2019 09:40:54 -0700 Eric Biggers <ebiggers@kernel.org> wrote: Hello, > Hi Michal, > > The stack trace shows that crypto_remove_spawns() is being called from > crypto_unregister_instance(). Therefore, the instance should already be > registered and have initialized cra_users. Now, I don't claim to understand the > spawn lists stuff that well, so I could have missed something; but if there *is* > a bug, I'd like to see a proper explanation. Unfortunately, I don't have an explanation either. > > Did you check whether this is actually reproducible on mainline, and not just > the SUSE v4.12 based kernel? Mainline crashes on boot:/ Need to find a recent working kernel. Thanks Michal
On Tue, 25 Jun 2019 09:40:54 -0700 Eric Biggers <ebiggers@kernel.org> wrote: > Hi Michal, > > On Tue, Jun 25, 2019 at 09:16:24AM +0200, Michal Suchanek wrote: > > Reportedly on Linux 4.12 the LTP testsuite crashes at pcrypt_aead01 infrequently. > > > > To get it reproduce more frequently I tried > > > > n=0 ; while true ; do /opt/ltp/testcases/bin/pcrypt_aead01 >& /dev/null ; n=$(expr $n + 1) ; echo -ne $n\\r ; done > > > > but this is quite stable. However, holding ^C in the terminal where the loop is running tends to trigger the crash. > > > > > > The code looks like this: > > > > 0xc000000000520e10 <+0>: c8 00 4c 3c addis r2,r12,200 > > 0xc000000000520e14 <+4>: f0 9b 42 38 addi r2,r2,-25616 > > 0xc000000000520e18 <+8>: a6 02 08 7c mflr r0 > > 0xc000000000520e1c <+12>: 00 00 00 60 nop > > 0xc000000000520e20 <+16>: 79 2b ab 7c mr. r11,r5 > > 0xc000000000520e24 <+20>: f0 ff c1 fb std r30,-16(r1) > > 0xc000000000520e28 <+24>: e8 ff a1 fb std r29,-24(r1) > > 0xc000000000520e2c <+28>: f8 ff e1 fb std r31,-8(r1) > > 0xc000000000520e30 <+32>: 91 ff 21 f8 stdu r1,-112(r1) > > 0xc000000000520e34 <+36>: 78 1b 69 7c mr r9,r3 > > 0xc000000000520e38 <+40>: 78 23 9e 7c mr r30,r4 > > 0xc000000000520e3c <+44>: 08 00 82 41 beq 0xc000000000520e44 <crypto_remove_spawns+52> > > 0xc000000000520e40 <+48>: 78 5b 69 7d mr r9,r11 > > 0xc000000000520e44 <+52>: 40 00 e1 3b addi r31,r1,64 > > 0xc000000000520e48 <+56>: 30 00 c1 38 addi r6,r1,48 > > # 0xc000000000520e4c <+60>: 10 00 43 e9 ld r10,16(r3) > > 0xc000000000520e50 <+64>: 20 00 a9 83 lwz r29,32(r9) > > 0xc000000000520e54 <+68>: 20 00 a1 38 addi r5,r1,32 > > 0xc000000000520e58 <+72>: 40 00 e1 fb std r31,64(r1) > > 0xc000000000520e5c <+76>: 48 00 e1 fb std r31,72(r1) > > 0xc000000000520e60 <+80>: 30 00 c1 f8 std r6,48(r1) > > 0xc000000000520e64 <+84>: 38 00 c1 f8 std r6,56(r1) > > 0xc000000000520e68 <+88>: 20 00 a1 f8 std r5,32(r1) > > 0xc000000000520e6c <+92>: 28 00 a1 f8 std r5,40(r1) > > 0xc000000000520e70 <+96>: 10 00 03 38 addi r0,r3,16 > > & 0xc000000000520e74 <+100>: 40 50 a0 7f cmpld cr7,r0,r10 > > 0xc000000000520e78 <+104>: 78 53 47 7d mr r7,r10 > > * 0xc000000000520e7c <+108>: 00 00 0a e9 ld r8,0(r10) > > 0xc000000000520e80 <+112>: 64 00 9e 41 beq cr7,0xc000000000520ee4 <crypto_remove_spawns+212> > > > > #) This looks like alg->cra_users.next is loaded to r10 > > &) This looks like r10 is compared with &alg->cra_users calculated on the line > > above to terminate the loop > > *) This looks like *alg->cra_users.next loaded into r8 which causes the null > > pointer dereference > > > > So the fixup needs to be applied to the first dereference of > > alg->cra_users.next as well to prevent crash. > > > > Fixes: 9a00674213a3 ("crypto: algapi - fix NULL dereference in crypto_remove_spawns()") > > > > Reported-by: chetjain@in.ibm.com > > The stack trace shows that crypto_remove_spawns() is being called from > crypto_unregister_instance(). Therefore, the instance should already be > registered and have initialized cra_users. Now, I don't claim to understand the > spawn lists stuff that well, so I could have missed something; but if there *is* > a bug, I'd like to see a proper explanation. > > Did you check whether this is actually reproducible on mainline, and not just > the SUSE v4.12 based kernel? This is the crash with mainline: BUG: Kernel NULL pointer dereference at 0x00000000 Faulting instruction address: 0xc0000000005bb280 Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries Modules linked in: authenc pcrypt crypto_user kvm_pr kvm ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet ibmveth(xX) vmx_crypto rtc_generic gf128mul btrfs libcrc32c xor zstd_decompress(nN) zstd_compress(nN) raid6_pq sd_mod sg dm_multipath dm_mod ibmvscsi(xX) scsi_dh_rdac scsi_dh_emc scsi_transport_srp scsi_dh_alua crc32c_vpmsum scsi_mod autofs4 Supported: No, Unreleased kernel CPU: 6 PID: 24816 Comm: pcrypt_aead01 Kdump: loaded Tainted: G 5.2.0-rc6-11.g9d2be15-default #1 SLE15 (unreleased) NIP: c0000000005bb280 LR: c0000000005bc108 CTR: c0000000005bc0b0 REGS: c0000005b574b590 TRAP: 0300 Tainted: G (5.2.0-rc6-11.g9d2be15-default) MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 44002840 XER: 20040000 CFAR: c00000000000e244 DAR: 0000000000000000 DSISR: 40000000 IRQMASK: 0 GPR00: c0000000005bc108 c0000005b574b820 c000000001406900 c0000005b2eabc00 GPR04: c0000005b574b8b0 0000000000000000 c0000005b574b850 0000000000000000 GPR08: 0000000000000000 c0000005b2eabc00 ffffffff00000001 c0000005b574b860 GPR12: c0000005b2eabc10 c000000007fa7800 0000000131b90ee0 00007fffc975b748 GPR16: 0000000131bb2d80 0000000131bb2d88 00007fffc975b5e0 00007fffc975b5d4 GPR20: 00007fffc975b628 00007fffc975b5f0 0000000000000010 0000000000000000 GPR24: 0000000000000000 0000000000000000 fffffffffffff000 0000000000000000 GPR28: c0000005b574b8b0 0000000000000cb3 c0000000013366f8 c0000005b574b840 CFAR: c00000000000e244 DAR: 0000000000000000 DSISR: 40000000 IRQMASK: 0 GPR00: c0000000005bc108 c0000005b574b820 c000000001406900 c0000005b2eabc00 GPR04: c0000005b574b8b0 0000000000000000 c0000005b574b850 0000000000000000 GPR08: 0000000000000000 c0000005b2eabc00 ffffffff00000001 c0000005b574b860 GPR12: c0000005b2eabc10 c000000007fa7800 0000000131b90ee0 00007fffc975b748 GPR16: 0000000131bb2d80 0000000131bb2d88 00007fffc975b5e0 00007fffc975b5d4 GPR20: 00007fffc975b628 00007fffc975b5f0 0000000000000010 0000000000000000 GPR24: 0000000000000000 0000000000000000 fffffffffffff000 0000000000000000 GPR28: c0000005b574b8b0 0000000000000cb3 c0000000013366f8 c0000005b574b840 NIP [c0000000005bb280] crypto_remove_spawns+0x70/0x2e0 LR [c0000000005bc108] crypto_unregister_instance+0x58/0xa0 Call Trace: [c0000005b574b820] [000000000000000c] 0xc (unreliable) [c0000005b574b890] [fffffffffffff000] 0xfffffffffffff000 [c0000005b574b8d0] [c0080000048811c4] crypto_del_alg+0xdc/0x110 [crypto_user] [c0000005b574b900] [c0080000048802b8] crypto_user_rcv_msg+0xe0/0x270 [crypto_user] [c0000005b574ba00] [c00000000095d8e4] netlink_rcv_skb+0x84/0x1a0 [c0000005b574ba70] [c008000004880074] crypto_netlink_rcv+0x4c/0x80 [crypto_user] [c0000005b574baa0] [c00000000095ce1c] netlink_unicast+0x1dc/0x2a0 [c0000005b574bb00] [c00000000095d25c] netlink_sendmsg+0x20c/0x430 [c0000005b574bba0] [c0000000008a09d0] sock_sendmsg+0x60/0x90 [c0000005b574bbd0] [c0000000008a151c] ___sys_sendmsg+0x31c/0x370 [c0000005b574bd80] [c0000000008a320c] __sys_sendmsg+0x6c/0xe0 [c0000005b574be20] [c00000000000b688] system_call+0x5c/0x70 Instruction dump: e9030010 83a90020 39610040 fbe10020 fbe10028 f8c10030 f8c10038 f9610040 f9610048 39830010 7c2c4040 7d074378 <e9480000> 41820060 60000000 60000000 ---[ end trace 4ff8403d5fbae222 ]--- Attaching config and dmesg. Thanks Michal [ 0.000000] printk: debug: ignoring loglevel setting. [ 0.000000] Reserving 512MB of memory at 128MB for crashkernel (System RAM: 23552MB) [ 0.000000] hash-mmu: Page sizes from device-tree: [ 0.000000] hash-mmu: base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=0 [ 0.000000] hash-mmu: base_shift=12: shift=16, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=7 [ 0.000000] hash-mmu: base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=56 [ 0.000000] hash-mmu: base_shift=16: shift=16, sllp=0x0110, avpnm=0x00000000, tlbiel=1, penc=1 [ 0.000000] hash-mmu: base_shift=16: shift=24, sllp=0x0110, avpnm=0x00000000, tlbiel=1, penc=8 [ 0.000000] hash-mmu: base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001, tlbiel=0, penc=0 [ 0.000000] hash-mmu: base_shift=34: shift=34, sllp=0x0120, avpnm=0x000007ff, tlbiel=0, penc=3 [ 0.000000] Page orders: linear mapping = 24, virtual = 16, io = 16, vmemmap = 24 [ 0.000000] Using 1TB segments [ 0.000000] hash-mmu: Initializing hash mmu with SLB [ 0.000000] Linux version 5.2.0-rc6-13.g21111ce-default (geeko@buildhost) (gcc version 9.1.1 20190611 [gcc-9-branch revision 272147] (SUSE Linux)) #1 SMP Sun Jun 30 19:30:23 UTC 2019 (21111ce) [ 0.000000] Found initrd at 0xc00000000c800000:0xc00000000d3274ec [ 0.000000] Using pSeries machine description [ 0.000000] printk: bootconsole [udbg0] enabled [ 0.000000] Partition configured for 8 cpus. [ 0.000000] CPU maps initialized for 8 threads per core [ 0.000000] (thread shift is 3) [ 0.000000] Allocated 4672 bytes for 8 pacas [ 0.000000] ----------------------------------------------------- [ 0.000000] phys_mem_size = 0x5c0000000 [ 0.000000] dcache_bsize = 0x80 [ 0.000000] icache_bsize = 0x80 [ 0.000000] cpu_features = 0x0000c07f8f5f91a7 [ 0.000000] possible = 0x0000fbffcf5fb1a7 [ 0.000000] always = 0x0000006f8b5c91a1 [ 0.000000] cpu_user_features = 0xdc0065c2 0xefe00000 [ 0.000000] mmu_features = 0x7c006001 [ 0.000000] firmware_features = 0x00000017c45bfc57 [ 0.000000] hash-mmu: ppc64_pft_size = 0x1d [ 0.000000] hash-mmu: htab_hash_mask = 0x3fffff [ 0.000000] hash-mmu: kernel vmalloc start = 0xc008000000000000 [ 0.000000] hash-mmu: kernel IO start = 0xc00a000000000000 [ 0.000000] hash-mmu: kernel vmemmap start = 0xc00c000000000000 [ 0.000000] ----------------------------------------------------- [ 0.000000] numa: NODE_DATA [mem 0x5bffa7000-0x5bffabfff] [ 0.000000] numa: NODE_DATA(0) on node 1 [ 0.000000] numa: NODE_DATA [mem 0x5bffa2000-0x5bffa6fff] [ 0.000000] rfi-flush: fallback displacement flush available [ 0.000000] rfi-flush: mttrig type flush available [ 0.000000] rfi-flush: patched 9 locations (mttrig type flush) [ 0.000000] count-cache-flush: software flush disabled. [ 0.000000] stf-barrier: eieio barrier available [ 0.000000] stf-barrier: patched 61 entry locations (eieio barrier) [ 0.000000] stf-barrier: patched 9 exit locations (eieio barrier) [ 0.000000] PPC64 nvram contains 15360 bytes [ 0.000000] barrier-nospec: using ORI speculation barrier [ 0.000000] barrier-nospec: patched 476 locations [ 0.000000] Top of RAM: 0x5c0000000, Total RAM: 0x5c0000000 [ 0.000000] Memory hole size: 0MB [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000000000000-0x00000005bfffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 1: [mem 0x0000000000000000-0x00000005bfffffff] [ 0.000000] Could not find start_pfn for node 0 [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000000000000] [ 0.000000] On node 0 totalpages: 0 [ 0.000000] Initmem setup node 1 [mem 0x0000000000000000-0x00000005bfffffff] [ 0.000000] On node 1 totalpages: 376832 [ 0.000000] Normal zone: 368 pages used for memmap [ 0.000000] Normal zone: 0 pages reserved [ 0.000000] Normal zone: 376832 pages, LIFO batch:3 [ 0.000000] percpu: Embedded 11 pages/cpu s624536 r0 d96360 u1048576 [ 0.000000] pcpu-alloc: s624536 r0 d96360 u1048576 alloc=1*1048576 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7 [ 0.000000] Built 2 zonelists, mobility grouping on. Total pages: 376464 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinux-5.2.0-rc6-13.g21111ce-default root=UUID=d00455bf-362e-4322-932a-e13a4bfe6a8a sysrq_always_enabled panic=100 ignore_loglevel unknown_nmi_panic console=hvc0 console=ttyS0,57600 splash=silent quiet showopts crashkernel=512M [ 0.000000] sysrq: sysrq always enabled. [ 0.000000] Memory: 0K/24117248K available (11200K kernel code, 1664K rwdata, 3328K rodata, 4672K init, 11694K bss, 597376K reserved, 0K cma-reserved) [ 0.000000] random: get_random_u32 called from cache_grow_begin+0x218/0x880 with crng_init=0 [ 0.000000] ftrace: allocating 29620 entries in 11 pages [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=2048 to nr_cpu_ids=8. [ 0.000000] Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=8 [ 0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16 [ 0.000000] pic: no ISA interrupt controller [ 0.000000] rcu: Offload RCU callbacks from CPUs: (none). [ 0.000000] time_init: decrementer frequency = 512.000000 MHz [ 0.000000] time_init: processor frequency = 2300.000000 MHz [ 0.000002] time_init: 56 bit decrementer (max: 7fffffffffffff) [ 0.000046] clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0x761537d007, max_idle_ns: 440795202126 ns [ 0.000132] clocksource: timebase mult[1f40000] shift[24] registered [ 0.000179] clockevent: decrementer mult[83126f] shift[24] cpu[0] [ 0.000503] Console: colour dummy device 80x25 [ 0.004726] printk: console [hvc0] enabled [ 0.004790] mempolicy: Disabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl [ 0.004889] pid_max: default: 32768 minimum: 301 [ 0.005027] LSM: Security Framework initializing [ 0.005096] AppArmor: AppArmor initialized [ 0.007123] Dentry cache hash table entries: 4194304 (order: 9, 33554432 bytes) [ 0.008163] Inode-cache hash table entries: 2097152 (order: 8, 16777216 bytes) [ 0.008259] Mount-cache hash table entries: 65536 (order: 3, 524288 bytes) [ 0.008346] Mountpoint-cache hash table entries: 65536 (order: 3, 524288 bytes) [ 0.008603] *** VALIDATE proc *** [ 0.008711] *** VALIDATE cgroup1 *** [ 0.008728] *** VALIDATE cgroup2 *** [ 0.008949] EEH: pSeries platform initialized [ 0.008974] POWER9 performance monitor hardware support registered [ 0.009041] rcu: Hierarchical SRCU implementation. [ 0.009385] smp: Bringing up secondary CPUs ... [ 0.011715] smp: Brought up 2 nodes, 8 CPUs [ 0.020018] numa: Node 0 CPUs: [ 0.020034] numa: Node 1 CPUs: 0-7 [ 0.020055] Using small cores at SMT level [ 0.020081] Using shared cache scheduler topology [ 0.023422] node 1 initialised, 366473 pages in 0ms [ 0.024316] devtmpfs: initialized [ 0.025870] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.025956] futex hash table entries: 2048 (order: 2, 262144 bytes) [ 0.026327] NET: Registered protocol family 16 [ 0.026504] audit: initializing netlink subsys (disabled) [ 0.026580] audit: type=2000 audit(1561928628.020:1): state=initialized audit_enabled=0 res=1 [ 0.026664] cpuidle: using governor ladder [ 0.026699] cpuidle: using governor menu [ 0.026770] RTAS daemon started [ 0.026886] pstore: Registered nvram as persistent store backend [ 0.028280] PCI: Probing PCI hardware [ 0.028299] EEH: No capable adapters found [ 0.028327] PCI: Probing PCI hardware done [ 0.028446] pseries-rng: Registering arch random hook. [ 0.029256] WARNING: workqueue cpumask: online intersect > possible intersect [ 0.029375] HugeTLB registered 16.0 MiB page size, pre-allocated 0 pages [ 0.029433] HugeTLB registered 16.0 GiB page size, pre-allocated 0 pages [ 0.161542] alg: No test for lzo-rle (lzo-rle-generic) [ 0.161616] alg: No test for lzo-rle (lzo-rle-scomp) [ 0.171901] vgaarb: loaded [ 0.171967] EDAC MC: Ver: 3.0.0 [ 0.172145] NetLabel: Initializing [ 0.172162] NetLabel: domain hash size = 128 [ 0.172191] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.172246] NetLabel: unlabeled traffic allowed by default [ 0.172379] clocksource: Switched to clocksource timebase [ 0.184431] VFS: Disk quotas dquot_6.6.0 [ 0.184474] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes) [ 0.184546] *** VALIDATE hugetlbfs *** [ 0.184671] AppArmor: AppArmor Filesystem Enabled [ 0.186046] NET: Registered protocol family 2 [ 0.186235] tcp_listen_portaddr_hash hash table entries: 16384 (order: 2, 262144 bytes) [ 0.186336] TCP established hash table entries: 262144 (order: 5, 2097152 bytes) [ 0.186782] TCP bind hash table entries: 65536 (order: 4, 1048576 bytes) [ 0.186932] TCP: Hash tables configured (established 262144 bind 65536) [ 0.186997] UDP hash table entries: 16384 (order: 3, 524288 bytes) [ 0.187096] UDP-Lite hash table entries: 16384 (order: 3, 524288 bytes) [ 0.187248] NET: Registered protocol family 1 [ 0.187276] PCI: CLS 0 bytes, default 128 [ 0.187333] Trying to unpack rootfs image as initramfs... [ 1.440198] IOMMU table initialized, virtual merging enabled [ 1.469747] hv-24x7: found a duplicate event RESERVED, ct=1 [ 1.469785] hv-24x7: found a duplicate event RESERVED, ct=2 [ 1.469831] hv-24x7: found a duplicate event RESERVED, ct=3 [ 1.469877] hv-24x7: found a duplicate event RESERVED, ct=4 [ 1.469918] hv-24x7: found a duplicate event RESERVED, ct=5 [ 1.469964] hv-24x7: found a duplicate event RESERVED, ct=6 [ 1.470008] hv-24x7: found a duplicate event RESERVED, ct=7 [ 1.470068] hv-24x7: found a duplicate event RESERVED, ct=8 [ 1.470111] hv-24x7: found a duplicate event RESERVED, ct=9 [ 1.470156] hv-24x7: found a duplicate event RESERVED, ct=10 [ 1.470204] hv-24x7: found a duplicate event RESERVED, ct=11 [ 1.470250] hv-24x7: found a duplicate event RESERVED, ct=12 [ 1.470296] hv-24x7: found a duplicate event RESERVED, ct=13 [ 1.470338] hv-24x7: found a duplicate event RESERVED, ct=14 [ 1.470385] hv-24x7: found a duplicate event RESERVED, ct=15 [ 1.470429] hv-24x7: found a duplicate event RESERVED, ct=16 [ 1.470474] hv-24x7: found a duplicate event RESERVED, ct=17 [ 1.470521] hv-24x7: found a duplicate event RESERVED, ct=18 [ 1.470568] hv-24x7: found a duplicate event RESERVED, ct=19 [ 1.470616] hv-24x7: found a duplicate event RESERVED, ct=20 [ 1.470663] hv-24x7: found a duplicate event RESERVED, ct=21 [ 1.470713] hv-24x7: found a duplicate event RESERVED, ct=22 [ 1.470755] hv-24x7: found a duplicate event RESERVED, ct=23 [ 1.470801] hv-24x7: found a duplicate event RESERVED, ct=24 [ 1.470846] hv-24x7: found a duplicate event RESERVED, ct=25 [ 1.470892] hv-24x7: found a duplicate event RESERVED, ct=26 [ 1.470941] hv-24x7: found a duplicate event RESERVED, ct=27 [ 1.470999] hv-24x7: found a duplicate event RESERVED, ct=28 [ 1.471046] hv-24x7: found a duplicate event RESERVED, ct=29 [ 1.471094] hv-24x7: found a duplicate event RESERVED, ct=30 [ 1.471141] hv-24x7: found a duplicate event RESERVED, ct=31 [ 1.471185] hv-24x7: found a duplicate event RESERVED, ct=32 [ 1.471233] hv-24x7: found a duplicate event RESERVED, ct=33 [ 1.471279] hv-24x7: found a duplicate event RESERVED, ct=34 [ 1.471326] hv-24x7: found a duplicate event RESERVED, ct=35 [ 1.471371] hv-24x7: found a duplicate event RESERVED, ct=36 [ 1.471414] hv-24x7: found a duplicate event RESERVED, ct=37 [ 1.471458] hv-24x7: found a duplicate event RESERVED, ct=38 [ 1.471502] hv-24x7: found a duplicate event RESERVED, ct=39 [ 1.471548] hv-24x7: found a duplicate event RESERVED, ct=40 [ 1.471592] hv-24x7: found a duplicate event RESERVED, ct=41 [ 1.471637] hv-24x7: found a duplicate event RESERVED, ct=42 [ 1.471681] hv-24x7: found a duplicate event RESERVED, ct=43 [ 1.471726] hv-24x7: found a duplicate event RESERVED, ct=44 [ 1.471773] hv-24x7: found a duplicate event RESERVED, ct=45 [ 1.471820] hv-24x7: found a duplicate event RESERVED, ct=46 [ 1.471865] hv-24x7: found a duplicate event RESERVED, ct=47 [ 1.471952] hv-24x7: read 1463 catalog entries, created 470 event attrs (0 failures), 275 descs [ 1.473167] Initialise system trusted keyrings [ 1.473240] workingset: timestamp_bits=38 max_order=19 bucket_order=0 [ 1.473401] zbud: loaded [ 1.483435] Key type asymmetric registered [ 1.483456] Asymmetric key parser 'x509' registered [ 1.483498] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249) [ 1.483612] io scheduler mq-deadline registered [ 1.483652] io scheduler kyber registered [ 1.483694] io scheduler bfq registered [ 1.484059] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled [ 1.484239] Non-volatile memory driver v1.3 [ 1.484274] pseries_rng: Registering IBM pSeries RNG driver [ 1.484513] tpm_ibmvtpm 30000004: CRQ initialization completed [ 1.484672] mousedev: PS/2 mouse device common for all mice [ 1.484807] pseries_idle_driver registered [ 1.484882] ledtrig-cpu: registered to indicate activity on CPUs [ 1.484982] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sync_size new:65536 old:0 [ 1.485103] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sync_sg new:510 old:0 [ 1.485220] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sg_len new:4080 old:0 [ 1.485367] alg: No test for 842 (842-nx) [ 1.486448] hidraw: raw HID events driver (C) Jiri Kosina [ 1.486578] NET: Registered protocol family 10 [ 1.489375] Segment Routing with IPv6 [ 1.489409] NET: Registered protocol family 15 [ 1.489797] registered taskstats version 1 [ 1.489820] Loading compiled-in X.509 certificates [ 1.520515] Loaded X.509 cert 'Unsupported: 20a8b0cf2d570fb2c20316bf6f6d9681f4981f2c' [ 1.520602] zswap: loaded using pool lzo/zbud [ 1.520681] page_owner is disabled [ 1.520740] pstore: Using crash dump compression: deflate [ 1.520791] Key type trusted registered [ 1.523316] Key type encrypted registered [ 1.523339] AppArmor: AppArmor sha1 policy hashing enabled [ 1.523388] ima: Allocated hash algorithm: sha256 [ 1.524239] No architecture policies found [ 1.524269] evm: Initialising EVM extended attributes: [ 1.524308] evm: security.selinux [ 1.524330] evm: security.apparmor [ 1.524353] evm: security.ima [ 1.524369] evm: security.capability [ 1.524394] evm: HMAC attrs: 0x1 [ 1.525926] Freeing unused kernel memory: 4672K [ 1.525951] This architecture does not have kernel memory protection. [ 1.526005] Run /init as init process [ 1.547342] systemd[1]: systemd 234 running in system mode. (+PAM -AUDIT +SELINUX -IMA +APPARMOR -SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 -IDN default-hierarchy=hybrid) [ 1.547637] systemd[1]: Detected architecture ppc64-le. [ 1.547672] systemd[1]: Running in initial RAM disk. [ 1.547721] systemd[1]: Set hostname to <vugava-5>. [ 1.587095] random: systemd: uninitialized urandom read (16 bytes read) [ 1.587162] systemd[1]: Reached target Local File Systems. [ 1.587235] random: systemd: uninitialized urandom read (16 bytes read) [ 1.587316] systemd[1]: Listening on Journal Socket (/dev/log). [ 1.587366] random: systemd: uninitialized urandom read (16 bytes read) [ 1.587857] systemd[1]: Created slice System Slice. [ 1.587912] systemd[1]: Listening on udev Kernel Socket. [ 1.587988] systemd[1]: Listening on Journal Socket. [ 1.588498] systemd[1]: Started Entropy Daemon based on the HAVEGE algorithm. [ 1.824646] synth uevent: /devices/vio: failed to send uevent [ 1.824748] vio vio: uevent: failed to send synthetic uevent [ 1.824919] synth uevent: /devices/vio/4000: failed to send uevent [ 1.825027] vio 4000: uevent: failed to send synthetic uevent [ 1.825148] synth uevent: /devices/vio/4001: failed to send uevent [ 1.825267] vio 4001: uevent: failed to send synthetic uevent [ 1.825380] synth uevent: /devices/vio/4002: failed to send uevent [ 1.825504] vio 4002: uevent: failed to send synthetic uevent [ 1.825633] synth uevent: /devices/vio/4004: failed to send uevent [ 1.825771] vio 4004: uevent: failed to send synthetic uevent [ 1.892883] random: crng init done [ 1.892908] random: 7 urandom warning(s) missed due to ratelimiting [ 2.043492] SCSI subsystem initialized [ 2.044427] alua: device handler registered [ 2.045321] emc: device handler registered [ 2.046045] rdac: device handler registered [ 2.046751] ibmvscsi 30000003: SRP_VERSION: 16.a [ 2.046903] ibmvscsi 30000003: Maximum ID: 64 Maximum LUN: 32 Maximum Channel: 3 [ 2.046976] scsi host0: IBM POWER Virtual SCSI Adapter 1.5.9 [ 2.047216] ibmvscsi 30000003: partner initialization complete [ 2.047308] ibmvscsi 30000003: host srp version: 16.a, host partition vios1 (2), OS 3, max io 262144 [ 2.047458] ibmvscsi 30000003: Client reserve enabled [ 2.047516] ibmvscsi 30000003: sent SRP login [ 2.047564] ibmvscsi 30000003: SRP_LOGIN succeeded [ 2.050841] device-mapper: uevent: version 1.0.3 [ 2.050928] device-mapper: ioctl: 4.40.0-ioctl (2019-01-18) initialised: dm-devel@redhat.com [ 2.072851] scsi 0:0:1:0: Direct-Access AIX VDASD 0001 PQ: 0 ANSI: 3 [ 2.073133] scsi 0:0:1:0: Attached scsi generic sg0 type 0 [ 2.105681] sd 0:0:1:0: [sda] 10485760 4096-byte logical blocks: (42.9 GB/40.0 GiB) [ 2.105808] sd 0:0:1:0: [sda] Write Protect is off [ 2.105846] sd 0:0:1:0: [sda] Mode Sense: 17 00 00 08 [ 2.105936] sd 0:0:1:0: [sda] Cache data unavailable [ 2.105976] sd 0:0:1:0: [sda] Assuming drive cache: write through [ 2.132465] sda: sda1 sda2 sda3 sda4 sda5 [ 2.133154] sd 0:0:1:0: [sda] Attached SCSI disk [ 2.572383] raid6: vpermxor8 gen() 18658 MB/s [ 2.742383] raid6: vpermxor4 gen() 16405 MB/s [ 2.912384] raid6: vpermxor2 gen() 15206 MB/s [ 3.082384] raid6: vpermxor1 gen() 8727 MB/s [ 3.252386] raid6: altivecx8 gen() 14560 MB/s [ 3.422385] raid6: altivecx4 gen() 16406 MB/s [ 3.592379] raid6: altivecx2 gen() 13519 MB/s [ 3.762386] raid6: altivecx1 gen() 8741 MB/s [ 3.932391] raid6: int64x8 gen() 6500 MB/s [ 4.102388] raid6: int64x8 xor() 2759 MB/s [ 4.272378] raid6: int64x4 gen() 12704 MB/s [ 4.442389] raid6: int64x4 xor() 4448 MB/s [ 4.612378] raid6: int64x2 gen() 9865 MB/s [ 4.782381] raid6: int64x2 xor() 3611 MB/s [ 4.952381] raid6: int64x1 gen() 5543 MB/s [ 5.122396] raid6: int64x1 xor() 2231 MB/s [ 5.122419] raid6: using algorithm vpermxor8 gen() 18658 MB/s [ 5.122463] raid6: using intx1 recovery algorithm [ 5.126113] xor: measuring software checksum speed [ 5.222378] 8regs : 20153.600 MB/sec [ 5.322385] 8regs_prefetch: 16729.600 MB/sec [ 5.422380] 32regs : 20403.200 MB/sec [ 5.522376] 32regs_prefetch: 18073.600 MB/sec [ 5.622379] altivec : 23686.400 MB/sec [ 5.622405] xor: using function: altivec (23686.400 MB/sec) [ 5.664870] Btrfs loaded, crc32c=crc32c-vpmsum, assert=on [ 5.671431] BTRFS: device label SLES devid 1 transid 9578 /dev/sda5 [ 5.726717] BTRFS info (device sda5): disk space caching is enabled [ 5.726777] BTRFS info (device sda5): has skinny extents [ 6.007805] systemd-journald[229]: Received SIGTERM from PID 1 (systemd). [ 6.063928] printk: systemd: 13 output lines suppressed due to ratelimiting [ 6.782505] BTRFS info (device sda5): disk space caching is enabled [ 6.844482] synth uevent: /devices/vio: failed to send uevent [ 6.844522] vio vio: uevent: failed to send synthetic uevent [ 6.844766] synth uevent: /devices/vio/4000: failed to send uevent [ 6.844809] vio 4000: uevent: failed to send synthetic uevent [ 6.844861] synth uevent: /devices/vio/4001: failed to send uevent [ 6.844909] vio 4001: uevent: failed to send synthetic uevent [ 6.844962] synth uevent: /devices/vio/4002: failed to send uevent [ 6.845011] vio 4002: uevent: failed to send synthetic uevent [ 6.845059] synth uevent: /devices/vio/4004: failed to send uevent [ 6.845107] vio 4004: uevent: failed to send synthetic uevent [ 7.015498] rtc-generic rtc-generic: registered as rtc0 [ 7.044578] crypto_register_alg 'aes' = 0 [ 7.068278] crypto_register_alg 'cbc(aes)' = 0 [ 7.080059] ibmveth: IBM Power Virtual Ethernet Driver 1.06 [ 7.086405] crypto_register_alg 'ctr(aes)' = 0 [ 7.128241] crypto_register_alg 'xts(aes)' = 0 [ 7.259826] Adding 1048512k swap on /dev/sda3. Priority:-2 extents:1 across:1048512k FS [ 7.383980] systemd-journald[429]: Received request to flush runtime journal from PID 1 [ 7.435628] audit: type=1400 audit(1561928635.430:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="ping" pid=964 comm="apparmor_parser" [ 7.451141] audit: type=1400 audit(1561928635.440:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="klogd" pid=974 comm="apparmor_parser" [ 8.841878] NET: Registered protocol family 17
On Mon, Jul 01, 2019 at 03:31:54PM +0200, Michal Suchánek wrote: > On Tue, 25 Jun 2019 09:40:54 -0700 > Eric Biggers <ebiggers@kernel.org> wrote: > > > Hi Michal, > > > > On Tue, Jun 25, 2019 at 09:16:24AM +0200, Michal Suchanek wrote: > > > Reportedly on Linux 4.12 the LTP testsuite crashes at pcrypt_aead01 infrequently. > > > > > > To get it reproduce more frequently I tried > > > > > > n=0 ; while true ; do /opt/ltp/testcases/bin/pcrypt_aead01 >& /dev/null ; n=$(expr $n + 1) ; echo -ne $n\\r ; done > > > > > > but this is quite stable. However, holding ^C in the terminal where the loop is running tends to trigger the crash. > > > > > > > > > > The code looks like this: > > > > > > 0xc000000000520e10 <+0>: c8 00 4c 3c addis r2,r12,200 > > > 0xc000000000520e14 <+4>: f0 9b 42 38 addi r2,r2,-25616 > > > 0xc000000000520e18 <+8>: a6 02 08 7c mflr r0 > > > 0xc000000000520e1c <+12>: 00 00 00 60 nop > > > 0xc000000000520e20 <+16>: 79 2b ab 7c mr. r11,r5 > > > 0xc000000000520e24 <+20>: f0 ff c1 fb std r30,-16(r1) > > > 0xc000000000520e28 <+24>: e8 ff a1 fb std r29,-24(r1) > > > 0xc000000000520e2c <+28>: f8 ff e1 fb std r31,-8(r1) > > > 0xc000000000520e30 <+32>: 91 ff 21 f8 stdu r1,-112(r1) > > > 0xc000000000520e34 <+36>: 78 1b 69 7c mr r9,r3 > > > 0xc000000000520e38 <+40>: 78 23 9e 7c mr r30,r4 > > > 0xc000000000520e3c <+44>: 08 00 82 41 beq 0xc000000000520e44 <crypto_remove_spawns+52> > > > 0xc000000000520e40 <+48>: 78 5b 69 7d mr r9,r11 > > > 0xc000000000520e44 <+52>: 40 00 e1 3b addi r31,r1,64 > > > 0xc000000000520e48 <+56>: 30 00 c1 38 addi r6,r1,48 > > > # 0xc000000000520e4c <+60>: 10 00 43 e9 ld r10,16(r3) > > > 0xc000000000520e50 <+64>: 20 00 a9 83 lwz r29,32(r9) > > > 0xc000000000520e54 <+68>: 20 00 a1 38 addi r5,r1,32 > > > 0xc000000000520e58 <+72>: 40 00 e1 fb std r31,64(r1) > > > 0xc000000000520e5c <+76>: 48 00 e1 fb std r31,72(r1) > > > 0xc000000000520e60 <+80>: 30 00 c1 f8 std r6,48(r1) > > > 0xc000000000520e64 <+84>: 38 00 c1 f8 std r6,56(r1) > > > 0xc000000000520e68 <+88>: 20 00 a1 f8 std r5,32(r1) > > > 0xc000000000520e6c <+92>: 28 00 a1 f8 std r5,40(r1) > > > 0xc000000000520e70 <+96>: 10 00 03 38 addi r0,r3,16 > > > & 0xc000000000520e74 <+100>: 40 50 a0 7f cmpld cr7,r0,r10 > > > 0xc000000000520e78 <+104>: 78 53 47 7d mr r7,r10 > > > * 0xc000000000520e7c <+108>: 00 00 0a e9 ld r8,0(r10) > > > 0xc000000000520e80 <+112>: 64 00 9e 41 beq cr7,0xc000000000520ee4 <crypto_remove_spawns+212> > > > > > > #) This looks like alg->cra_users.next is loaded to r10 > > > &) This looks like r10 is compared with &alg->cra_users calculated on the line > > > above to terminate the loop > > > *) This looks like *alg->cra_users.next loaded into r8 which causes the null > > > pointer dereference > > > > > > So the fixup needs to be applied to the first dereference of > > > alg->cra_users.next as well to prevent crash. > > > > > > Fixes: 9a00674213a3 ("crypto: algapi - fix NULL dereference in crypto_remove_spawns()") > > > > > > Reported-by: chetjain@in.ibm.com > > > > > The stack trace shows that crypto_remove_spawns() is being called from > > crypto_unregister_instance(). Therefore, the instance should already be > > registered and have initialized cra_users. Now, I don't claim to understand the > > spawn lists stuff that well, so I could have missed something; but if there *is* > > a bug, I'd like to see a proper explanation. > > > > Did you check whether this is actually reproducible on mainline, and not just > > the SUSE v4.12 based kernel? > > This is the crash with mainline: > > BUG: Kernel NULL pointer dereference at 0x00000000 > Faulting instruction address: 0xc0000000005bb280 > Oops: Kernel access of bad area, sig: 11 [#1] > LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries > Modules linked in: authenc pcrypt crypto_user kvm_pr kvm ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables x_tables af_packet ibmveth(xX) vmx_crypto rtc_generic gf128mul btrfs libcrc32c xor zstd_decompress(nN) zstd_compress(nN) raid6_pq sd_mod sg dm_multipath dm_mod ibmvscsi(xX) scsi_dh_rdac scsi_dh_emc scsi_transport_srp scsi_dh_alua crc32c_vpmsum scsi_mod autofs4 > Supported: No, Unreleased kernel > CPU: 6 PID: 24816 Comm: pcrypt_aead01 Kdump: loaded Tainted: G 5.2.0-rc6-11.g9d2be15-default #1 SLE15 (unreleased) > NIP: c0000000005bb280 LR: c0000000005bc108 CTR: c0000000005bc0b0 > REGS: c0000005b574b590 TRAP: 0300 Tainted: G (5.2.0-rc6-11.g9d2be15-default) > MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 44002840 XER: 20040000 > CFAR: c00000000000e244 DAR: 0000000000000000 DSISR: 40000000 IRQMASK: 0 > GPR00: c0000000005bc108 c0000005b574b820 c000000001406900 c0000005b2eabc00 > GPR04: c0000005b574b8b0 0000000000000000 c0000005b574b850 0000000000000000 > GPR08: 0000000000000000 c0000005b2eabc00 ffffffff00000001 c0000005b574b860 > GPR12: c0000005b2eabc10 c000000007fa7800 0000000131b90ee0 00007fffc975b748 > GPR16: 0000000131bb2d80 0000000131bb2d88 00007fffc975b5e0 00007fffc975b5d4 > GPR20: 00007fffc975b628 00007fffc975b5f0 0000000000000010 0000000000000000 > GPR24: 0000000000000000 0000000000000000 fffffffffffff000 0000000000000000 > GPR28: c0000005b574b8b0 0000000000000cb3 c0000000013366f8 c0000005b574b840 > CFAR: c00000000000e244 DAR: 0000000000000000 DSISR: 40000000 IRQMASK: 0 > GPR00: c0000000005bc108 c0000005b574b820 c000000001406900 c0000005b2eabc00 > GPR04: c0000005b574b8b0 0000000000000000 c0000005b574b850 0000000000000000 > GPR08: 0000000000000000 c0000005b2eabc00 ffffffff00000001 c0000005b574b860 > GPR12: c0000005b2eabc10 c000000007fa7800 0000000131b90ee0 00007fffc975b748 > GPR16: 0000000131bb2d80 0000000131bb2d88 00007fffc975b5e0 00007fffc975b5d4 > GPR20: 00007fffc975b628 00007fffc975b5f0 0000000000000010 0000000000000000 > GPR24: 0000000000000000 0000000000000000 fffffffffffff000 0000000000000000 > GPR28: c0000005b574b8b0 0000000000000cb3 c0000000013366f8 c0000005b574b840 > NIP [c0000000005bb280] crypto_remove_spawns+0x70/0x2e0 > LR [c0000000005bc108] crypto_unregister_instance+0x58/0xa0 > Call Trace: > [c0000005b574b820] [000000000000000c] 0xc (unreliable) > [c0000005b574b890] [fffffffffffff000] 0xfffffffffffff000 > [c0000005b574b8d0] [c0080000048811c4] crypto_del_alg+0xdc/0x110 [crypto_user] > [c0000005b574b900] [c0080000048802b8] crypto_user_rcv_msg+0xe0/0x270 [crypto_user] > [c0000005b574ba00] [c00000000095d8e4] netlink_rcv_skb+0x84/0x1a0 > [c0000005b574ba70] [c008000004880074] crypto_netlink_rcv+0x4c/0x80 [crypto_user] > [c0000005b574baa0] [c00000000095ce1c] netlink_unicast+0x1dc/0x2a0 > [c0000005b574bb00] [c00000000095d25c] netlink_sendmsg+0x20c/0x430 > [c0000005b574bba0] [c0000000008a09d0] sock_sendmsg+0x60/0x90 > [c0000005b574bbd0] [c0000000008a151c] ___sys_sendmsg+0x31c/0x370 > [c0000005b574bd80] [c0000000008a320c] __sys_sendmsg+0x6c/0xe0 > [c0000005b574be20] [c00000000000b688] system_call+0x5c/0x70 > Instruction dump: > e9030010 83a90020 39610040 fbe10020 fbe10028 f8c10030 f8c10038 f9610040 > f9610048 39830010 7c2c4040 7d074378 <e9480000> 41820060 60000000 60000000 > ---[ end trace 4ff8403d5fbae222 ]--- > > Attaching config and dmesg. > Thanks, I was able to reproduce this, and I came up with a different fix. I sent it out, but for some reason it doesn't seem to have reached any of the lists... If I still don't see after a little while, I'll resend it. - Eric
diff --git a/crypto/algapi.c b/crypto/algapi.c index 313a7682cef1..82125b82ffba 100644 --- a/crypto/algapi.c +++ b/crypto/algapi.c @@ -151,6 +151,18 @@ void crypto_remove_spawns(struct crypto_alg *alg, struct list_head *list, LIST_HEAD(top); spawns = &alg->cra_users; + + /* + * We may encounter an unregistered instance here, since an instance's + * spawns are set up prior to the instance being registered. + * An unregistered instance will have NULL ->cra_users.next, since + * ->cra_users isn't properly initialized until registration. But an + * unregistered instance cannot have any users, so treat it the same as + * ->cra_users being empty. + */ + if (spawns->next == NULL) + return; + list_for_each_entry_safe(spawn, n, spawns, list) { if ((spawn->alg->cra_flags ^ new_type) & spawn->mask) continue; @@ -177,15 +189,7 @@ void crypto_remove_spawns(struct crypto_alg *alg, struct list_head *list, spawn->alg = NULL; spawns = &inst->alg.cra_users; - /* - * We may encounter an unregistered instance here, since - * an instance's spawns are set up prior to the instance - * being registered. An unregistered instance will have - * NULL ->cra_users.next, since ->cra_users isn't - * properly initialized until registration. But an - * unregistered instance cannot have any users, so treat - * it the same as ->cra_users being empty. - */ + /* Guard against unregistered instance */ if (spawns->next == NULL) break; }
Reportedly on Linux 4.12 the LTP testsuite crashes at pcrypt_aead01 infrequently. To get it reproduce more frequently I tried n=0 ; while true ; do /opt/ltp/testcases/bin/pcrypt_aead01 >& /dev/null ; n=$(expr $n + 1) ; echo -ne $n\\r ; done but this is quite stable. However, holding ^C in the terminal where the loop is running tends to trigger the crash. The backtrace is: [ 100.615804] Unable to handle kernel paging request for data at address 0x00000000 [ 100.615876] Faulting instruction address: 0xc000000000520e7c [ 100.615943] Oops: Kernel access of bad area, sig: 11 [#1] [ 100.616001] SMP NR_CPUS=2048 [ 100.616002] NUMA [ 100.616030] pSeries [ 100.616054] Modules linked in: authenc pcrypt crypto_user kvm_pr kvm ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink ip_tables x_tables af_packet rtc_generic vmx_crypto ibmveth(X) gf128mul btrfs xor raid6_pq sd_mod ibmvscsi(X) scsi_transport_srp crc32c_vpmsum sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod autofs4 [ 100.616478] Supported: Yes, External [ 100.616509] CPU: 5 PID: 6270 Comm: pcrypt_aead01 Tainted: G 4.12.14-150.22-default #1 SLE15 [ 100.616632] task: c000000595084d80 task.stack: c0000005be6dc000 [ 100.616708] NIP: c000000000520e7c LR: c000000000521e3c CTR: c000000000521de0 [ 100.616801] REGS: c0000005be6df620 TRAP: 0300 Tainted: G (4.12.14-150.22-default) [ 100.616906] MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> [ 100.616912] CR: 24002844 XER: 20040000 [ 100.617003] CFAR: c000000000008860 DAR: 0000000000000000 DSISR: 40000000 SOFTE: 1 GPR00: c0000005a331f810 c0000005be6df8a0 c00000000119aa00 c0000005a331f800 GPR04: c0000005be6df930 c0000005be6df8c0 c0000005be6df8d0 0000000000000000 GPR08: 7269632929290000 c0000005a331f800 0000000000000000 0000000000000000 GPR12: c000000000521de0 c000000007a33700 00000001271a0ee0 00007fffcb9e7bb8 GPR16: 00000001271c2d80 00000001271c2d88 00007fffcb9e7a50 00007fffcb9e7a44 GPR20: 00007fffcb9e7a98 00007fffcb9e7a60 0000000000000010 0000000000000010 GPR24: 0000000000000000 0000000000000000 fffffffffffff000 c0000005be6dfaf0 GPR28: c0000005b9929d00 0000000000000c93 c0000005be6df930 c0000005be6df8e0 [ 100.617774] NIP [c000000000520e7c] crypto_remove_spawns+0x6c/0x2e0 [ 100.617816] LR [c000000000521e3c] crypto_unregister_instance+0x5c/0xa0 [ 100.617881] Call Trace: [ 100.617903] [c0000005be6df8a0] [c0000005b9929d00] 0xc0000005b9929d00 (unreliable) [ 100.617971] [c0000005be6df910] [0000000000000000] (null) [ 100.618021] [c0000005be6df960] [d0000000098d0894] crypto_del_alg+0xdc/0x110 [crypto_user] [ 100.618119] [c0000005be6df990] [d0000000098d0b58] crypto_user_rcv_msg+0xe0/0x260 [crypto_user] [ 100.618222] [c0000005be6dfa30] [c00000000086d678] netlink_rcv_skb+0x78/0x170 [ 100.618309] [c0000005be6dfaa0] [d0000000098d0064] crypto_netlink_rcv+0x4c/0x80 [crypto_user] [ 100.618407] [c0000005be6dfad0] [c00000000086cb98] netlink_unicast+0x208/0x2f0 [ 100.618488] [c0000005be6dfb40] [c00000000086d170] netlink_sendmsg+0x380/0x440 [ 100.618582] [c0000005be6dfbd0] [c0000000007e9ba4] sock_sendmsg+0x64/0x90 [ 100.618650] [c0000005be6dfc00] [c0000000007eb94c] ___sys_sendmsg+0x2cc/0x330 [ 100.618710] [c0000005be6dfd90] [c0000000007ed02c] __sys_sendmsg+0x5c/0xc0 [ 100.618766] [c0000005be6dfe30] [c00000000000b188] system_call+0x3c/0x130 [ 100.618822] Instruction dump: [ 100.618839] e9430010 83a90020 38a10020 fbe10040 fbe10048 f8c10030 f8c10038 f8a10020 [ 100.618902] f8a10028 38030010 7fa05040 7d475378 <e90a0000> 419e0064 60000000 60000000 [ 100.618980] ---[ end trace 60475621348ca387 ]--- The code looks like this: 0xc000000000520e10 <+0>: c8 00 4c 3c addis r2,r12,200 0xc000000000520e14 <+4>: f0 9b 42 38 addi r2,r2,-25616 0xc000000000520e18 <+8>: a6 02 08 7c mflr r0 0xc000000000520e1c <+12>: 00 00 00 60 nop 0xc000000000520e20 <+16>: 79 2b ab 7c mr. r11,r5 0xc000000000520e24 <+20>: f0 ff c1 fb std r30,-16(r1) 0xc000000000520e28 <+24>: e8 ff a1 fb std r29,-24(r1) 0xc000000000520e2c <+28>: f8 ff e1 fb std r31,-8(r1) 0xc000000000520e30 <+32>: 91 ff 21 f8 stdu r1,-112(r1) 0xc000000000520e34 <+36>: 78 1b 69 7c mr r9,r3 0xc000000000520e38 <+40>: 78 23 9e 7c mr r30,r4 0xc000000000520e3c <+44>: 08 00 82 41 beq 0xc000000000520e44 <crypto_remove_spawns+52> 0xc000000000520e40 <+48>: 78 5b 69 7d mr r9,r11 0xc000000000520e44 <+52>: 40 00 e1 3b addi r31,r1,64 0xc000000000520e48 <+56>: 30 00 c1 38 addi r6,r1,48 # 0xc000000000520e4c <+60>: 10 00 43 e9 ld r10,16(r3) 0xc000000000520e50 <+64>: 20 00 a9 83 lwz r29,32(r9) 0xc000000000520e54 <+68>: 20 00 a1 38 addi r5,r1,32 0xc000000000520e58 <+72>: 40 00 e1 fb std r31,64(r1) 0xc000000000520e5c <+76>: 48 00 e1 fb std r31,72(r1) 0xc000000000520e60 <+80>: 30 00 c1 f8 std r6,48(r1) 0xc000000000520e64 <+84>: 38 00 c1 f8 std r6,56(r1) 0xc000000000520e68 <+88>: 20 00 a1 f8 std r5,32(r1) 0xc000000000520e6c <+92>: 28 00 a1 f8 std r5,40(r1) 0xc000000000520e70 <+96>: 10 00 03 38 addi r0,r3,16 & 0xc000000000520e74 <+100>: 40 50 a0 7f cmpld cr7,r0,r10 0xc000000000520e78 <+104>: 78 53 47 7d mr r7,r10 * 0xc000000000520e7c <+108>: 00 00 0a e9 ld r8,0(r10) 0xc000000000520e80 <+112>: 64 00 9e 41 beq cr7,0xc000000000520ee4 <crypto_remove_spawns+212> #) This looks like alg->cra_users.next is loaded to r10 &) This looks like r10 is compared with &alg->cra_users calculated on the line above to terminate the loop *) This looks like *alg->cra_users.next loaded into r8 which causes the null pointer dereference So the fixup needs to be applied to the first dereference of alg->cra_users.next as well to prevent crash. Fixes: 9a00674213a3 ("crypto: algapi - fix NULL dereference in crypto_remove_spawns()") Reported-by: chetjain@in.ibm.com Signed-off-by: Michal Suchanek <msuchanek@suse.de> --- I cannot really test if this fix is effective because the crash is some heisenbug that heavily depends on timing. When the bug is not triggered it does not really mean anything. It is also qestionable if we should be getting these algs with uninitialized spawns. crypto/algapi.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)