Page MenuHomePhabricator

Kernel Oops running VyOS 1.2
Closed, ResolvedPublicBUG

Description

Running VyOS on AWS and we encountered a kernel Oops and a reboot. Please let me know if you need any more information

show version

Version:          VyOS 1.2.0
Built by:         Sentrium S.L.
Built on:         Sun 27 Jan 2019 19:08 UTC
Build ID:         795d6338-c1ce-4ebb-992f-d064f5af9309

Architecture:     x86_64
Boot via:         installed image
System type:      Xen HVM guest

Hardware vendor:  Xen
Hardware model:   HVM domU
Hardware S/N:     Unknown
Hardware UUID:    Unknown

Copyright:        VyOS maintainers and contributors

Kernel Panic

[612890.055958] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038
[612890.060532] PGD 80000001e0fca067 P4D 80000001e0fca067 PUD 1dc3ed067 PMD 0 
[612890.064112] Oops: 0000 [#1] SMP PTI
[612890.065973] CPU: 0 PID: 1717 Comm: rs:main Q:Reg Not tainted 4.19.12-amd64-vyos #1
[612890.069521] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
[612890.072719] RIP: 0010:0xffffffffc0475737
[612890.074747] Code: 77 10 8b 08 8b 45 70 0f c8 89 87 b4 00 00 00 75 11 48 8b 86 e8 04 00 00 8b 80 78 13 00 00 85 c0 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40 38 04 74 68 44 89 ab b4 00 00 00 48 8b 45 20 48 89 df 48 39
[612890.083936] RSP: 0000:ffff97a228003b90 EFLAGS: 00010246
[612890.086681] RAX: 0000000000000000 RBX: ffff97a21953c500 RCX: 0000000000000002
[612890.090243] RDX: ffff97a224986e80 RSI: ffff97a2269f9000 RDI: ffff97a21953c500
[612890.093741] RBP: ffff97a2194c88c0 R08: ffff97a21f1b2d00 R09: ffffffffc048d119
[612890.097263] R10: 000896fb435c2406 R11: 240642ee86126f06 R12: ffff97a2194c8000
[612890.100868] R13: 0000000000000000 R14: 0000000000000004 R15: ffff97a21953c500
[612890.104412] FS:  00007f1687352700(0000) GS:ffff97a228000000(0000) knlGS:0000000000000000
[612890.108368] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[612890.111176] CR2: 0000000000000038 CR3: 00000001d8c36006 CR4: 00000000001606f0
[612890.114665] Call Trace:
[612890.115973]  <IRQ>
[612890.116997]  0xffffffff87b49c71
[612890.118628]  0xffffffff87b56a4e
[612890.120198]  0xffffffff87b4a105
[612890.121764]  ? 0xffffffff87b49770
[612890.123446]  0xffffffff87b22778
[612890.125103]  0xffffffff87b229c1
[612890.126661]  0xffffffff87b234ff
[612890.128178]  0xffffffff87aedfea
[612890.129739]  0xffffffff87aee5e6
[612890.131270]  ? 0xffffffff87aedf50
[612890.132942]  0xffffffff87aee6ad
[612890.134490]  ? 0xffffffff87aee120
[612890.136132]  0xffffffff87aa28bd
[612890.137610]  0xffffffff87aa1baf
[612890.139131]  0xffffffff87aa397c
[612890.140713]  0xffffffffc014f52f
[612890.142251]  0xffffffff87aa319e
[612890.143860]  0xffffffff87e000fb
[612890.145450]  0xffffffff8766a123
[612890.146987]  0xffffffff87990d97
[612890.148552]  0xffffffff87c00ecf
[612890.150100]  </IRQ>
[612890.151158] RIP: 0033:0x0000000000433280
[612890.153165] Code: 48 8b 75 08 48 8d 3d ef 6e 03 00 31 c0 e8 78 76 00 00 e9 fc fe ff ff e8 5e 97 fd ff 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 <41> 57 41 56 49 89 ce 41 55 41 54 31 c9 55 53 49 89 fc 48 89 f3 4c
[612890.161972] RSP: 002b:00007f1687350ef8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff0c
[612890.165773] RAX: 0000000000000011 RBX: 00000000009ff340 RCX: 00007f1687350f60
[612890.169158] RDX: 00000000009ff350 RSI: 0000000000000000 RDI: 00007f1680004290
[612890.172531] RBP: 00007f1687351020 R08: 00007f1687350f50 R09: 0000000000000000
[612890.175982] R10: 0000000000000000 R11: 0000000000000006 R12: 000000000068cf8c
[612890.179343] R13: 00007f1687351020 R14: 0000000000a02e80 R15: 00007f1680004290
[612890.182662] Modules linked in: binfmt_misc drbg ansi_cprng authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel twofish_generic twofish_avx_x86_64 twofish_x86_64_3way twofish_x86_64 twofish_common serpent_avx2 serpent_avx_x86_64 serpent_sse2_x86_64 serpent_generic blowfish_generic blowfish_x86_64 blowfish_common cast5_avx_x86_64 cast5_generic cast_common ctr ecb des_generic cbc algif_skcipher camellia_generic camellia_aesni_avx2 camellia_aesni_avx_x86_64 camellia_x86_64 xcbc sha512_ssse3 sha512_generic md4 algif_hash af_alg xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key xfrm_algo ip_vti ip_tunnel dummy xt_nat ipt_MASQUERADE xt_comment iptable_nat nf_nat_ipv4 ip6table_mangle ip6table_filter ip6table_raw ip6_tables iptable_filter xt_tcpudp xt_CT nfnetlink_cthelper nfnetlink iptable_raw
[612890.214923]  ip_tables x_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_h323 nf_conntrack_h323 nf_nat_sip nf_conntrack_sip nf_nat_proto_gre nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc32c_generic fuse sb_edac crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd glue_helper intel_rapl_perf button autofs4 usb_storage ohci_hcd ehci_hcd uhci_hcd sd_mod squashfs zstd_decompress xxhash loop overlay ext4 crc16 mbcache jbd2 md_mod nls_ascii ata_generic ata_piix ixgbevf xen_blkfront libata crc32c_intel scsi_mod i2c_piix4
[612890.241255] CR2: 0000000000000038
[612890.242857] ---[ end trace a72a770643543159 ]---
[612890.245061] RIP: 0010:0xffffffffc0475737
[612890.246964] Code: 77 10 8b 08 8b 45 70 0f c8 89 87 b4 00 00 00 75 11 48 8b 86 e8 04 00 00 8b 80 78 13 00 00 85 c0 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40 38 04 74 68 44 89 ab b4 00 00 00 48 8b 45 20 48 89 df 48 39
[612890.255463] RSP: 0000:ffff97a228003b90 EFLAGS: 00010246
[612890.257912] RAX: 0000000000000000 RBX: ffff97a21953c500 RCX: 0000000000000002
[612890.261158] RDX: ffff97a224986e80 RSI: ffff97a2269f9000 RDI: ffff97a21953c500
[612890.264394] RBP: ffff97a2194c88c0 R08: ffff97a21f1b2d00 R09: ffffffffc048d119
[612890.267740] R10: 000896fb435c2406 R11: 240642ee86126f06 R12: ffff97a2194c8000
[612890.271069] R13: 0000000000000000 R14: 0000000000000004 R15: ffff97a21953c500
[612890.274412] FS:  00007f1687352700(0000) GS:ffff97a228000000(0000) knlGS:0000000000000000
[612890.278167] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[612890.280860] CR2: 0000000000000038 CR3: 00000001d8c36006 CR4: 00000000001606f0
[612890.284123] Kernel panic - not syncing: Fatal exception in interrupt
[612890.287385] Kernel Offset: 0x6600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

uname -a

Linux [...]  4.19.12-amd64-vyos #1 SMP Tue Jan 22 01:09:06 CET 2019 x86_64 GNU/Linux

Details

Difficulty level
Unknown (require assessment)
Version
1.2
Why the issue appeared?
Will be filled on close

Event Timeline

breu created this task.Feb 28 2019, 9:27 PM
syncer added a subscriber: syncer.Feb 28 2019, 9:37 PM

can you trigger such panic or it happens randomly?

breu updated the task description. (Show Details)Feb 28 2019, 9:45 PM

@syncer so far it has happened randomly. We haven't identified any correlation between systems events yet.

Can you also tell region and instance type please

breu added a comment.Feb 28 2019, 9:55 PM

Instance type is c4.xlarge running in US-Gov-West. Let me know if you need anything else.

pasik added a subscriber: pasik.Mar 2 2019, 4:27 PM
syncer changed the task status from Open to Needs testing.Apr 17 2019, 7:55 PM
syncer triaged this task as Normal priority.

Please retest on 1.2.1

syncer closed this task as Resolved.Aug 31 2019, 8:56 PM
syncer claimed this task.