Page MenuHomeVyOS Platform

Kernel Oops running VyOS 1.2
Closed, ResolvedPublicBUG

Description

Running VyOS on AWS and we encountered a kernel Oops and a reboot. Please let me know if you need any more information

show version

Version:          VyOS 1.2.0
Built by:         Sentrium S.L.
Built on:         Sun 27 Jan 2019 19:08 UTC
Build ID:         795d6338-c1ce-4ebb-992f-d064f5af9309

Architecture:     x86_64
Boot via:         installed image
System type:      Xen HVM guest

Hardware vendor:  Xen
Hardware model:   HVM domU
Hardware S/N:     Unknown
Hardware UUID:    Unknown

Copyright:        VyOS maintainers and contributors

Kernel Panic

[612890.055958] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038
[612890.060532] PGD 80000001e0fca067 P4D 80000001e0fca067 PUD 1dc3ed067 PMD 0 
[612890.064112] Oops: 0000 [#1] SMP PTI
[612890.065973] CPU: 0 PID: 1717 Comm: rs:main Q:Reg Not tainted 4.19.12-amd64-vyos #1
[612890.069521] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
[612890.072719] RIP: 0010:0xffffffffc0475737
[612890.074747] Code: 77 10 8b 08 8b 45 70 0f c8 89 87 b4 00 00 00 75 11 48 8b 86 e8 04 00 00 8b 80 78 13 00 00 85 c0 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40 38 04 74 68 44 89 ab b4 00 00 00 48 8b 45 20 48 89 df 48 39
[612890.083936] RSP: 0000:ffff97a228003b90 EFLAGS: 00010246
[612890.086681] RAX: 0000000000000000 RBX: ffff97a21953c500 RCX: 0000000000000002
[612890.090243] RDX: ffff97a224986e80 RSI: ffff97a2269f9000 RDI: ffff97a21953c500
[612890.093741] RBP: ffff97a2194c88c0 R08: ffff97a21f1b2d00 R09: ffffffffc048d119
[612890.097263] R10: 000896fb435c2406 R11: 240642ee86126f06 R12: ffff97a2194c8000
[612890.100868] R13: 0000000000000000 R14: 0000000000000004 R15: ffff97a21953c500
[612890.104412] FS:  00007f1687352700(0000) GS:ffff97a228000000(0000) knlGS:0000000000000000
[612890.108368] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[612890.111176] CR2: 0000000000000038 CR3: 00000001d8c36006 CR4: 00000000001606f0
[612890.114665] Call Trace:
[612890.115973]  <IRQ>
[612890.116997]  0xffffffff87b49c71
[612890.118628]  0xffffffff87b56a4e
[612890.120198]  0xffffffff87b4a105
[612890.121764]  ? 0xffffffff87b49770
[612890.123446]  0xffffffff87b22778
[612890.125103]  0xffffffff87b229c1
[612890.126661]  0xffffffff87b234ff
[612890.128178]  0xffffffff87aedfea
[612890.129739]  0xffffffff87aee5e6
[612890.131270]  ? 0xffffffff87aedf50
[612890.132942]  0xffffffff87aee6ad
[612890.134490]  ? 0xffffffff87aee120
[612890.136132]  0xffffffff87aa28bd
[612890.137610]  0xffffffff87aa1baf
[612890.139131]  0xffffffff87aa397c
[612890.140713]  0xffffffffc014f52f
[612890.142251]  0xffffffff87aa319e
[612890.143860]  0xffffffff87e000fb
[612890.145450]  0xffffffff8766a123
[612890.146987]  0xffffffff87990d97
[612890.148552]  0xffffffff87c00ecf
[612890.150100]  </IRQ>
[612890.151158] RIP: 0033:0x0000000000433280
[612890.153165] Code: 48 8b 75 08 48 8d 3d ef 6e 03 00 31 c0 e8 78 76 00 00 e9 fc fe ff ff e8 5e 97 fd ff 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 <41> 57 41 56 49 89 ce 41 55 41 54 31 c9 55 53 49 89 fc 48 89 f3 4c
[612890.161972] RSP: 002b:00007f1687350ef8 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff0c
[612890.165773] RAX: 0000000000000011 RBX: 00000000009ff340 RCX: 00007f1687350f60
[612890.169158] RDX: 00000000009ff350 RSI: 0000000000000000 RDI: 00007f1680004290
[612890.172531] RBP: 00007f1687351020 R08: 00007f1687350f50 R09: 0000000000000000
[612890.175982] R10: 0000000000000000 R11: 0000000000000006 R12: 000000000068cf8c
[612890.179343] R13: 00007f1687351020 R14: 0000000000a02e80 R15: 00007f1680004290
[612890.182662] Modules linked in: binfmt_misc drbg ansi_cprng authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel twofish_generic twofish_avx_x86_64 twofish_x86_64_3way twofish_x86_64 twofish_common serpent_avx2 serpent_avx_x86_64 serpent_sse2_x86_64 serpent_generic blowfish_generic blowfish_x86_64 blowfish_common cast5_avx_x86_64 cast5_generic cast_common ctr ecb des_generic cbc algif_skcipher camellia_generic camellia_aesni_avx2 camellia_aesni_avx_x86_64 camellia_x86_64 xcbc sha512_ssse3 sha512_generic md4 algif_hash af_alg xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key xfrm_algo ip_vti ip_tunnel dummy xt_nat ipt_MASQUERADE xt_comment iptable_nat nf_nat_ipv4 ip6table_mangle ip6table_filter ip6table_raw ip6_tables iptable_filter xt_tcpudp xt_CT nfnetlink_cthelper nfnetlink iptable_raw
[612890.214923]  ip_tables x_tables nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_h323 nf_conntrack_h323 nf_nat_sip nf_conntrack_sip nf_nat_proto_gre nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc32c_generic fuse sb_edac crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd cryptd glue_helper intel_rapl_perf button autofs4 usb_storage ohci_hcd ehci_hcd uhci_hcd sd_mod squashfs zstd_decompress xxhash loop overlay ext4 crc16 mbcache jbd2 md_mod nls_ascii ata_generic ata_piix ixgbevf xen_blkfront libata crc32c_intel scsi_mod i2c_piix4
[612890.241255] CR2: 0000000000000038
[612890.242857] ---[ end trace a72a770643543159 ]---
[612890.245061] RIP: 0010:0xffffffffc0475737
[612890.246964] Code: 77 10 8b 08 8b 45 70 0f c8 89 87 b4 00 00 00 75 11 48 8b 86 e8 04 00 00 8b 80 78 13 00 00 85 c0 74 0e 48 8b 43 58 48 83 e0 fe <f6> 40 38 04 74 68 44 89 ab b4 00 00 00 48 8b 45 20 48 89 df 48 39
[612890.255463] RSP: 0000:ffff97a228003b90 EFLAGS: 00010246
[612890.257912] RAX: 0000000000000000 RBX: ffff97a21953c500 RCX: 0000000000000002
[612890.261158] RDX: ffff97a224986e80 RSI: ffff97a2269f9000 RDI: ffff97a21953c500
[612890.264394] RBP: ffff97a2194c88c0 R08: ffff97a21f1b2d00 R09: ffffffffc048d119
[612890.267740] R10: 000896fb435c2406 R11: 240642ee86126f06 R12: ffff97a2194c8000
[612890.271069] R13: 0000000000000000 R14: 0000000000000004 R15: ffff97a21953c500
[612890.274412] FS:  00007f1687352700(0000) GS:ffff97a228000000(0000) knlGS:0000000000000000
[612890.278167] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[612890.280860] CR2: 0000000000000038 CR3: 00000001d8c36006 CR4: 00000000001606f0
[612890.284123] Kernel panic - not syncing: Fatal exception in interrupt
[612890.287385] Kernel Offset: 0x6600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

uname -a

Linux [...]  4.19.12-amd64-vyos #1 SMP Tue Jan 22 01:09:06 CET 2019 x86_64 GNU/Linux

Details

Difficulty level
Unknown (require assessment)
Version
1.2
Why the issue appeared?
Will be filled on close

Event Timeline

can you trigger such panic or it happens randomly?

@syncer so far it has happened randomly. We haven't identified any correlation between systems events yet.

Can you also tell region and instance type please

Instance type is c4.xlarge running in US-Gov-West. Let me know if you need anything else.

syncer changed the task status from Open to Needs testing.Apr 17 2019, 7:55 PM
syncer triaged this task as Normal priority.

Please retest on 1.2.1