Bug #421
More unaligned access traps in CeroWRT 3.7.2
Closed
Normal
Dave Täht
Description
After running the CeroWRT 3.7.2-3 build overnight with unaligned_action=2, I found two new unaligned access traps that were causing problems.The first is in tcp_v6_syn_recv_sock, and appears to be tclass-related:
[74530.100000] Cpu 0
[74530.100000] $ 0 : 00000000 00000001 86b5f83e 00000000
[74530.100000] $ 4 : 85c83480 000005dc 00000000 00000000
[74530.110000] $ 8 : 00000000 00000000 0000ffff ac1e2a01
[74530.110000] $12 : 00000000 00000000 00000000 00000000
[74530.120000] $16 : 85c83910 85c83480 00000000 86b6a840
[74530.130000] $20 : 00000000 85c81500 85fae240 85c81990
[74530.130000] $24 : 00000000 80232e24
[74530.140000] $28 : 80324000 80325b48 8033e190 86ca5d98
[74530.140000] Hi : 00000000
[74530.140000] Lo : 00000000
[74530.150000] epc : 86ca5de8 tcp_v6_syn_recv_sock+0x11c/0x468 [ipv6]
[74530.150000] Tainted: G O
[74530.160000] ra : 86ca5d98 tcp_v6_syn_recv_sock+0xcc/0x468 [ipv6]
[74530.160000] Status: 1000fc03 KERNEL EXL IE
[74530.170000] Cause : 00800010
[74530.170000] BadVA : 86b5f83e
[74530.170000] PrId : 00019374 (MIPS 24Kc)
[74530.180000] Modules linked in: sch_teql sch_tbf sch_sfq sch_red sch_qfq sch_prio sch_ns2_codel sch_nfq_codel sch_netem sch_htb sch_gred sch_efq_codel sch_dsmark em_text em_nbyte em_meta em_cmp cls_basic act_police act_ipt act_connmark act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_fq_codel sch_codel sch_ingress usb_storage ath79_wdt ohci_hcd xt_hashlimit xt_set(O) ip_set_list_set(O) ip_set_hash_netport(O) ip_set_hash_netiface(O) ip_set_hash_net(O) ip_set_hash_ipportnet(O) ip_set_hash_ipportip(O) ip_set_hash_ipport(O) ip_set_hash_ip(O) ip_set_bitmap_port(O) ip_set_bitmap_ipmac(O) ip_set_bitmap_ip(O) ip_set(O) ip6t_REJECT ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6table_mangle ip6table_filter ip6_tables nf_conntrack_ipv6 nf_defrag_ipv6 nfnetlink nf_nat_irc nf_nat_ftp nf_conntrack_irc nf_conntrack_ftp xt_policy xt_esp ipt_ah xt_HL xt_hl xt_ecn ipt_ECN xt_CLASSIFY xt_time xt_tcpmss xt_statistic xt_mark xt_length xt_DSCP xt_dscp xt_string xt_layer7 xt_quota xt_pkttype xt_physdev xt_owner xt_addrtype ipt_MASQUERADE iptable_nat xt_nat nf_nat_ipv4 nf_nat xt_recent xt_helper xt_connmark xt_connbytes pptp pppoe xt_conntrack xt_CT iptable_raw xt_state nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ehci_hcd sd_mod pppox ipt_REJECT xt_TCPMSS xt_LOG xt_comment xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables ip_gre gre ifb sit ipcomp xfrm4_tunnel xfrm4_mode_tunnel xfrm4_mode_transport xfrm4_mode_beet esp4 ah4 tunnel4 tun tcp_ledbat(O) ppp_async ppp_generic slhc af_key xfrm_user xfrm_ipcomp xfrm_algo vfat fat ext4 jbd2 mbcache autofs4 button_hotplug(O) ath9k(O) ath9k_common(O) ath9k_hw(O) ath(O) nls_utf8 nls_iso8859_2 nls_iso8859_15 nls_iso8859_13 nls_iso8859_1 nls_cp437 mac80211(O) usbcore usb_common scsi_mod nls_base ts_fsm ts_bm ts_kmp crc16 crc_ccitt ipv6 input_polldev cfg80211(O) compat(O) input_core sha1_generic md5 hmac des_generic deflate cbc authenc arc4 zlib_inflate zlib_deflate ledtrig_timer ledtrig_default_on leds_gpio gpio_button_hotplug(O)
[74530.360000] Process swapper (pid: 0, threadinfo=80324000, task=80328d50, tls=00000000)
[74530.370000] Stack : 00000000 00000002 2a67d6a1 86912228 00000000 00000000 00000014 00000000
[74530.370000] 00000002 00000001 85d213a0 869671b8 06410002 00000014 00000000 00000000
[74530.370000] 00000000 00000001 00000001 86b5f83e 85fae240 85fae240 85c81500 86b6a840
[74530.370000] 00100000 00000000 00000000 00000000 85c7cd54 80255228 8781a000 00000000
[74530.370000] 80325bd8 00000014 00000000 02000004 86c39824 50f7ee85 0306e65e 0306e65e
[74530.370000] ...
[74530.410000] Call Trace:
[74530.410000] [<86ca5de8>] tcp_v6_syn_recv_sock+0x11c/0x468 [ipv6]
[74530.420000] [<80255228>] tcp_check_req+0x330/0x4cc
[74530.420000] [<8025219c>] tcp_v4_do_rcv+0x12c/0x274
[74530.430000] [<802547ec>] tcp_v4_rcv+0x5b4/0x974
[74530.430000] [<80232bf0>] ip_local_deliver_finish+0x168/0x29c
[74530.440000] [<80207c00>] __netif_receive_skb+0x63c/0x6c0
[74530.440000] [<801ef7e0>] ag71xx_poll+0x400/0x600
[74530.450000] [<802080c8>] net_rx_action+0x88/0x1fc
[74530.450000] [<80078ad8>] __do_softirq+0xb0/0x184
[74530.460000] [<80078c5c>] do_softirq+0x48/0x68
[74530.460000] [<80078e78>] irq_exit+0x4c/0x7c
[74530.460000] [<8006316c>] ret_from_irq+0x0/0x4
[74530.470000] [<80063380>] r4k_wait+0x20/0x40
[74530.470000] [<80064e60>] cpu_idle+0x30/0x60
[74530.480000] [<803428fc>] start_kernel+0x374/0x394
[74530.480000]
[74530.480000]
[74530.480000] Code: 7c6279c4 ae2204e4 8e620094 <8c420000> 00021502 0c0931ec a22204f6 09b29842 8fbf0074
The second is in xfrm_state_lookup:
[74929.640000] Cpu 0
[74929.650000] $ 0 : 00000000 00000000 984ebd38 00000007
[74929.650000] $ 4 : 8033c640 00000002 86b4684e 984ebd3c
[74929.660000] $ 8 : 87bc587c 00000000 00000000 00000000
[74929.660000] $12 : 00000002 00000000 00000000 5169fd51
[74929.670000] $16 : 00000004 86b4684e 00000002 984ebd3c
[74929.670000] $20 : 86b4684e 00000004 00000000 80340000
[74929.680000] $24 : 00000000 86915620
[74929.680000] $28 : 80324000 80325bd8 8033d570 8027af3c
[74929.690000] Hi : 00000063
[74929.690000] Lo : dabc6800
[74929.690000] epc : 8027ade0 __xfrm_state_lookup+0x60/0x164
[74929.700000] Tainted: G O
[74929.700000] ra : 8027af3c xfrm_state_lookup+0x58/0x88
[74929.710000] Status: 1000fc03 KERNEL EXL IE
[74929.710000] Cause : 80800010
[74929.720000] BadVA : 86b4684e
[74929.720000] PrId : 00019374 (MIPS 24Kc)
[74929.720000] Modules linked in: sch_teql sch_tbf sch_sfq sch_red sch_qfq sch_prio sch_ns2_codel sch_nfq_codel sch_netem sch_htb sch_gred sch_efq_codel sch_dsmark em_text em_nbyte em_meta em_cmp cls_basic act_police act_ipt act_connmark act_skbedit act_mirred em_u32 cls_u32 cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_fq_codel sch_codel sch_ingress usb_storage ath79_wdt ohci_hcd xt_hashlimit xt_set(O) ip_set_list_set(O) ip_set_hash_netport(O) ip_set_hash_netiface(O) ip_set_hash_net(O) ip_set_hash_ipportnet(O) ip_set_hash_ipportip(O) ip_set_hash_ipport(O) ip_set_hash_ip(O) ip_set_bitmap_port(O) ip_set_bitmap_ipmac(O) ip_set_bitmap_ip(O) ip_set(O) ip6t_REJECT ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6table_mangle ip6table_filter ip6_tables nf_conntrack_ipv6 nf_defrag_ipv6 nfnetlink nf_nat_irc nf_nat_ftp nf_conntrack_irc nf_conntrack_ftp xt_policy xt_esp ipt_ah xt_HL xt_hl xt_ecn ipt_ECN xt_CLASSIFY xt_time xt_tcpmss xt_statistic xt_mark xt_length xt_DSCP xt_dscp xt_string xt_layer7 xt_quota xt_pkttype xt_physdev xt_owner xt_addrtype ipt_MASQUERADE iptable_nat xt_nat nf_nat_ipv4 nf_nat xt_recent xt_helper xt_connmark xt_connbytes pptp pppoe xt_conntrack xt_CT iptable_raw xt_state nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ehci_hcd sd_mod pppox ipt_REJECT xt_TCPMSS xt_LOG xt_comment xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables ip_gre gre ifb sit ipcomp xfrm4_tunnel xfrm4_mode_tunnel xfrm4_mode_transport xfrm4_mode_beet esp4 ah4 tunnel4 tun tcp_ledbat(O) ppp_async ppp_generic slhc af_key xfrm_user xfrm_ipcomp xfrm_algo vfat fat ext4 jbd2 mbcache autofs4 button_hotplug(O) ath9k(O) ath9k_common(O) ath9k_hw(O) ath(O) nls_utf8 nls_iso8859_2 nls_iso8859_15 nls_iso8859_13 nls_iso8859_1 nls_cp437 mac80211(O) usbcore usb_common scsi_mod nls_base ts_fsm ts_bm ts_kmp crc16 crc_ccitt ipv6 input_polldev cfg80211(O) compat(O) input_core sha1_generic md5 hmac des_generic deflate cbc authenc arc4 zlib_inflate zlib_deflate ledtrig_timer ledtrig_default_on leds_gpio gpio_button_hotplug(O)
[74929.910000] Process swapper (pid: 0, threadinfo=80324000, task=80328d50, tls=00000000)
[74929.910000] Stack : 85d14ae0 86910f18 00000001 85c26700 00000000 00000004 00000002 8033c640
[74929.910000] 00000000 86b4684e 984ebd3c 00000002 80340000 8027af3c 00000000 8022d254
[74929.910000] 86c2e900 00000000 00000004 00000002 80232a88 86b4684e 86c2e900 00000004
[74929.910000] 00000000 00000000 fffffffe 8027dce8 00000002 80327dd8 00000000 80232a88
[74929.910000] 00000004 00000002 00000000 80325c74 86c2e900 86ffe2a0 00000001 803281dc
[74929.910000] ...
[74929.950000] Call Trace:
[74929.950000] [<8027ade0>] __xfrm_state_lookup+0x60/0x164
[74929.960000] [<8027af3c>] xfrm_state_lookup+0x58/0x88
[74929.960000] [<8027dce8>] xfrm_input+0x134/0x480
[74929.970000] [<86cfd1cc>] tunnel64_rcv+0x60/0xa4 [tunnel4]
[74929.970000] [<80232bf0>] ip_local_deliver_finish+0x168/0x29c
[74929.980000] [<80207c00>] __netif_receive_skb+0x63c/0x6c0
[74929.990000] [<801ef7e0>] ag71xx_poll+0x400/0x600
[74929.990000] [<802080c8>] net_rx_action+0x88/0x1fc
[74929.990000] [<80078ad8>] __do_softirq+0xb0/0x184
[74930.000000] [<80078c5c>] do_softirq+0x48/0x68
[74930.000000] [<80078e78>] irq_exit+0x4c/0x7c
[74930.010000] [<8006316c>] ret_from_irq+0x0/0x4
[74930.010000] [<80063380>] r4k_wait+0x20/0x40
[74930.020000] [<80064e60>] cpu_idle+0x30/0x60
[74930.020000] [<803428fc>] start_kernel+0x374/0x394
[74930.030000]
[74930.030000]
[74930.030000] Code: 00000000 0809eb7a 8cc6000c <0809eb7c> 8cc50000 8e250008 00c52826 00451026 00023282
I think this last bug stems from the use of a union “xfrm_address_t”, which avoids all our unaligned access marking.
Attachments
History
Updated by Robert Bradley on Jan 17, 2013.
Patches for the two traps above, plus a patch to __ipv6_neigh_lookup
(similar to that in #371). The traps are pretty rare though even
without the patches included in 3.7.2-4 (155 in 24 hours).
Updated by Robert Bradley on Jan 17, 2013.
Updated by Robert Bradley on Jan 18, 2013.
Apparently there’s another trap in secure_tcpv6_sequence_number I
missed seeing yesterday:
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.730000] Cpu 0
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.730000] $ 0 : 00000000 00000001 0000000c 803aa160
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.740000] $ 4 : 00000010 85f15a64 803aa16c 86b57852
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.740000] $ 8 : 20010470 6aac0001 00000000 00000001
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.750000] $12 : 85f24960 00070011 00000000 00000000
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.750000] $16 : 86b57846 00000035 0000ceb5 85c81f80
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.760000] $20 : 00000000 86d4f2d0 00000000 85c82410
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.760000] $24 : 00000000 86c864e0
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.770000] $28 : 85f14000 85f15a38 8033e190 80206294
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.770000] Hi : 0000045c
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.780000] Lo : 61942800
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.780000] epc : 802062b4 secure_tcpv6_sequence_number+0x54/0x110
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.780000] Tainted: G O
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.790000] ra : 80206294 secure_tcpv6_sequence_number+0x34/0x110
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.800000] Status: 1000fc03 KERNEL EXL IE
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.800000] Cause : 00800010
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.800000] BadVA : 86b57852
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.810000] PrId : 00019374 (MIPS 24Kc)
Jan 18 13:03:49 OpenWrt kern.warn kernel: [truncated] [163026.810000] Modules linked in: sch_teql sch_tbf sch_sfq sch_red sch_qfq sch_prio sch_ns2_codel sch_nfq_codel sch_netem sch_htb sch_gred sch_efq_codel sch_dsmark em_text em_nbyte em_meta em_cmp cls_basic act_polic
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163026.990000] Process dnsmasq (pid: 4239, threadinfo=85f14000, task=85e748c8, tls=777eb440)
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.000000] Stack : 00000000 84c2da64 803a3b60 8007d738 20010470 6aac0001 00000000 00000001
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.000000] ce9b8eff 40e6ef00 0dc90d0d 84c2d9f8 00000000 00000001 86d5a9c0 86911720
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.000000] 8781a000 8793ef00 00000001 85f15b60 86cb9244 86d4f280 86d5a9c0 85c81f80
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.000000] 00000000 86b5783e 00000000 86d4f280 86d5a9c0 86ca6a54 00000001 00000024
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.000000] 85f15ac8 8793ef00 00000000 8020c7a0 80327fe8 80327fe8 87ba408c 038e70c8
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.000000] ...
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.040000] Call Trace:
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.040000] [<802062b4>] secure_tcpv6_sequence_number+0x54/0x110
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.050000] [<86ca6a54>] tcp_v6_conn_request+0x5b4/0x66c [ipv6]
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.050000] [<8024a7b0>] tcp_rcv_state_process+0xa8/0xc28
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.060000] [<86ca7f34>] tcp_v6_do_rcv+0x264/0x410 [ipv6]
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.070000] [<86ca8758>] tcp_v6_rcv+0x638/0x9d4 [ipv6]
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.070000] [<86c85d90>] ip6_input_finish+0x250/0x458 [ipv6]
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.080000] [<80207c00>] __netif_receive_skb+0x63c/0x6c0
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.080000] [<801ef7e0>] ag71xx_poll+0x400/0x600
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.090000] [<802080c8>] net_rx_action+0x88/0x1fc
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.090000] [<80078ad8>] __do_softirq+0xb0/0x184
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.100000] [<80078c5c>] do_softirq+0x48/0x68
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.100000] [<80078e78>] irq_exit+0x4c/0x7c
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.110000] [<8006316c>] ret_from_irq+0x0/0x4
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.110000] [<80076e8c>] do_exit+0x5c0/0x670
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.110000] [<80077294>] do_group_exit+0x7c/0xa8
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.120000] [<800772d4>] __wake_up_parent+0x0/0x18
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.120000]
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.130000]
Jan 18 13:03:49 OpenWrt kern.warn kernel: [163027.130000] Code: 00c22821 02023821 00623021 <8ce70000> 8cc60000 24420004 00e63021 1444fff7 aca60000
All these traps only seem to occur with TCP connections to the router over IPv6. TCP connections to the router over IPv4 are unaffected, as are forwarded TCP packets over either protocol. I should have a suitable patch available shortly.
Updated by Robert Bradley on Jan 18, 2013.
Patches for the new trap are now complete. The
tcp_v6_syn_recv_sock_v2.patch covers all three traps, along with an
additional patch for #419 (I packed the ICMPv6 RA header and aligned to
16-bit boundaries).
Updated by Robert Bradley on Jan 19, 2013.
“Assigning” this to Dave Taht since I cannot figure out the right way to
add other people as watchers.
Updated by David Taht on Jan 21, 2013.
———- Forwarded message ———-From: Dave Taht dave.taht@gmail.com
Date: Mon, Jan 21, 2013 at 4:17 PM
Subject: [bug #421]
To: cerowrt-devel@lists.bufferbloat.net
I ran a prodigous amount of benchmarks today…
they aint all dead yet. I plan to run the thc toolkit shortly.
[12963.671875] Process swapper (pid: 0, threadinfo=8032a000, task=8032f030,
tls=00000000)
[12963.679687] Stack : 80095f54 00000000 8033b4d0 00000029 00000001
00000001 00000080 80340000
[12963.679687] 00000006 8381a800 00000000 8321bc00 8381a800
00000000 8381a800 00000001
[12963.679687] 00000004 00000000 83245f98 82aba07c 00000005
00000001 803a0000 803a0000
[12963.679687] 00200000 800ab560 803a5428 8032bca4 8321bc00
8032dfd0 00000000 8022ea74
[12963.679687] 8032bcd8 00000000 00000000 832b84ec 83245f98
800aac3c 82abae60 00000002
[12963.679687] ...
[12963.714843] Call Trace:
[12963.718750] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c [nf_defrag_ipv6]
[12963.726562] [<82aba07c>] ipv6_defrag+0x74/0xc8 [nf_defrag_ipv6]
[12963.730468] [<8022ea74>] nf_iterate+0x98/0x104
[12963.734375] [<8022eb78>] nf_hook_slow+0x98/0x170
[12963.742187] [<8324643c>] ipv6_rcv+0x3fc/0x4a0 [ipv6]
[12963.746093] [<80209410>] __netif_receive_skb+0x63c/0x6c0
[12963.750000] [<8298f384>] ri_tasklet+0x104/0x2b0 [ifb]
[12963.757812] [<80079434>] tasklet_action+0x78/0xc8
[12963.761718] [<80078c48>] __do_softirq+0xb0/0x184
[12963.765625] [<80078dcc>] do_softirq+0x48/0x68
[12963.769531] [<80078fe8>] irq_exit+0x4c/0xb8
[12963.773437] [<8006316c>] ret_from_irq+0x0/0x4
[12963.777343] [<80063380>] r4k_wait+0x20/0x40
[12963.781250] [<80064e68>] cpu_idle+0x38/0x70
[12963.785156] [<803488fc>] start_kernel+0x374/0x394
[12963.792968]
[12963.792968]
[12963.792968] Code: 00003021 0c01e3a4 26160008 <8eb40004> 8fa20054
26130018 26f0b0a0 afa20014 afb60018
[13027.007812] Cpu 0
[13027.007812] $ 0 : 00000000 00000000 803a0000 00000100
[13027.007812] $ 4 : 80342e38 82abb0a0 00000000 01820d57
[13027.007812] $ 8 : 00000000 00000000 00000000 00000000
[13027.027343] $12 : 00060000 0014002b 00000000 00000000
[13027.031250] $16 : 8087283e 82007240 821fb240 000004d8
[13027.035156] $20 : 0000002c 80872866 80872846 82ac0000
[13027.042968] $24 : 00000000 82a00b20
[13027.046875] $28 : 8032a000 8032bbb8 80340000 82aba334
[13027.050781] Hi : 0000001b
[13027.054687] Lo : 00006c64
[13027.058593] epc : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.066406] Tainted: G O
[13027.070312] ra : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.078125] Status: 1000fc03 KERNEL EXL IE
[13027.082031] Cause : 00800010
[13027.082031] BadVA : 8087286a
[13027.085937] PrId : 00019374 (MIPS 24Kc)
[13027.089843] Modules linked in: sch_teql sch_tbf sch_sfq sch_red sch_qfq
sch_prio sch_ns2_codel sch_nfq_codel sch_netem sch_htb sch_gred
sch_efq_codel sch_dsmark em_text em_nbyte em_meta em_cmp cls_basic
act_police act_ipt act_connmark act_skbedit act_mirred em_u32 cls_u32
cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_fq_codel sch_codel
sch_ingress ath79_wdt xt_hashlimit xt_set(O) ip_set_list_set(O)
ip_set_hash_netport(O) ip_set_hash_netiface(O) ip_set_hash_net(O)
ip_set_hash_ipportnet(O) ip_set_hash_ipportip(O) ip_set_hash_ipport(O)
ip_set_hash_ip(O) ip_set_bitmap_port(O) ip_set_bitmap_ipmac(O)
ip_set_bitmap_ip(O) ip_set(O) ip6t_REJECT ip6t_rt ip6t_hbh ip6t_mh
ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6table_mangle
ip6table_filter ip6_tables nf_conntrack_ipv6 nf_defrag_ipv6 nfnetlink
nf_nat_irc nf_nat_ftp nf_conntrack_irc nf_conntrack_ftp xt_policy xt_esp
ipt_ah xt_HL xt_hl xt_ecn ipt_ECN xt_CLASSIFY xt_time xt_tcpmss
xt_statistic xt_mark xt_length xt_DSCP xt_dscp xt_string xt_layer7 xt_quota
xt_pkttype xt_physdev xt_owner xt_addrtype ipt_MASQUERADE iptable_nat
xt_nat nf_nat_ipv4 nf_nat xt_recent xt_helper xt_connmark xt_connbytes pptp
pppoe xt_conntrack xt_CT iptable_raw xt_state nf_conntrack_ipv4
nf_defrag_ipv4 nf_conntrack pppox ipt_REJECT xt_TCPMSS xt_LOG xt_comment
xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables
xt_tcpudp x_tables ip_gre gre ifb sit ipcomp xfrm4_tunnel xfrm4_mode_tunnel
xfrm4_mode_transport xfrm4_mode_beet esp4 ah4 tunnel4 tun tcp_ledbat(O)
ppp_async ppp_generic slhc af_key xfrm_user xfrm_ipcomp xfrm_algo vfat fat
autofs4 button_hotplug(O) ath9k(O) ath9k_common(O) ath9k_hw(O) ath(O)
nls_utf8 nls_iso8859_2 nls_iso8859_15 nls_iso8859_13 nls_iso8859_1
nls_cp437 mac80211(O) ts_fsm ts_bm ts_kmp crc_ccitt ipv6 input_polldev
cfg80211(O) compat(O) input_core sha1_generic md5 hmac des_generic deflate
cbc authenc arc4 usb_storage ohci_hcd ehci_hcd sd_mod ext4 jbd2 mbcache
usbcore usb_common scsi_mod nls_base crc16 zlib_inflate zlib_deflate
ledtrig_timer ledtrig_default_on leds_gpio gpio_button_hotplug(O)
[13027.273437] Process swapper (pid: 0, threadinfo=8032a000, task=8032f030,
tls=00000000)
[13027.281250] Stack : 8381a800 00000000 000086dd 80340000 80343870
800ae424 00000001 82b54e78
[13027.281250] 00000006 8381a800 00000000 821fb240 8381a800
00000000 8381a800 00000001
[13027.281250] 00000004 00000000 83245f98 82aba07c 8032be18
00000001 831f0400 80340000
[13027.281250] 8381a800 8006316c 80233f00 8032bca4 821fb240
8032dfd0 00000000 8022ea74
[13027.281250] 00000000 00000000 81084078 00400000 83245f98
8381a800 82abae60 8381a800
[13027.281250] ...
[13027.320312] Call Trace:
[13027.324218] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c [nf_defrag_ipv6]
[13027.328125] [<82aba07c>] ipv6_defrag+0x74/0xc8 [nf_defrag_ipv6]
[13027.335937] [<8022ea74>] nf_iterate+0x98/0x104
[13027.339843] [<8022eb78>] nf_hook_slow+0x98/0x170
[13027.343750] [<8324643c>] ipv6_rcv+0x3fc/0x4a0 [ipv6]
[13027.347656] [<80209410>] __netif_receive_skb+0x63c/0x6c0
[13027.355468] [<8298f384>] ri_tasklet+0x104/0x2b0 [ifb]
[13027.359375] [<80079434>] tasklet_action+0x78/0xc8
[13027.363281] [<80078c48>] __do_softirq+0xb0/0x184
[13027.371093] [<80078dcc>] do_softirq+0x48/0x68
[13027.375000] [<80078fe8>] irq_exit+0x4c/0xb8
[13027.378906] [<8006316c>] ret_from_irq+0x0/0x4
[13027.382812] [<80063380>] r4k_wait+0x20/0x40
[13027.386718] [<80064e68>] cpu_idle+0x38/0x70
[13027.390625] [<803488fc>] start_kernel+0x374/0x394
[13027.394531]
[13027.398437]
[13027.398437] Code: 00003021 0c01e3a4 26160008 <8eb40004> 8fa20054
26130018 26f0b0a0 afa20014 afb60018
[13027.410156] Cpu 0
[13027.410156] $ 0 : 00000000 00000001 803a0000 00000100
[13027.417968] $ 4 : 80342e38 82abb0a0 00000000 01820ea9
[13027.421875] $ 8 : 00000000 00000000 00000000 00000000
[13027.425781] $12 : 00060000 1403f2b2 00000000 00000000
[13027.433593] $16 : 8087083e 8083e540 83217480 000000e8
[13027.437500] $20 : 0000002c 80870866 80870846 82ac0000
[13027.441406] $24 : 00000000 83361334
[13027.449218] $28 : 8032a000 8032bbb8 80340000 82aba334
[13027.453125] Hi : 00000031
[13027.457031] Lo : 00000871
[13027.457031] epc : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.464843] Tainted: G O
[13027.468750] ra : 82aba334 nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[13027.476562] Status: 1000fc03 KERNEL EXL IE
[13027.480468] Cause : 00800010
[13027.484375] BadVA : 8087086a
[13027.488281] PrId : 00019374 (MIPS 24Kc)
[13027.492187] Modules linked in: sch_teql sch_tbf sch_sfq sch_red sch_qfq
sch_prio sch_ns2_codel sch_nfq_codel sch_netem sch_htb sch_gred
sch_efq_codel sch_dsmark em_text em_nbyte em_meta em_cmp cls_basic
act_police act_ipt act_connmark act_skbedit act_mirred em_u32 cls_u32
cls_tcindex cls_flow cls_route cls_fw sch_hfsc sch_fq_codel sch_codel
sch_ingress ath79_wdt xt_hashlimit xt_set(O) ip_set_list_set(O)
ip_set_hash_netport(O) ip_set_hash_netiface(O) ip_set_hash_net(O)
ip_set_hash_ipportnet(O) ip_set_hash_ipportip(O) ip_set_hash_ipport(O)
ip_set_hash_ip(O) ip_set_bitmap_port(O) ip_set_bitmap_ipmac(O)
ip_set_bitmap_ip(O) ip_set(O) ip6t_REJECT ip6t_rt ip6t_hbh ip6t_mh
ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6table_mangle
ip6table_filter ip6_tables nf_conntrack_ipv6 nf_defrag_ipv6 nfnetlink
nf_nat_irc nf_nat_ftp nf_conntrack_irc nf_conntrack_ftp xt_policy xt_esp
ipt_ah xt_HL xt_hl xt_ecn ipt_ECN xt_CLASSIFY xt_time xt_tcpmss
xt_statistic xt_mark xt_length xt_DSCP xt_dscp xt_string xt_layer7 xt_quota
xt_pkttype xt_physdev xt_owner xt_addrtype ipt_MASQUERADE iptable_nat
xt_nat nf_nat_ipv4 nf_nat xt_recent xt_helper xt_connmark xt_connbytes pptp
pppoe xt_conntrack xt_CT iptable_raw xt_state nf_conntrack_ipv4
nf_defrag_ipv4 nf_conntrack pppox ipt_REJECT xt_TCPMSS xt_LOG xt_comment
xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables
xt_tcpudp x_tables ip_gre gre ifb sit ipcomp xfrm4_tunnel xfrm4_mode_tunnel
xfrm4_mode_transport xfrm4_mode_beet esp4 ah4 tunnel4 tun tcp_ledbat(O)
ppp_async ppp_generic slhc af_key xfrm_user xfrm_ipcomp xfrm_algo vfat fat
autofs4 button_hotplug(O) ath9k(O) ath9k_common(O) ath9k_hw(O) ath(O)
nls_utf8 nls_iso8859_2 nls_iso8859_15 nls_iso8859_13 nls_iso8859_1
nls_cp437 mac80211(O) ts_fsm ts_bm ts_kmp crc_ccitt ipv6 input_polldev
cfg80211(O) compat(O) input_core sha1_generic md5 hmac des_generic deflate
cbc authenc arc4 usb_storage ohci_hcd ehci_hcd sd_mod ext4 jbd2 mbcache
usbcore usb_common scsi_mod nls_base crc16 zlib_inflate zlib_deflate
ledtrig_timer ledtrig_default_on leds_gpio gpio_button_hotplug(O)
[13027.675781] Process swapper (pid: 0, threadinfo=8032a000, task=8032f030,
tls=00000000)
[13027.683593] Stack : 8381a000 8381a800 c196c5e8 829cc77c c1c3d570
829cc77c 00000005 80064b68
[13027.683593] 00000006 8381a800 00000000 83217480 8381a800
00000000 8381a800 00000001
[13027.683593] 00000004 00000000 83245f98 82aba07c a20046c0
00000001 8381a000 82a643e0
[13027.683593] 8032bcac 823040c0 803a5428 8032bca4 83217480
8032dfd0 00000000 8022ea74
[13027.683593] 00000000 82304840 00000000 187ac42a 83245f98
80881d48 82abae60 83217480
[13027.683593] ...
[13027.718750] Call Trace:
[13027.722656] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c [nf_defrag_ipv6]
[12596.171875] 000008a0 00000000 00000000 3596c669 83245f98
80881d48 82a
[12596.171875]
...
[12596.207031] Call
Trace:
[12596.210937] [<82aba334>] nf_ct_frag6_gather+0x178/0x92c
[nf_defrag_ipv6]
[12596.218750] [<82aba07c>] ipv6_defrag+0x74/0xc8
[nf_defrag_ipv6]
[12596.222656] [<8022ea74>]
nf_iterate+0x98/0x104
[12596.226562] [<8022eb78>]
nf_hook_slow+0x98/0x170
[12596.234375] [<8324643c>] ipv6_rcv+0x3fc/0x4a0
[ipv6]
[12596.238281] [<80209410>]
__netif_receive_skb+0x63c/0x6c0
[12596.242187] [<8298f384>] ri_tasklet+0x104/0x2b0
[ifb]
[12596.250000] [<80079434>]
tasklet_action+0x78/0xc8
[12596.253906] [<80078c48>]
__do_softirq+0xb0/0x184
[12596.257812] [<80078dcc>]
do_softirq+0x48/0x68
[12596.261718] [<80078fe8>]
irq_exit+0x4c/0xb8
[12596.265625] [<8006316c>]
ret_from_irq+0x0/0x4
[12596.269531] [<80063380>]
r4k_wait+0x20/0x40
[12596.273437] [<80064e68>]
cpu_idle+0x38/0x70
[12596.277343] [<803488fc>]
start_kernel+0x374/0x394
[12596.285156]
Updated by David Taht on Apr 11, 2013.
This is a static export of the original bufferbloat.net issue
database. As such, no further commenting is possible; the
information is solely here for archival purposes.
Cerowrt Project
Recent Updates
Find us elsewhere
Bufferbloat Mailing Lists
#bufferbloat on Twitter
Google+ group
Archived Bufferbloat pages from the Wayback Machine
Nlnet Foundation
Shuttleworth Foundation
GoFundMe
Congestion Control Blog
Flent Network Test Suite
Sqm-Scripts
The Cake shaper
AQMs in BSD
IETF AQM WG
CeroWrt (where it all started)
Jim Gettys' Blog - The chairman of the Fjord
Toke's Blog - Karlstad University's work on bloat
Voip Users Conference - Weekly Videoconference mostly about voip
Candelatech - A wifi testing company that "gets it".
#bufferbloat on Twitter
Google+ group
Archived Bufferbloat pages from the Wayback Machine
Sponsors
Comcast Research Innovation FundNlnet Foundation
Shuttleworth Foundation
GoFundMe
Bufferbloat Related Projects
OpenWrt ProjectCongestion Control Blog
Flent Network Test Suite
Sqm-Scripts
The Cake shaper
AQMs in BSD
IETF AQM WG
CeroWrt (where it all started)
Network Performance Related Resources
Jim Gettys' Blog - The chairman of the Fjord
Toke's Blog - Karlstad University's work on bloat
Voip Users Conference - Weekly Videoconference mostly about voip
Candelatech - A wifi testing company that "gets it".