OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#8045 — bhs-s4-6k
Incident Report for Network & Infrastructure
Resolved
We are having a hardware problem on the sup2t of this router, we will proceed to a replacement:

Feb 7 16:39:41 GMT: %EARL-2-SWITCH_BUS_IDLE: Switching bus is idle for 5 seconds. The card grant is 0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x55] ME_KR_KDTX_ST_INTF_FC_STATUS = 0x1
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x6B] ME_KR_KDRX_ST_INTF_FC_STATUS = 0x1
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x125] ME_KR_KATX_AR_INTF_FC_STATUS = 0x3
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x153] ME_KR_KARX_AR_INTF_FC_STATUS = 0x7
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x129] ME_KR_KARX_AR_INTF_STATUS_4_0 = 0x8
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x484] ME_KR_KDRX2_DEADLOCK_FIFO_STATUS = 0x3
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x680] ME_KR_TESTMUX2_DEADLOCK_FIFO_STATUS = 0x3B803BF0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x609] ME_KR_KARX2_STA_KARX_ARB_0 = 0xC8884
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x60A] ME_KR_KARX2_STA_KARX_ARB_1 = 0x11800004
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x60B] ME_KR_KARX2_STA_KARX_ARB_2 = 0x40002
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x60C] ME_KR_KARX2_STA_KARX_ARB_3 = 0x80008
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x4B] ME_KR_KDTX_CNT_TX_CMPT_FR = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x6A] ME_KR_KDRX_CNT_RX_CMPT_FR = 0xC0000000
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x4C] ME_KR_KDTX_CNT_PKT_LOCAL_RESULT = 0x3
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x1D6] ME_KR_TESTMUX_TESTMUX_OUT_DATA__1 = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x1D7] ME_KR_TESTMUX_TESTMUX_OUT_DATA__0 = 0xA0100086
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x124] ME_KR_KATX_CNT_TX_PKT = 0x3
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x4D] ME_KR_KDTX_CNT_PKT_UNICAST = 0x3
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x4E] ME_KR_KDTX_CNT_PKT_MULTICAST = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x4F] ME_KR_KDTX_CNT_PKT_MACN = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x50] ME_KR_KDTX_CNT_PKT_RECIRC = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x51] ME_KR_KDTX_CNT_PKT_ZEROPSEL = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x52] ME_KR_KDTX_CNT_PKT_ZEROPSEL_ZEROCAP = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x53] ME_KR_KDTX_CNT_RECIRC_PKT_DROP_THR2 = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x54] ME_KR_KDTX_CNT_MACN_PKT_DROP_THR2 = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x12A] ME_KR_KARX_CNT_SEQ_ERR = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x12B] ME_KR_KARX_CNT_CRC_ERR = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x12C] ME_KR_KARX_CNT_LEN_ERR = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x20E] ME_KR_TI_PR_PR_PR_EGRESS_SPAN_0_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x20F] ME_KR_TI_PR_PR_PR_EGRESS_SPAN_1_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x210] ME_KR_TI_PR_PR_PR_INGRESS_SPAN_0_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x211] ME_KR_TI_PR_PR_PR_INGRESS_SPAN_1_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x212] ME_KR_TI_PR_PR_PR_ML3_PKT_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x213] ME_KR_TI_PR_PR_PR_ML2_PKT_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0xFD] ME_KR_URE_PKT_ERR_S_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0xFE] ME_KR_URE_PKT_ERR_E_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x1C3] ME_KR_ERROR_CNT_SEQ_NUM_ERR_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x1C7] ME_KR_ERROR_CNT_RBUS_FCS_ERR_CNT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x74] ME_KR_RXHF_PROTOCOL_INTERRUPT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x79] ME_KR_URE_INTERRUPT_37_32 = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x7A] ME_KR_URE_INTERRUPT_31_0 = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x46] ME_KR_KDTX_KARX_LE_SEQ_ERR_ITRP = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x3F] ME_KR_KDTX_RSLT_SEQ_ERR_ITRP = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x126] ME_KR_KARX_INTERRUPT_4_0 = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x2C] ME_AR_DIF_INTERFACE_MON = 0xFF
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0xC3] ME_AR_FIF_A2F_FCTL_STATE = 0xAA
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0xC4] ME_AR_FIF_F2A_FCTL_STATE = 0x8888
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x17E] ME_AR_KIF_TX_TX_STAT = 0x0
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x1E5] ME_AR_P1MMU_FREE_PAGE_CNT = 0x5F8
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x1E6] ME_AR_P1MMU_FREE_HEAD = 0x13
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x1E7] ME_AR_P1MMU_FREE_TAIL = 0x9
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x20A] ME_AR_P2MMU_FREE_PAGE_CNT = 0x5B8
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x20B] ME_AR_P2MMU_FREE_HEAD = 0x4B
.Feb 7 16:39:41 GMT: %PF_ASIC-3-ASIC_DUMP: [0:0x20C] ME_AR_P2MMU_FREE_TAIL = 0x5

Update(s):

Date: 2013-02-14 23:52:28 UTC
Done.

The card is operational.

bhs-s4-6k# sh platform hardware cef exception status
Current IPv4 FIB exception state = FALSE
Current IPv6 FIB exception state = FALSE
Current MPLS FIB exception state = FALSE
Current EoM/VPLS FIB TCAM exception state = FALSE

bhs-s4-6k#at
bhs-s4-6k#attach 1
Trying Switch ...
Entering CONSOLE for Switch
Type \"^C^C^C\" to end this session


bhs-s4-6k-dfc1#sh platform hardware cef exception status
Current IPv4 FIB exception state = FALSE
Current IPv6 FIB exception state = FALSE
Current MPLS FIB exception state = FALSE
Current EoM/VPLS FIB TCAM exception state = FALSE

bhs-s4-6k-dfc1#exit

[Connection to Switch closed by foreign host]
bhs-s4-6k#attach 2
Trying Switch ...
Entering CONSOLE for Switch
Type \"^C^C^C\" to end this session


bhs-s4-6k-dfc2#sh platform hardware cef exception status
Current IPv4 FIB exception state = FALSE
Current IPv6 FIB exception state = FALSE
Current MPLS FIB exception state = FALSE
Current EoM/VPLS FIB TCAM exception state = FALSE

bhs-s4-6k-dfc2#


Date: 2013-02-14 15:28:36 UTC
The router is isolated from the network.

Date: 2013-02-14 15:28:18 UTC
We have received a new card.
We will proceed to replacement.

Date: 2013-02-12 00:40:46 UTC
We are rebooting the chassis.

Date: 2013-02-12 00:40:25 UTC
bhs-s4-6k#attach 1
Trying Switch ...
Entering CONSOLE for Switch
Type \"^C^C^C\" to end this session


bhs-s4-6k-dfc1#sh platform hardware cef exception status
Current IPv4 FIB exception state = TRUE
Current IPv6 FIB exception state = FALSE
Current MPLS FIB exception state = FALSE
Current EoM/VPLS FIB TCAM exception state = FALSE

bhs-s4-6k#attach 2
Trying Switch ...
Entering CONSOLE for Switch
Type \"^C^C^C\" to end this session


bhs-s4-6k-dfc2#sh platform hardware cef exception status
Current IPv4 FIB exception state = TRUE
Current IPv6 FIB exception state = TRUE
Current MPLS FIB exception state = FALSE
Current EoM/VPLS FIB TCAM exception state = FALSE

bhs-s4-6k-dfc2#exit

[Connection to Switch closed by foreign host]
bhs-s4-6k#sh platform hardware cef exception status
Current IPv4 FIB exception state = TRUE
Current IPv6 FIB exception state = TRUE
Current MPLS FIB exception state = FALSE
Current EoM/VPLS FIB TCAM exception state = FALSE

Date: 2013-02-12 00:40:06 UTC
We are switching off the router then replacing the sup2T.
Posted Feb 12, 2013 - 00:39 UTC