rssLink RSS for all categories
 
icon_red
icon_green
icon_red
icon_red
icon_red
icon_green
icon_green
icon_red
icon_red
icon_red
icon_orange
icon_green
icon_green
icon_green
icon_red
icon_blue
icon_red
icon_orange
icon_red
icon_red
icon_red
icon_red
icon_red
icon_red
icon_red
icon_red
icon_red
icon_orange
icon_green
 

FS#1876 — FS#5851 — vss-5b6k RBX4

Attached to Project— Network
Maintenance
the whole network
CLOSED
100%
We have a strange problem on the router. 10G ports crush gradually on the router and then return without a clear reason. No CRC, no problem on the links. And almost all links on each card.

The consequence is that there must be small cuts in the service here and there because the cuts are a matter of seconds, not enough so the PMO / PSO / HSRP / GLBP gets recalculated.
Date:  Thursday, 06 October 2011, 16:04PM
Reason for closing:  Done
Comment by OVH - Sunday, 02 October 2011, 01:04AM

Oct 1 22:31:49 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 1 22:32:02 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to down
Oct 1 22:32:02 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 1 22:36:06 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:36:07 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:36:11 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:36:11 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:36:23 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to down
Oct 1 22:36:23 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to down
Oct 1 22:36:27 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 1 22:36:27 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 1 22:40:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Oct 1 22:40:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 1 22:40:34 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to up
Oct 1 22:40:36 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:40:36 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:40:41 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:40:41 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:41:15 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to down
Oct 1 22:41:17 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 1 22:41:40 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Oct 1 22:41:42 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 1 22:41:51 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to down
Oct 1 22:41:51 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 1 22:42:45 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 1 22:42:45 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 1 22:43:06 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to down
Oct 1 22:43:06 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to down
Oct 1 22:43:10 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to up
Oct 1 22:43:10 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to up
Oct 1 22:46:34 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 1 22:46:40 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:46:40 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:46:45 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:46:44 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:46:53 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 1 22:46:54 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 1 22:47:40 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 1 22:47:40 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 1 22:47:53 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 1 22:49:40 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 1 22:50:30 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to down
Oct 1 22:50:30 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to down
Oct 1 22:50:35 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to up
Oct 1 22:50:35 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to up
Oct 1 22:50:46 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to down
Oct 1 22:50:46 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 1 22:51:08 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to down
Oct 1 22:51:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to down
Oct 1 22:51:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to up
Oct 1 22:51:13 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to up
Oct 1 22:51:45 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:51:45 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:51:49 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:51:49 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:52:02 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 22:52:02 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 22:52:06 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 22:52:06 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 22:55:46 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 1 22:55:46 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 1 22:55:51 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 1 22:55:52 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 1 22:56:15 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to down
Oct 1 22:56:15 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to down
Oct 1 22:56:19 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to up
Oct 1 22:56:18 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to up
Oct 1 22:57:57 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to down
Oct 1 22:57:57 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to down
Oct 1 22:58:02 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 1 22:58:02 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 1 22:59:06 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 22:59:06 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 22:59:11 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 22:59:11 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 22:59:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:59:14 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 22:59:19 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:59:19 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 22:59:25 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 1 23:06:08 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 23:06:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 23:06:10 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 1 23:06:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 23:06:12 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 23:10:04 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to down
Oct 1 23:10:04 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to down
Oct 1 23:10:09 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to up
Oct 1 23:10:09 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to up
Oct 1 23:10:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 1 23:12:08 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to down
Oct 1 23:12:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to down
Oct 1 23:12:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to up
Oct 1 23:12:13 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to up
Oct 1 23:12:15 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 1 23:12:54 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to down
Oct 1 23:12:54 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to down
Oct 1 23:12:58 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to up
Oct 1 23:12:58 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to up
Oct 1 23:15:08 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 23:15:07 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 23:15:12 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 23:15:12 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 23:21:11 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 23:21:11 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 23:21:16 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 23:21:16 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 23:22:08 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to down
Oct 1 23:22:07 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to down
Oct 1 23:22:12 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to up
Oct 1 23:22:12 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/3, changed state to up
Oct 1 23:22:49 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/4, changed state to down
Oct 1 23:22:49 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/4, changed state to down
Oct 1 23:22:58 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/4, changed state to up
Oct 1 23:22:58 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/4, changed state to up
Oct 1 23:24:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to down
Oct 1 23:24:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 1 23:25:14 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 1 23:25:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 1 23:25:19 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 1 23:25:19 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
.Oct 1 23:26:04 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Oct 1 23:26:04 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
.Oct 1 23:26:09 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
Oct 1 23:26:09 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to up
.Oct 1 23:28:03 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 23:28:03 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
.Oct 1 23:28:07 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 23:28:07 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 23:29:53 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 1 23:32:00 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1, changed state to down
Oct 1 23:32:00 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1, changed state to down
Oct 1 23:32:04 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1, changed state to up
Oct 1 23:32:04 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/1, changed state to up
Oct 1 23:35:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to down
Oct 1 23:35:13 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to down
Oct 1 23:35:17 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to up
Oct 1 23:35:17 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to up
Oct 1 23:36:58 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 23:36:58 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to down
Oct 1 23:37:03 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 23:37:03 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/1, changed state to up
Oct 1 23:37:07 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to down
Oct 1 23:37:07 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to down
Oct 1 23:37:08 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 1 23:37:13 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 1 23:37:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 1 23:39:59 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to down
Oct 1 23:39:59 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to down
Oct 1 23:40:03 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to up
Oct 1 23:40:03 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/2, changed state to up
Oct 1 23:42:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to down
Oct 1 23:42:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to up
Oct 1 23:42:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to down
Oct 1 23:42:13 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to down
Oct 1 23:42:13 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to up
Oct 1 23:42:14 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to down
Oct 1 23:42:17 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to up
Oct 1 23:42:17 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/4, changed state to up
Oct 1 23:43:08 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 1 23:45:10 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 1 23:45:10 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 1 23:45:16 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 1 23:45:16 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 1 23:45:33 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to up
Oct 1 23:46:45 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 1 23:47:07 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/2, changed state to up


Comment by OVH - Sunday, 02 October 2011, 02:47AM

We are preparing the reboot.

No traffic on the server anymore,vss-5a provides routing.
IP failover have been cut,vss-5a provides routing.


Comment by OVH - Sunday, 02 October 2011, 02:52AM

We cut HSRP and GLBP on the B.
Around 10-15 seconds should be given to the router to use subnet traffic again.


Comment by OVH - Sunday, 02 October 2011, 02:59AM

The router is back. But it is not going better. We will change the SUP card of the router.

Meanwhile vss-5a-6k continues routing. But it has problems. We will plan the switching of some networks from vss5AB to vss-6AB in order to allow vss-5AB to operate normally. We have unusually many attacks that must be studied and check what it is.
It will take time and during that time it has to keep operating properly.


Comment by OVH - Sunday, 02 October 2011, 06:03AM

The SUP has been replaced,but the problem continues.
We searched the origin of this inconvenience but we didn't find it.
Meanwhile,the A is not doing well at all.
We set back the traffic on the 2 routers,they are running both but it's not their best performance.

We will keep searching for the origin of this overload in order to define the kind of attack we are going through.
At the same time we are thinking about the cuts of the ports.


Comment by OVH - Sunday, 02 October 2011, 13:58PM

The problem is still there. The origin of the problem is not known for
the time being.

We have 2 different problems :

- On the Router B the ports are interrupted, sometimes, without no explication.

-The servers do not ping well and it has nothing to do with the first problem.

We have several tracks that we're going to browse for the 2 problems.


Comment by OVH - Sunday, 02 October 2011, 14:00PM

We changd again the time of the infrastructure to avoid
the expiry of the MAC positions on the ports.
This fixes a lot of problems.


Comment by OVH - Sunday, 02 October 2011, 14:01PM

We took 6 10G links which flapped since 1 hour et we check
the whole links to understand why it flaps.


Comment by OVH - Sunday, 02 October 2011, 14:03PM

We are going to change the way of server configuration
on the router to find a configuration of RBX1/2/3 .


Comment by OVH - Sunday, 02 October 2011, 14:05PM

B done.

We tackle the same thing on the A.

Then we reset the Ip failover factory to execute everything
in progress.


Comment by OVH - Sunday, 02 October 2011, 14:08PM

We think that the problem of port comes from the fact
that the LACP is falling because the router is overloaded.
But why B only and not A.

We interrupted one of the 3 route reflectors on A and B.


Comment by OVH - Sunday, 02 October 2011, 14:21PM

We are preparing the migation of all the 100Mbps servers
from vss-5AB to the new vss-6AB.

We interrupt these servers on the B. A is still to be routed.
We are going to recable the ports of vss-5B on vss-6B and
then reactivate the routing on vss-6B with its ip Failover.


Comment by OVH - Sunday, 02 October 2011, 14:28PM

It's done. All the 100M server are uniquely on the vss-5A.
The vss-5B is catching its breath and vss-5A it's OK. The change
consists in the fact that the router of each subnet is down. Maybe
the attack is turned toward this IP ?

We give some 20 minutes to B to see if the ports flap or not.
If it does not flap, it's all see. We switch all this on
vss-6B.


Comment by OVH - Sunday, 02 October 2011, 14:28PM

We are going to desactivate ipv6 on the vss-5b.


Comment by OVH - Sunday, 02 October 2011, 14:28PM

It's done.


Comment by OVH - Sunday, 02 October 2011, 15:20PM

It's breathing.

We are going to check if the attack was on ipv6 or not.

We set this working task :
http://status.ovh.net/?do=details&id=1877

and we are going to reactivate ipv6 on vss-5b.


Comment by OVH - Sunday, 02 October 2011, 15:21PM

It's done and the ports are flapping again.
What's that ???


Comment by OVH - Sunday, 02 October 2011, 15:21PM

We restart the ports of the 100M servers.


Comment by OVH - Sunday, 02 October 2011, 15:44PM

We reopened a TAC ticket by Cisco to understand from
where comes the problem.

In parallel, we switch the configuration from A to the
same type as B. It's almost 1h30 of work. Let's go !!!


Comment by OVH - Sunday, 02 October 2011, 16:06PM

We added a new 6704 card and we are going to switch some ports
which flap on this new card.


Comment by OVH - Sunday, 02 October 2011, 16:11PM

Switch done.


Comment by OVH - Sunday, 02 October 2011, 16:31PM

We synchronise the configuration between the 2 routers
then we relaunch the ip failover machine.

The router seems to be stable but we still have a flap on
vss-5b.
Oups, there's a flap on vss-5a ...
Help;


Comment by OVH - Sunday, 02 October 2011, 16:45PM

We change this type of port channel on the uplinks
of the 2 routers.


Comment by OVH - Sunday, 02 October 2011, 17:18PM

We desactivate flowcontrol on the router's ports (default configuration)


Comment by OVH - Sunday, 02 October 2011, 17:26PM

The fact of switching on another type of port channel
seem to fix the flapping problem. We tackle
this change with the switches at the heart of the router.


Comment by OVH - Sunday, 02 October 2011, 17:40PM

We checked the parametring of HSRP/GLBP to avoid
that the router gets down when the ports flaps.


Comment by OVH - Sunday, 02 October 2011, 17:42PM

The router will find the stability when the ports will
stop to flap. Without that, it's not possible.
Whatever.


Comment by OVH - Sunday, 02 October 2011, 17:43PM

vss-5b-6k#sh log | i Ten
Oct 2 16:13:57 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to down
Oct 2 16:14:47 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/3, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet7/1, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to down
Oct 2 16:14:48 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet7/1, changed state to down
Oct 2 16:14:52 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet7/1, changed state to up
Oct 2 16:14:52 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 2 16:14:52 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 2 16:14:54 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 2 16:14:56 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:14:56 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/3, changed state to up
Oct 2 16:14:58 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet7/1, changed state to up
Oct 2 16:15:00 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 2 16:15:02 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 2 16:15:03 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:15:04 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 2 16:15:06 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:15:07 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 2 16:15:10 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 2 16:15:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to up
Oct 2 16:20:46 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to down
Oct 2 16:20:55 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up

vss-5a-6k#sh log | i Ten
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/3, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to down
Oct 2 16:15:08 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to down
Oct 2 16:15:19 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 2 16:15:11 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 2 16:15:11 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:15:11 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:15:12 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 2 16:15:12 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to up
Oct 2 16:15:12 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 2 16:15:13 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 2 16:15:16 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 2 16:15:17 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 2 16:15:17 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 2 16:15:17 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:15:18 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/3, changed state to up
Oct 2 16:15:21 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:15:22 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:15:24 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 2 16:15:25 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/1, changed state to up
Oct 2 16:15:26 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 2 16:15:29 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 2 16:15:33 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 2 16:15:33 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/3, changed state to up
Oct 2 16:15:35 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/3, changed state to up
Oct 2 16:15:34 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to down
Oct 2 16:15:37 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 2 16:15:38 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:15:38 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 2 16:15:45 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/4, changed state to up
Oct 2 16:18:59 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to down
Oct 2 16:18:59 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 2 16:19:00 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to down
Oct 2 16:19:00 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to down
Oct 2 16:19:02 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 2 16:19:02 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/4, changed state to up
Oct 2 16:19:39 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 2 16:19:39 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to down
Oct 2 16:19:42 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 2 16:19:42 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/1, changed state to up
Oct 2 16:20:34 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:20:35 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to down
Oct 2 16:20:38 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:20:35 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to down
Oct 2 16:20:38 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:24:44 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:24:45 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:24:48 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:24:48 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:28:01 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to down
Oct 2 16:28:01 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to down
Oct 2 16:28:05 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 2 16:28:05 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/1, changed state to up
Oct 2 16:28:31 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:28:32 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to down
Oct 2 16:28:32 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to down
Oct 2 16:28:36 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:28:36 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet4/2, changed state to up
Oct 2 16:30:43 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:30:43 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:30:44 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:30:46 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:30:44 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:30:46 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:31:06 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to down
Oct 2 16:31:06 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to down
Oct 2 16:31:10 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 2 16:31:10 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 2 16:33:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:33:13 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:33:14 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:33:16 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:33:14 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to down
Oct 2 16:33:16 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:33:19 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:33:19 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:33:17 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/2, changed state to up
Oct 2 16:33:19 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:34:32 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/4, changed state to up
Oct 2 16:36:17 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Oct 2 16:36:17 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 2 16:36:19 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Oct 2 16:36:19 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Oct 2 16:36:21 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 2 16:36:21 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to up
Oct 2 16:37:38 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:37:38 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to down
Oct 2 16:37:41 GMT: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up
Oct 2 16:37:41 GMT: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet6/2, changed state to up


Comment by OVH - Sunday, 02 October 2011, 23:44PM

A few days ago we have strengthened the security of our routers to avoid attacks on our infrastructure.
We noted that our monitoring system has detected an attack since some time and the target was a router at OVH.
How could it happen? A taping mistake in the protection.
We just fixed it.It's locked actually and now we have a strong proof that the attacks are the origin of the problems that we encountred in the last few days:

http://status.ovh.co.uk/?do=details&id=1833
http://status.ovh.co.uk/?do=details&id=1865

Looking at the graphs, we noted that it takes about 5Gbps of traffic:

http://yfrog.com/z/nxxpbbsj
http://yfrog.com/z/nwr8ejwj
http://yfrog.com/z/kjwq0bej

The easiest way is to send an email to noc@ovh.net with the problem's origin ..


Comment by OVH - Monday, 03 October 2011, 00:31AM

After we set the port channel on static,the routers became stable and do not load.it's working with 30% of the CPU.

We have now an attack on the networks in the middle of the router and we will therefore work to identify the target of the attack,then block it in advance.


Comment by OVH - Monday, 03 October 2011, 01:03AM

A human error on the configuration of the network
:
46.105.116.0/24
46.105.117.0/24
46.105.118.0/24
46.105.119.0/24
176.31.226.0/24
176.31.227.0/24

It can be explained by the tireness of our team's members.
We have been on this task since 48 hours :(


Comment by OVH - Monday, 03 October 2011, 01:05AM

The CPU of the routers for the last 2 days. "it's going better"(c)

----9999999999999999999999999999999999999999999999999999999999999999999999
----9999999999999999999999999999999999999999999999999999999999999999999999
100-******************#*#*************************************************
-90-******************###*************************************************
-80-*********#*##*########***#********************************************
-70-*******#####################*######***********************************
-60-****#*##############################**********************************
-50-****#################################*******######*#******************
-40-######################################################################
-30-######################################################################
-20-######################################################################
-10-######################################################################
---0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
-------------0----5----0----5----0----5----0----5----0----5----0----5----0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%