The story of the dying internet

So my internet went down again this week. And again, it took a few days to fix, but nothing compared to the almost 2 weeks long outage that happened earlier this month. This time is was partly my upstream's fault, as they misinterpreted a message from Bell.

At the same time, Bell announced they would come onsite basically an hour before arriving, so that was a little problematic. Given the fact that the problem wasn't actually within the premises, it is quite frustrating that they expect people to be home during work hours to fix their internet...

I haven't migrated any services this time, being too busy with work, and hoping the situation would resolve quickly. It hasn't, but hopefully not too many mails were lost during the downtime.

All services should now be back to normal.

Full timeline

Here's the detailed timeline of the outage, mostly based on the email exchanges i had with my upstream provider, EI Catalyst (EIC, below). Other parties involved are Bell Canada (Bell, below) and Teksavvy Internet (TSI), for some obscure reason.

• 2015-05-26 (tuesday)
  • ~04:30 DSL sync goes down
  • 06:03 ticket opened with upstream EIC
  • 12:09 first response from EIC: "still no sync?"
  • 12:55 ticket opened with Bell by TSI
  • 14:01 followup from TSI: "what is the OID?"
• 2015-05-27 (wednesday)
  • 08:22 test completed at Bell
  • 09:25 update requested from EIC: "still no sync"
  • 12:58 mail from Bell to EIC announcing visit before 18h00
  • 13:33 response from EIC: "bell say they fixed it"
  • 13:55 ticket "assigned" at Bell
  • probaby around that time: first visit from a Bell operator onsite, not present
  • 14:33 response to EIC: "they did not fix it, still no network"
  • 16:36 bell updates shared by EIC, now expecting bell to show up thursday
• 2015-05-28 (thursday)
  • 07:30 still no sync
  • 12:36 talked with neighbor, mentionned Bell visited the day before
  • 12:44 call with EIC: "they came yesterday!"
  • 13:04 call back from EIC: "escalated upstream, visit expected by 18:00
  • 14:30 visit from Bell operator, traced the line to the telephone pole
  • 15:00 Bell operator still working on te pole
  • 15:29 DSL sync restored, the line was hooked to the wrong terminal

The line was originally synced at 8mbps upstream, i requested that to be tuned to 11mbps as previously.

The stats are now:

Mode:   VDSL2 Annex A
Traffic Type:   PTM
Status: Up
Link Power State:   L0
Copper Loop(kft):   0.0

    Downstream  Upstream
Line Coding(Trellis):   On  On
SNR Margin (0.1 dB):    230 84
Attenuation (0.1 dB):   141 0
Output Power (0.1 dBm): 142 -33
Attainable Rate (Kbps): 62629   14792

    Path 0      Path 1   
    Downstream  Upstream    Downstream  Upstream
Rate (Kbps):    26943   11321   0   0

B (# of bytes in Mux Data Frame):   237 240 0   0
M (# of Mux Data Frames in an RS codeword): 1   1   0   0
T (# of Mux Data Frames in an OH sub-frame):    64  22  0   0
R (# of redundancy bytes in the RS codeword):   16  14  0   0
S (# of data symbols over which the RS code word spans):    0.2810  0.6764  0.0000  0.0000
L (# of bits transmitted in each data symbol):  7232    3016    0   0
D (interleaver depth):  1   1   0   0
I (interleaver block size in bytes):    254 255 0   0
N (RS codeword size):   254 255 0   0
Delay (msec):   0   0   0   0
INP (DMT symbol):   0.00    0.00    0.00    0.00

OH Frames:  0   0   0   0
OH Frame Errors:    2074    1   0   0
RS Words:   13003820    1127732 0   0
RS Correctable Errors:  0   0   0   0
RS Uncorrectable Errors:    0   0   0   0

HEC Errors: 14  0   0   0
OCD Errors: 0   0   0   0
LCD Errors: 0   0   0   0
Total Cells:    47565692    0   0   0
Data Cells: 4204814 0   0   0
Bit Errors: 0   0   0   0

Total ES:   10  2
Total SES:  10  1
Total UAS:  171 161