Is Illinois Century Network down?

Illinois Century Network is UP

CURRENT STATUS April 29 Outage - Comcast Summary of Event... Customer: STATE OF ILLINOIS Root Cause Analysis Report Incident Synopsis: On April 29, 2024, at 09:10 AM, Comcast was investigating the service outage for the STATE OF ILLINOIS, Comcast monitoring system logged the service down at 08:07 AM, on April 29, 2024. • Comcast engineer requested a same day dispatch to troubleshoot a the headend. • 09:53 AM: An update was provided and no Estimated Time for Repair (ETR) was given. The Local Contact (LCON) was informed of the dispatch. • 10:08 AM: Bridge was set up and in progress, investigation ongoing. Multiple tickets opened for the service issue linked under parent Service Incident (SI) ticket, SI066047234. • 11:07 AM: Headend team troubleshooting verified no Receiver signal (Rx) from the location to the headend site. • 11:12 AM: Strategic Account Operations Manager (SAOM) reached out to the customer and advised no ETR, investigation ongoing. • 11:32 AM: Headend team verified Rx signals received from other nodes. • 11:56 AM: Headend teams continued to troubleshoot. An Optical Time Domain Reflectometer (OTDR) test was conducted. OTDR test found optical light signal loss at approximately thirty-two thousand seven hundred feet out in the Comcast outside network plant. Investigation ongoing to pinpoint cause, no ETR. • 12:23 PM: Dispatch request updated to add fiber repair. • 12:55 PM: Fiber team engaged, enroute to the site for further investigation. • 01:46 PM: Fiber team on site, troubleshot further, no ETR. Further investigation located fiber damage at 100 W Randolph St., Chicago, IL. 60601. Construction company performing renovation in the lower level damaged the underground Comcast fiber feed. Assessment and repairs underway, no ETR, updates to follow. • 02:51 PM: Customer called in for update on SI ticket, SI066046919. Comcast advised that ticket was resolved the outage is being worked under parent ticket, SI066047234. • 03:05 PM: Headend and fiber teams collaborated to roll fibers, no ETR. • 05:05 PM: Fiber teams verified light signal however, the routers remained offline. Troubleshooting ongoing, no ETR. • 06:16 PM: Remediation efforts continued, links started to recover. • 07:35 PM: Remediation efforts identified fiber optics for reroute, no ETR. • 09:21 PM Fiber remediation efforts successful after spare fiber swap. • At 10:24 PM, headend and fiber teams declared hands off. A Scheduled Maintenance Root Cause: Construction company performing renovation work at The Thompson Center, 100 W. Randolph St in the lower level damaged the Comcast fiber feed. Comcast has installed two redundant fiber paths to the Illinois Century Network Chicago POP. Redundant hardware installation pending. Once complete Comcast will have service redundancy eliminating single point of failure caused by a fiber cut. April 29 Outage 8:35PM - We confirm that all circuits have restored except for about 6 circuits which Comcast continues to work on. This will be our last update tonight unless there are new developments. Comcast will provide a summary of and cause of the outage and we will share that as soon as it is available. 7:15PM - We are seeing most sites restored. Some sites remain down and we continue to work with Comcast for full restoration. 3:44PM - Comcast is working to reroute ICN connections to new fiber path. That work has started. Still no ETA. 2:22PM - Comcast believes they have identified a fiber cut and working to locate where. 12:15PM - Comcast reporting possible fiber cut. To be verified. All comcast services to ICN Chicago POP facility are down. Comcast engineers are troubleshooting their hardware at our ICN Chicago POP facility. 8:30AM Comcast is reporting an outage on their network which is impacting multiple K12 Schools in the Chicago area. Engineers are working to resolve the issue.

CURRENT STATUS April 29 Outage 8:35PM - We confirm that all circuits have restored except for about 6 circuits which Comcast continues to work on. This will be our last update tonight unless there are new developments. Comcast will provide a summary of and cause of the outage and we will share that as soon as it is available. 7:15PM - We are seeing most sites restored. Some sites remain down and we continue to work with Comcast for full restoration. 3:44PM - Comcast is working to reroute ICN connections to new fiber path. That work has started. Still no ETA. 2:22PM - Comcast believes they have identified a fiber cut and working to locate where. 12:15PM - Comcast reporting possible fiber cut. To be verified. All comcast services to ICN Chicago POP facility are down. Comcast engineers are troubleshooting their hardware at our ICN Chicago POP facility. 8:30AM Comcast is reporting an outage on their network which is impacting multiple K12 Schools in the Chicago area. Engineers are working to resolve the issue.

CURRENT STATUS April 29 Outage 8:35PM - We confirm that all circuits have restored except for about 6 circuits which Comcast continues to work on. This will be our last update tonight unless there are new developments. Comcast will provide a summary of and cause of the outage and we will share that with you Tuesday April 30. 7:15PM - We are seeing most sites restored. Some sites remain down and we continue to work with Comcast for full restoration. 3:44PM - Comcast is working to reroute ICN connections to new fiber path. That work has started. Still no ETA. 2:22PM - Comcast believes they have identified a fiber cut and working to locate where. 12:15PM - Comcast reporting possible fiber cut. To be verified. All comcast services to ICN Chicago POP facility are down. Comcast engineers are troubleshooting their hardware at our ICN Chicago POP facility. 8:30AM Comcast is reporting an outage on their network which is impacting multiple K12 Schools in the Chicago area. Engineers are working to resolve the issue.

CURRENT STATUS April 29 Outage Comcast engineers are troubleshooting their hardware at our ICN Chicago POP facility. 8:30AM Comcast is reporting an outage on their network which is impacting multiple K12 Schools in the Chicago area. Engineers are working to resolve the issue.

Network Maintenance ICN engineers will perform network maintenance on Saturday April 22, 2023, starting at 11:00PM. A software upgrade will be installed on the SDWAN hardware located at schools participating in the DoIT K-12 Consortium Broadband Network. There will be a 10-minute interruption of service during the upgrade. Maintenance will be complete by 11:59PM. Should you experience any connectivity problems after the maintenance, please contact our Network Operations Center (NOC) at 312.814.3648 Option 2 or 217.524.3648 Option 2.