Wednesday, August 25, 2010

Explanation from Perform

The company behind the streaming of live commentary on Bulls Player has given an explanation to webmasters of the problems they encountered last Saturday when many subscribers were unable to listen to the commentary from Chesterfield.

Dear Webmasters,

Please find below a explanation of the issues that were experienced by Perform on Saturday afternoon. This explanation was posted on Club Central on Monday but this email is to ensure that all communication channels are covered off.

As you are aware we unfortunately experienced some technology issues that affected subscribers logging and webmasters from accessing the toolkit. Outlined below is a summary of events with the timelines involved which we hope will provide some background to the issues experienced:


At 2.40pm we started to experience major log-in issues to FLi Player and the Toolkit. Our software engineers identified a serious issue with the DNS server, that is effectively the piece of hardware that manages much of the communication around our infrastructure platform (importantly it would send user/CMS data to the Oracle database to verify and authenticate subscriber information).

At 2.58pm we restarted the DNS server and this allowed users for a ten minute period to log-in to FLi Player sites.

Positively this captured the vast majority of subscribers looking to log-in. Across FLi, we were peaking at 12,000 commentary streams which is in line with the end of last season and the first Saturday of this season, but marginally down (10% on last week). Our customer service enquiries were only marginally higher than a usual weekend, which proves we were lucky to capture the vast majority of users in the first 10 minutes of the game (when most people log in).

The DNS server then started to fail again at 3.10pm, which meant log in to Player was intermittent from 3.10pm to 3.45pm. Our engineers identified what the route cause was at 3.40pm which was a log of communications that was blocking the authentication process. The core system DNS was fixed and fully functional during half time to allow seamless log in to both the tool kit and Player.

The second half had no further issues. No other services were affected (SMS, Websites, Wap Sites, Site Data, Applications etc)

Obviously we have had a number of queries from subscribers who were affected. In all cases we have offered to extend their subscription by a week or month to alleviate any customer dissatisfaction.

Sincerest apologies from all at Perform and we appreciate the undue work this will have caused you.