CLS commissioned IBM and Interoute (formerly Vtesse) to implement a new data centre and network. Regulated by the US Federal Reserve, CLS Bank as a designated critical component of the World Financial Industry, is required according to the US Federal Reserve’s Interagency Paper on Sound Practices to Strengthen the Resilience of the U.S. Financial System to withstand any regional disaster and be able to resume its service from another location in another region, within 2 hours. Synchronous replication of data is the only way to achieve service resumption into another region within 2 hours.
Historically, replicating synchronously between two data centres is limited to a maximum distance of 120kms to 150kms: to comply with the Interagency Paper required greater than 400kms such that one data centre could reside in a different region.
Working in partnership with CLS and IBM, an innovative, long-distance, resilient solution across a multi-route fibre network between data centres in the UK and mainland Europe was designed, developed and deployed. Infrastructure and application software were upgraded to enable synchronous data replication over 800kms. Data centres could then be sited at this extended distance, reducing the risk of data loss.
The solution required innovative breakthroughs in the communications network, the server platform and the Bank’s settlement system. A long-distance network that presented the required bandwidth, interfaces and scalability, whilst guaranteeing a crucial 1,600km round trip delay of less than 10 milliseconds was designed.
After careful evaluation of risks, fibre routes were aggregated from several different infrastructure owners, incorporating two separate English Channel crossings. Resilience was underpinned by ensuring that network routes avoided commercial co-location centres and major city centres liable to high instances of route disturbance. A network latency simulator was used to reflect the effects of network latency in a simulated distance of 800km between the two data centres. This practical construction of a complete replica of the network involved actual networking hardware, and fibre cable spooled on drums to represent the actual network distances. The results were much more credible than testing via software simulation. In an iterative process over several weeks, IBM modelled and tuned the server and storage performance, and worked with CLS Bank to optimise and recode the applications for maximum performance.