Stock exchange real-time monitoring: from silos to enterprise wide

Typically, stock exchanges around the world have grown organically. They start with a stock exchange business unit and then a systems department. This systems department caters to all the IT needs of the business unit. It usually consists of a team made up of network and hardware engineers, system administrators and application teams who perform development, as well as operations tasks (DevOps).

As the volume on the exchange grows, so does the need for better hardware, networks and applications. Hence, the exchange begins to grow organically. Separate teams of network, hardware and application support are formed to focus on these areas. Technical architects design and deploy scalable and distributable systems resulting in an exponential increase in the number of servers in the estate. The IT Ops teams then adapt and deploy monitoring tools suitable for their particular architectures. This gives birth to what we call “silo-based” monitoring where different IT teams use different monitoring tools to monitor their specific areas and applications without having visibility of developments and events in other IT departments.

The disadvantages of such silo-based monitoring are numerous:

• Teams work as independent units and, hence, often work on the same problem without realising as they are not aware of what other departments are doing.
• There is no owner of “ghost” problems.
• In order to rectify a problem, one or more teams take parallel resolution actions even though they are not sure where the problem is.
• The business as a whole suffers, particularly when there is an outage.

The solution to this is enterprise level monitoring or monitoring the exchange as one team. This can be achieved by visualizing the state of IT operations, particularly monitoring, in a single frame of glass: “one view”. By “one view”, I don’t just mean one static view for all. This view is built on the same underlying data but can be tailored to end consumers (whether business or IT). 

The three main constituents required to achieve “one view” are:

Envision business and technology as one

A monitoring tool to monitor the availability and performance of infrastructure and applications, as well as end user experience. To convey the latter, it’s vital that data is analysed and presented in relevant metrics: transaction latency, delayed messages, order status, connectivity, and so forth.

One dashboard depicting the health of the entire enterprise in real-time

Said dashboard helps break down the hidden Chinese walls (silos) between various IT teams and helps them to work as one. Any incident pertaining to one department is visible to other departments at a level which keeps them informed but, at the same time, does not divulge “to the bolt” details of the incident, which is primarily the responsibility of the affected IT team. Teams learn to collaborate as they are made aware of the downstream impact of the actions they take.

Moreover, senior management benefit from a service level view of their enterprise.

An enterprise level monitoring tool

An enterprise level monitoring tool helps exchanges to leverage data from across their entire IT estate and, in turn, is an enabler of modern day IT operations analytics (ITOA).

An enterprise level monitoring tool is one which can monitor, in real-time, the availability and performance of:

• Infrastructure (network and hardware).
• Middleware.
• Databases.
• Applications (vendor or custom built).
• End user experience.