Scope

MiFID II requirements cover many different areas, such as staffing; general testing methodology; conformance and algorithm testing; testing environments; stress testing and the role of self-assessments. The focus of this white paper is restricted to the trading venue capacity/performance testing and the associated challenges.

MiFID II testing requirements for trading venues are stated in the Article 48 and these are further clarified by the regulatory technical standard (RTS) 7.

Article 48 requires regulated markets to ensure that their trading systems can perform orderly trading under conditions of severe market stress and meet strict testing criteria. These requirements apply to regulated markets as well as multilateral trading facilities (MTFs) and organized trading facilities (OTFs).

Summary of MiFID II: Capacity/Performance testing requirements

1. Trading venues shall ensure that their trading systems have sufficient capacity to perform their functions without systems failures, outages or errors in matching transactions at least at twice the highest number of “messages per second” recorded on that system, during the previous five years.

2. Regular assessment of trading system capacity shall ensure that trading venue systems are able to cope with rising message flows without substantial degradation of system performance. In particular, the design of the trading system shall be scalable, so its capacity can be expanded within reasonable time, whenever necessary.

3. Trading venues shall perform stress tests where they simulate adverse scenarios, to verify the performance of the hardware, software and communications and identify the scenarios under which the trading system or parts of the trading system perform their functions with systems failures, outages or errors in matching transactions.

4. Stress tests shall cover all trading phases, trading segments and types of instruments traded by a trading venue and shall simulate members’ activities with the existing connectivity set-up.

The challenge

The above mentioned MiFID II test requirements present a number of challenges for QA engineer when developing a comprehensive test strategy.

1. The simulation of realistic trading order flow from all of the trading venue clients for both FIX and native sessions (if applicable), with the ability to control system load (message rate).

   One can devise a strategy that employs a functional test that covers typical trading scenarios. But, the test setup normally requires pairing of Buy/Sell counter-parties and the trade message flows that are strictly time-lined, to ensure synchronization of trade actions and the repeatability of order matching. While this setup, when replicated across different client sessions and provided they are segregated in groups trading different symbols, can serve the purpose of applying the required test load to the system – the distribution of message load is unfortunately evenly balanced across the order book, which is atypical in the real life trading situations.

2. To validate that the various feeds (Market Data, Regulatory, Surveillance, Drop-copy, etc.) operate consistently and meet performance targets, under variable high load trading conditions.

In a true production environment, the output from market feeds is non-predictive. The feed output will have a different message sequence and data content – even if, the trade orders for each client session are replayed with the same timed sequence. This is due to the effect of competing (asynchronous) trading actions from multiple client sessions, leading to a different order of trade matching between counter-parties. The order of matching is affected by many factors such as, order submission rates and the number of clients competing for the same trade.

3. When the speed/capacity requirements change, the QA department must continually adjust their test strategy to apply a different stimulus setup to the trading system.

   Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run. It would be nice to keep the market feed verification setup the same –saving on the time and effort required to update the test strategy to meet the new requirement. Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run.

Test Strategy

Let us map the above mentioned MiFID II system test requirements into the requirements for an automated test environment capable of addressing those challenges.

We need a test environment that provides: