MiFID II – Trading Venue Capacity/performance Testing USING TESTBENCH and LOGCMP AND EMPLOYING ADAPTIVE TESTING

Scope

MiFID II requirements cover many different areas, such as Staffing; general testing methodology; conformance and algorithm testing; testing environments; stress testing and the role of self-assessments. The focus of this white paper is restricted to the trading venue capacity/performance testing and the associated challenges.

MiFID II testing requirements for trading venues are stated in the Article 48 and these are further clarified by the regulatory technical standards 7.

Article 48 requires regulated markets to ensure that their trading systems can perform orderly trading under conditions of severe market stress and meet strict testing criteria. These requirements apply to regulated markets as well as multilateral trading facilities (MTFs) and organized trading facilities (OTFs).

MiFID II: Capacity/Performance test requirements

  1. Trading venues shall ensure that their trading systems have sufficient capacity to perform their functions without systems failures, outages or errors in matching transactions at least at twice the highest number of “messages per second” recorded on that system, during the previous five years.

  2. Regular assessment of trading system capacity shall ensure that trading venue systems are able to cope with rising message flows without substantial degradation of system performance. In particular, the design of the trading system shall be scalable, so its capacity can be expanded within reasonable time, whenever necessary.

  3. Trading venues shall perform stress tests where they simulate adverse scenarios, to verify the performance of the hardware, software and communications and identify the scenarios under which the trading system or parts of the trading system perform their functions with systems failures, outages or errors in matching transactions.

  4. Stress tests shall cover all trading phases, trading segments and types of instruments traded by a trading venue and shall simulate members’ activities with the existing connectivity set-up.

The challenge

The above mentioned MiFID II test requirements present a number of challenges for QA engineer when developing a comprehensive test strategy.

  1. The simulation of realistic trading order flow from all of the trading venue clients for both FIX and native sessions (if applicable), with the ability to control system load (message rate).

    One can devise a strategy that employs a functional test that covers typical trading scenarios. But, the test setup normally requires pairing of Buy/Sell counter-parties and the trade message flows that are strictly time-lined, to ensure synchronization of trade actions and the repeatability of order matching. While this setup, when replicated across different client sessions and provided they are segregated in groups trading different symbols, can serve the purpose of applying the required test load to the system – the distribution of message load is unfortunately evenly balanced across the order book, which is atypical in the real life trading situations.

  2. To validate that the various feeds (Market Data, Regulatory, Surveillance, Drop-copy, etc.) operate consistently and meet performance targets, under variable high load trading conditions.

In a true production environment, the output from market feeds is non-predictive. The feed output will have a different message sequence and data content – even if, the trade orders for each client session are replayed with the same timed sequence. This is due to the effect of competing (asynchronous) trading actions from multiple client sessions, leading to a different order of trade matching between counter-parties. The order of matching is affected by many factors such as, order submission rates and the number of clients competing for the same trade.

  1. When the speed/capacity requirements change, the QA department must continually adjust their test strategy to apply a different stimulus setup to the trading system.

    Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run. It would be nice to keep the market feed verification setup the same –saving on the time and effort required to update the test strategy to meet the new requirement. Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run.

Test strategy

Let us map the above mentioned MiFID II system test requirements into the requirements for an automated test environment capable of addressing those challenges.

We need a test environment that provides:

You need to login to view the rest of the content. Please . Not a Member? Join Us

Offline Data Feed Verification using Log Compare (“LOGCMP”) an Automated Verification Tool employing Adaptive Testing from INCEPTRUM

Share Facebooktwittergoogle_pluspinterestlinkedinmail

Ottawa, Canada, October 3, 2017 – INCEPTRUM Technologies Inc., a solutions provider to the capital markets industry, today announced the availability of its latest automated verification tool, called Log Compare (“LOGCMP”) which supports quality assurance verification (offline auditing) of Market Data and Regulatory (IIROC) Feeds.

LOGCMP is the latest addition to the Data Sentinel suite of automated verification tools from INCEPTRUM and employs the concept called “Adaptive Testing” for true verification test coverage (offline) of Market Feed output.

This new verification approach uses information such as, Drop-Copy and Execution Reports, sourced in from trading system production logs, to re-create a dynamic data reference with which to verify any feed under test.

LOGCMP automatically adapts to different trade flows recorded in production data logs, removing the need for a test engineer to support multiple test setups associated with different trading scenarios. A single test setup in LOGCMP, is all you require to test different trading flows presented to the trading system and verifies that the logged output from multiple back-end data feeds was as expected.

Dmitry Kursov, President & CTO, Inceptrum said, “Our “Adaptive Testing” concept provides a much needed solution for verifying Market Feed output under high load conditions — which is not feasible using traditional functional test setups, due to the non-predictive order of trade matching between counter-parties under variable load conditions. Comparing feed output against a static data reference is not an appropriate test approach under high load conditions, due to changes in trade message sequence and content (i.e. price, volume, etc.) for each test run.”

LOGCMP can be deployed for non-intrusive post-transaction auditing in both development and production trading environments  checking for message drop-outs, out of sequence messages, malformed messages and inconsistent data content on your Market Feeds. One of the key advantages of LOGCMP tool is its ability to run multiple analysis passes on the same post transaction production data logs. The analysis coverage provided by LOGCMP, is identical to the highly acclaimed real-time monitoring tool “FEEDCMP”, released by INCEPTRUM earlier this year and adopted by a major Canadian Exchange.

Please visit https://www.inceptrum.com/en/logcmp/ for more information on this product.

About INCEPTRUM

INCEPTRUM Technologies Inc. is a Canadian privately held company, located in Ottawa, Ontario, Canada. Providing creative, leading-edge, low data latency solutions for the financial industry.
For more information, visit 
www.inceptrum.com.

Media Contact:

Ian Hughes,
Inceptrum Technologies Inc.

ian.hughes@inceptrum.com
1-613.699.2016 (direct)
1-877.763.6996 x103 (toll-free)