MiFID II – Trading Venue Capacity/performance Testing USING TESTBENCH and LOGCMP AND EMPLOYING ADAPTIVE TESTING

Scope

MiFID II requirements cover many different areas, such as Staffing; general testing methodology; conformance and algorithm testing; testing environments; stress testing and the role of self-assessments. The focus of this white paper is restricted to the trading venue capacity/performance testing and the associated challenges.

MiFID II testing requirements for trading venues are stated in the Article 48 and these are further clarified by the regulatory technical standards 7.

Article 48 requires regulated markets to ensure that their trading systems can perform orderly trading under conditions of severe market stress and meet strict testing criteria. These requirements apply to regulated markets as well as multilateral trading facilities (MTFs) and organized trading facilities (OTFs).

MiFID II: Capacity/Performance test requirements

  1. Trading venues shall ensure that their trading systems have sufficient capacity to perform their functions without systems failures, outages or errors in matching transactions at least at twice the highest number of “messages per second” recorded on that system, during the previous five years.

  2. Regular assessment of trading system capacity shall ensure that trading venue systems are able to cope with rising message flows without substantial degradation of system performance. In particular, the design of the trading system shall be scalable, so its capacity can be expanded within reasonable time, whenever necessary.

  3. Trading venues shall perform stress tests where they simulate adverse scenarios, to verify the performance of the hardware, software and communications and identify the scenarios under which the trading system or parts of the trading system perform their functions with systems failures, outages or errors in matching transactions.

  4. Stress tests shall cover all trading phases, trading segments and types of instruments traded by a trading venue and shall simulate members’ activities with the existing connectivity set-up.

The challenge

The above mentioned MiFID II test requirements present a number of challenges for QA engineer when developing a comprehensive test strategy.

  1. The simulation of realistic trading order flow from all of the trading venue clients for both FIX and native sessions (if applicable), with the ability to control system load (message rate).

    One can devise a strategy that employs a functional test that covers typical trading scenarios. But, the test setup normally requires pairing of Buy/Sell counter-parties and the trade message flows that are strictly time-lined, to ensure synchronization of trade actions and the repeatability of order matching. While this setup, when replicated across different client sessions and provided they are segregated in groups trading different symbols, can serve the purpose of applying the required test load to the system – the distribution of message load is unfortunately evenly balanced across the order book, which is atypical in the real life trading situations.

  2. To validate that the various feeds (Market Data, Regulatory, Surveillance, Drop-copy, etc.) operate consistently and meet performance targets, under variable high load trading conditions.

In a true production environment, the output from market feeds is non-predictive. The feed output will have a different message sequence and data content – even if, the trade orders for each client session are replayed with the same timed sequence. This is due to the effect of competing (asynchronous) trading actions from multiple client sessions, leading to a different order of trade matching between counter-parties. The order of matching is affected by many factors such as, order submission rates and the number of clients competing for the same trade.

  1. When the speed/capacity requirements change, the QA department must continually adjust their test strategy to apply a different stimulus setup to the trading system.

    Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run. It would be nice to keep the market feed verification setup the same –saving on the time and effort required to update the test strategy to meet the new requirement. Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run.

Test strategy

Let us map the above mentioned MiFID II system test requirements into the requirements for an automated test environment capable of addressing those challenges.

We need a test environment that provides:

You need to login to view the rest of the content. Please . Not a Member? Join Us

Offline Data Feed Verification using Log Compare (“LOGCMP”) an Automated Verification Tool employing Adaptive Testing from INCEPTRUM

Share Facebooktwittergoogle_pluspinterestlinkedinmail

Ottawa, Canada, October 3, 2017 – INCEPTRUM Technologies Inc., a solutions provider to the capital markets industry, today announced the availability of its latest automated verification tool, called Log Compare (“LOGCMP”) which supports quality assurance verification (offline auditing) of Market Data and Regulatory (IIROC) Feeds.

LOGCMP is the latest addition to the Data Sentinel suite of automated verification tools from INCEPTRUM and employs the concept called “Adaptive Testing” for true verification test coverage (offline) of Market Feed output.

This new verification approach uses information such as, Drop-Copy and Execution Reports, sourced in from trading system production logs, to re-create a dynamic data reference with which to verify any feed under test.

LOGCMP automatically adapts to different trade flows recorded in production data logs, removing the need for a test engineer to support multiple test setups associated with different trading scenarios. A single test setup in LOGCMP, is all you require to test different trading flows presented to the trading system and verifies that the logged output from multiple back-end data feeds was as expected.

Dmitry Kursov, President & CTO, Inceptrum said, “Our “Adaptive Testing” concept provides a much needed solution for verifying Market Feed output under high load conditions — which is not feasible using traditional functional test setups, due to the non-predictive order of trade matching between counter-parties under variable load conditions. Comparing feed output against a static data reference is not an appropriate test approach under high load conditions, due to changes in trade message sequence and content (i.e. price, volume, etc.) for each test run.”

LOGCMP can be deployed for non-intrusive post-transaction auditing in both development and production trading environments  checking for message drop-outs, out of sequence messages, malformed messages and inconsistent data content on your Market Feeds. One of the key advantages of LOGCMP tool is its ability to run multiple analysis passes on the same post transaction production data logs. The analysis coverage provided by LOGCMP, is identical to the highly acclaimed real-time monitoring tool “FEEDCMP”, released by INCEPTRUM earlier this year and adopted by a major Canadian Exchange.

Please visit https://www.inceptrum.com/en/logcmp/ for more information on this product.

About INCEPTRUM

INCEPTRUM Technologies Inc. is a Canadian privately held company, located in Ottawa, Ontario, Canada. Providing creative, leading-edge, low data latency solutions for the financial industry.
For more information, visit 
www.inceptrum.com.

Media Contact:

Ian Hughes,
Inceptrum Technologies Inc.

ian.hughes@inceptrum.com
1-613.699.2016 (direct)
1-877.763.6996 x103 (toll-free)

MiFID II – Trading Venue Capacity/Performance Testing using TESTBENCH and FEEDCMP test tools and employing adaptive testing

Scope

MiFID II requirements cover many different areas, such as staffing; general testing methodology; conformance and algorithm testing; testing environments; stress testing and the role of self-assessments. The focus of this white paper is restricted to the trading venue capacity/performance testing and the associated challenges.

MiFID II testing requirements for trading venues are stated in the Article 48 and these are further clarified by the regulatory technical standard (RTS) 7.

Article 48 requires regulated markets to ensure that their trading systems can perform orderly trading under conditions of severe market stress and meet strict testing criteria. These requirements apply to regulated markets as well as multilateral trading facilities (MTFs) and organized trading facilities (OTFs).

Summary of MiFID II: Capacity/Performance testing requirements

  1. Trading venues shall ensure that their trading systems have sufficient capacity to perform their functions without systems failures, outages or errors in matching transactions at least at twice the highest number of “messages per second” recorded on that system, during the previous five years.

  2. Regular assessment of trading system capacity shall ensure that trading venue systems are able to cope with rising message flows without substantial degradation of system performance. In particular, the design of the trading system shall be scalable, so its capacity can be expanded within reasonable time, whenever necessary.

  3. Trading venues shall perform stress tests where they simulate adverse scenarios, to verify the performance of the hardware, software and communications and identify the scenarios under which the trading system or parts of the trading system perform their functions with systems failures, outages or errors in matching transactions.

  4. Stress tests shall cover all trading phases, trading segments and types of instruments traded by a trading venue and shall simulate members’ activities with the existing connectivity set-up.

The challenge

The above mentioned MiFID II test requirements present a number of challenges for QA engineer when developing a comprehensive test strategy.

  1. The simulation of realistic trading order flow from all of the trading venue clients for both FIX and native sessions (if applicable), with the ability to control system load (message rate).

    One can devise a strategy that employs a functional test that covers typical trading scenarios. But, the test setup normally requires pairing of Buy/Sell counter-parties and the trade message flows that are strictly time-lined, to ensure synchronization of trade actions and the repeatability of order matching. While this setup, when replicated across different client sessions and provided they are segregated in groups trading different symbols, can serve the purpose of applying the required test load to the system – the distribution of message load is unfortunately evenly balanced across the order book, which is atypical in the real life trading situations.

  2. To validate that the various feeds (Market Data, Regulatory, Surveillance, Drop-copy, etc.) operate consistently and meet performance targets, under variable high load trading conditions.

In a true production environment, the output from market feeds is non-predictive. The feed output will have a different message sequence and data content – even if, the trade orders for each client session are replayed with the same timed sequence. This is due to the effect of competing (asynchronous) trading actions from multiple client sessions, leading to a different order of trade matching between counter-parties. The order of matching is affected by many factors such as, order submission rates and the number of clients competing for the same trade.

  1. When the speed/capacity requirements change, the QA department must continually adjust their test strategy to apply a different stimulus setup to the trading system.

    Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run. It would be nice to keep the market feed verification setup the same –saving on the time and effort required to update the test strategy to meet the new requirement. Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run.

Test Strategy

Let us map the above mentioned MiFID II system test requirements into the requirements for an automated test environment capable of addressing those challenges.

We need a test environment that provides:

You need to login to view the rest of the content. Please . Not a Member? Join Us

Adaptive Testing of Capital Market Feeds improves Quality Assurance

Share Facebooktwittergoogle_pluspinterestlinkedinmail

Participants in today’s Capital Markets, consume an ever growing volume of real-time data
from multiple feeds in order to accurately report price and trade-related data for a range of
financial instruments.

Traders, Investors and Regulators, rely on the integrity of market feeds, to execute regulatory compliant
trades in equities, fixed-income products, derivatives and currencies and to maintain market stability.
It is therefore important that feed providers develop a comprehensive test strategy for feed verification
that detects any anomalies under all trading conditions.

Does your feed test strategy cover dynamic high load conditions?

The answer is probably – NO.
Most strategies for feed testing employ a functional test, covering typical trading scenarios. These test
setups are usually comprised of a small number of paired Buy/Sell counter-parties and the trade
message flows are strictly time-lined, with the low submission rates, to ensure synchronization of trade
actions and the repeatability of order matching.
These functional tests are not designed to cover high load conditions.

Why are “High Load” tests difficult to setup?

In a true production environment, the output from Market Feeds is non-predictive.
The output of a feed will have a different message sequence and data content, even if, the trade orders
for each trading session are replayed with the same timed sequence. This is due to the effect of
competing (asynchronous) trading actions from multiple client sessions leading to a different order of
trade matching between counter-parties. The order of matching is affected by factors such as, order
submission rates and the number of clients competing for the same trade.
Since the feed is non-predictive, comparing it output against a static data reference is not a feasible test
setup under high load conditions. The sequence of trades changes and the message content (i.e. price,
volume, etc.) varies between each test run.
This presents a significant challenge to the development of a comprehensive test strategy.

Introducing “Adaptive Testing” for Market Feeds

Adaptive testing uses trade information (i.e. Drop-Copy, Execution Reports, etc.) from within the trading
system, sourced in real-time, to create a dynamic reference stream to verify the messages on the feed
under test.
What differentiates adaptive testing from other approaches is the real-time nature of the reference
data, used to compare both feed message sequence and content.

Single Market Feed test setup covers all trade flow scenarios.

This test technique automatically adapts to different trade message input stimulus without the test
engineer having to change the feed test setup. A single adaptive test setup adjusts to any front-end test
scenario presented to the trading system and automatically verifies that the back-end feed output is
correct.
Adaptive testing can be used to support real-time monitoring of data feeds in both development and
production environments – guarding against message drop-outs, malformed messages and inconsistent
data.
Inceptrum has incorporated this adaptive technique in its new test tool, called Feed Comparator
(“FEEDCMP”) for market data, regulatory and drop-copy feeds.
To learn more about data feed testing, follow the link to Feed Comparator (“FEEDCMP”)
at  www.inceptrum.com/en/feedcmp-2/

 

About INCEPTRUM

INCEPTRUM Technologies Inc. is a Canadian privately held company, located in Ottawa, Ontario, Canada. Providing creative, leading-edge, low data latency solutions for the financial industry.
For more information, visit www.inceptrum.com.

Media Contact:

Ian Hughes,
Dir. Business Development,
Inceptrum Technologies Inc.
ian.hughes@inceptrum.com
1-613.699.2016 (direct)
1-877.763.6996 x103 (toll-free)

IIROC DATA Feed CONTENT Verification Using Adaptive Testing

The client

A large Canadian Exchange (based in Toronto), servicing a wide cross section of members of the financial industry.

The Exchange wanted to expand its QA test coverage related to its regulatory (IIROC) data feed, by adding test strategies to verify the data content of trading messages and the transmission sequence of message, when trading under high load conditions.

The integrity of the regulatory data feed is key to facilitating trading transparency and maintaining the business reputation of the Exchange. Any inaccurate data transmission to the regulator, would lead to fines and other penalties, and have a direct impact directly on their reputation.

 

The challenge

To validate that the regulatory data feed operates consistently and meets performance targets under variable high load trading conditions.

In a true production environment, the output from regulatory data feed is non-predictive.

The regulatory data feed will have different message sequence and data content – even if, the trade orders for a trading session are replayed with the same timed sequence, due to the effect of competing (asynchronous) trading actions from multiple client sessions, leading to a different order of trade matching between counter-parties. Matching order is affected by many dynamic factors, such as order submission rates and the number of clients competing for the same trade.

Therefore, since the regulatory data feed is non-predictive, comparing its output against a static data reference is not a valid test setup. The sequence of trades changes and the message content varies between each trading session, when running under high load conditions.

This presents a significant challenge to the development a comprehensive regulatory feed test strategy.

 

Our solution

You need to login to view the rest of the content. Please . Not a Member? Join Us

Exchange Data Feed Verification Using Adaptive Testing

The client

Is a leading Canadian Exchange based in Toronto, servicing a wide and growing cross section of members from the financial industry, and have engaged INCEPTRUM on numerous occasions to provide enhanced test solutions for its trading network.

On this occasion the Exchange was preparing to deploy its next generation trading system, to improve trading performance (by reducing latency) and maintain their market presence as a cost effective and leading high performance trading venue.

The integrity of the market Data Feed is of paramount importance to their business reputation and market data volume and rates were expected to increase due to the improved performance of the new trading system.

The challenge

To validate (in real-time) that their Market Data Feed operates consistently and meets the new performance targets under variable high load trading conditions.

In a true production environment, the output from Market Data Feed is non-predictive.

The feed output will have a different message sequence and data content – even if, the trade orders for each trading session are replayed with the same timed sequence. This is due to the effect of competing (asynchronous) trading actions from multiple client sessions leading to a different order of trade matching between counter-parties. Order of matching is affected by many factors such as, order submission rates and the number of clients competing for the same trade.

Since the data feed is non-predictive, comparing its output against a static data reference is not a feasible test setup, when running under high load conditions. The sequence of trades changes and the message content (i.e. price, volume, etc.) varies between each test run.

This presents a significant challenge to the development of a comprehensive test strategy.

Our solution

You need to login to view the rest of the content. Please . Not a Member? Join Us

Leading Canadian Exchange Adopts New Market Feed Verification Solution From Inceptrum Technologies Inc.

Share Facebooktwittergoogle_pluspinterestlinkedinmail

 

INCEPTRUM Technologies Inc., a solutions provider to the capital markets, today announced the deployment of its automated verification tool, called “FEEDCMP”, at a leading Canadian Exchange, enhancing quality assurance testing of Market Data and Regulatory (IIROC) Feeds.

FEEDCMP is the latest addition to a suite of automated test tools from INCEPTRUM and employs a new concept called “Adaptive Testing” for the verification of Market Feeds under variable load conditions.

This new verification approach uses information such as, Drop-Copy and Execution Reports, sourced in real-time from the trading system, to create a dynamic data reference with which to verify the feed under test.

FEEDCMP automatically adapts to different trade flows, removing the need for a test engineer to support multiple feed test setups for different trading scenarios. A single test setup in FEEDCMP is all you require to support any test flow presented to the trading system and verify that multiple back-end feeds are running as expected.

Dmitry Kursov, President & CTO, Inceptrum said, “We are very pleased to have deployed FEEDCMP at a leading Canadian Exchange, less than a month after its official launch. Our “Adaptive Testing” concept provides a much-needed solution for verifying Market Feed output under high load conditions — which is not feasible using traditional functional test setups, due to the effect of asynchronous trading actions from multiple client sessions, producing a non-predictive order of trade matching between counter-parties. Comparing feed output against a static data reference is therefore not an appropriate test approach under high load conditions, due to changes in trade message sequence and content (i.e. price, volume, etc.) for each test run.”

FEEDCMP can be deployed for non-intrusive, real-time monitoring in both development and production trading environments guarding against message drop-outs, malformed messages and inconsistent data content on your Market Feeds.

Please visit https://www.inceptrum.com/en/feedcmp-2/ for more information on this product.

 

About INCEPTRUM

INCEPTRUM Technologies Inc. is a Canadian privately held company, located in Ottawa, Ontario, Canada. Providing creative, leading-edge, low data latency solutions for the financial industry.
For more information, visit www.inceptrum.com.

Media Contact:

Ian Hughes,
Dir. Business Development,
Inceptrum Technologies Inc.
ian.hughes@inceptrum.com
1-613.699.2016 (direct)
1-877.763.6996 x103 (toll-free)

Inceptrum introduces Feed Comparator (FEEDCMP) for automated market feed testing

Share Facebooktwittergoogle_pluspinterestlinkedinmail

 

Ottawa, Canada, May 18, 2017 – INCEPTRUM Technologies Inc., a solutions provider to the capital markets, today announced the release of a new verification tool called “Feed Compare” (FEEDCMP) to its existing suite of test automation tools.

FEEDCMP is a fully customizeable feed comparator service, automating testing of the Regulatory, Drop Copy and Market Data feeds.

Dmitry Kursov, President & CTO, Inceptrum said, “We believe that FEEDCMP provides a new and innovative approach to automated testing of a trading system data feeds and improves quality assurance testing. ”

Please visit https://www.inceptrum.com/en/feedcmp-2/ for more information on this product.

About Inceptrum Technologies Inc

INCEPTRUM Technologies Inc. is a Canadian privately held company, located in Ottawa, Ontario, Canada, providing software and hardware solutions to a wide range of business sectors. The company specializes in providing creative, leading-edge, low data latency solutions for the financial industry.

www.inceptrum.com

# # #

Media Contact

Ian Hughes
Business Development
613.699.2016 (direct)
1.877.763.6996 x103 (toll-free)
ian.hughes@inceptrum.com


Document: PR170518-1

Inceptrum introduces STUDIO a test case development and results analysis tool

Share Facebooktwittergoogle_pluspinterestlinkedinmail

Ottawa, Canada, May 18, 2017 – INCEPTRUM Technologies Inc., a solutions provider to the capital markets, today announced the introduction of new product called “STUDIO”, as a companion product to its existing suite of test automation tools.

STUDIO provides a test engineer with an integrated development and test results analysis environment that provides,

  • for the graphical (and text editor) development of test cases scenarios,
  • the facility for point and click execution of a test case,
  • easy integration with commercial source control systems (Git, Subversion, etc.),
  • built-in test result analysis and presentation of,
    • log files, tabular reports and charting
    • per message type statistics (i.e. for latency, rates, etc.)
    • reporting by individual client session
    • summary reports

STUDIO can be installed on Windows, Linux, Mac and Solaris O.S. platforms.

Please visit https://www.inceptrum.com/en/studio/ for more information on this product.

About Inceptrum Technologies Inc

INCEPTRUM Technologies Inc. is a Canadian privately held company, located in Ottawa, Ontario, Canada, providing software and hardware solutions to a wide range of business sectors. The company specializes in providing creative, leading-edge, low data latency solutions for the financial industry.

www.inceptrum.com

# # #

Media Contact

Ian Hughes
Business Development
613.699.2016 (direct)
1.877.763.6996 x103 (toll-free)
ian.hughes@inceptrum.com


Document: PR170518-3

Inceptrum releases TESTBENCH 1.1.0 for its automated test tool suite

Share Facebooktwittergoogle_pluspinterestlinkedinmail

Ottawa, Canada, Mar 18th, 2017 – INCEPTRUM Technologies Inc., a solutions provider to the capital markets, today announced the release of a new version of its verification tool called “TESTBENCH” (v1.1.0) for its suite of test automation tools.

This latest software release provides:

  • Support for Multicast UDP
  • Support for native trading protocols (i.e. ITCH/OUCH etc.)

Please visit https://www.inceptrum.com/en/testbench/ for more information on this product.

About Inceptrum Technologies Inc

INCEPTRUM Technologies Inc. is a Canadian privately held company, located in Ottawa, Ontario, Canada, providing software and hardware solutions to a wide range of business sectors. The company specialises in providing creative, leading-edge, low data latency solutions for the financial industry.

www.inceptrum.com

# # #

Media Contact

Ian Hughes
Business Development
613.699.2016 (direct)
1.877.763.6996 x103 (toll-free)
ian.hughes@inceptrum.com


Document: PR180318-2