Risk Analysis on the Fly: Fast Markets, Complex Portfolios

The Algorithmic Age has ushered in a level of capability never before seen in human history. The accomplishments have been impressive. At the core of it all is the ability to manage the markets’ most precious resource – data - with unprecedented speed, intelligence and agility. Capture, slice, dice, filter, sort, calculate, compare, analyze, decide and store it, all in less time than it takes a hummingbird to complete half a wing-stroke. Now, consider enough hummingbirds to cause an eclipse of the sun, and we begin to appreciate the volume, velocity and diversity of today’s data management landscape and the speed necessary to capture and use it.

Yet, the rest of the industry has not necessarily benefitted from the unprecedented amount of data and technology available to the black-box trader. In response to unprecedented market stresses in the past year, and subsequently based on recommendations from the Counterparty Risk Management Group (CRMPG), highly complex financial organizations, such as large multi-strategy hedge funds and bulge-bracket banks, need to enhance their data management capabilities, principally for purposes of on-demand portfolio risk analysis. This facility with data management, in large part, can be established by extending a unified and high-performance technical architecture to the entire enterprise.

This unified technical architecture needs to be designed for the improvement of data quality, dramatically higher data volumes, and much greater computational speed whether the underlying market conditions, strategy, or portfolio requirements warrant it today or not. The core components of this enhanced architecture already exist within those business units focusing on high-frequency algorithmic trading and sophisticated quantitative research.

The methods employed for data storage – of both streaming and historical data - are central to achieving enhanced enterprise data management (EDM) capabilities, particularly for the exploding volumes of time-series data. Whether these tools and methods already exist in-house or not, enhanced EDM capabilities are a critical requirement for achieving on-demand portfolio risk analysis of complex portfolios as well as multiple portfolios of varying degrees of complexity. Indeed, the difficulty of multi-portfolio risk analytics is effectively established by the most complex instruments held across the firm, which are typically of some structured or OTC variety.

Regardless of how tightly regulated the financial services industry becomes over the next several years, investors and trading partners will demand a new level of transparency and risk management from their money managers and counterparties. Financial institutions will need to support an on-demand reporting infrastructure, capable of reporting across asset classes, portfolios or business lines, as a standard cost of doing business.

While high-frequency trading strategies based on exchange-traded products have set the bar for being able to process, analyze and respond to market data, the requirement for more timely risk management is quickly spreading across the industry. As “complexity” and “financial innovation” become four-letter words in the pantheon of financial history, firms must be able to tame that complexity by breaking down instruments into byte-sized datasets that can be quickly and confidently valued.

The pursuit of an on-demand infrastructure – one that responds as fast as changes occur in the relevant underlying market(s) and occasionally predicts such changes - is the best strategy for taming complexity because it typically forces a new level of data and process automation (as well as a “data culture”). If your trading organization can respond on-demand, then it is likely that it can respond as fast as will ever be necessary.

To borrow a phrase: Impossible is nothing.

The TABB Group Vision Note on Risk Analysis On-the-Fly: Fast Markets, Complex Portfolios

This TABB Group Vision Note discusses the increasing demand for on-the-fly risk analysis, the importance of time-series to that risk analysis, the challenges associated with managing time-series data and various innovations in data management. It also discusses specific uses for time-series data, including quantitative research, algorithmic trading, and risk analysis on complex portfolios and the challenges in meeting the needs of those functions. We also discuss key characteristics firms should strive for in designing their enterprise data management strategy to ensure that data, the lifeblood of any trading enterprise, reaches as far and wide across the organization as possible. Finally, we cover various database techniques that improve the handling of time-series data, such columnar store, compression, encoding and data loading. The note is based on conversations and data obtained through interviews with technologists and business users at global investment banks, hedge funds and technology vendors.