[Back to Intro]

PUBLICATIONS

There is nothing more practical than a good theory. Many models successfully applied in scientific fields are tremendously useful in addressing critical investment management problems. I have been fortunate enough to meet and work extensively with some of the leading figures in Pure Mathematics, Mathematical Finance, Machine Learning, Market Microstructure and Econometrics. The majority of our findings are kept proprietary. From time to time, however, we decide to publish some of them, hence the irregular frequency of the following publications.

 

  

 

  SSRN Big Data Research Series   QuantumForQuants  

 

 

 

RECENT PEER-REVIEWED JOURNAL ARTICLES

 

AUTHORS

TITLE

REFERENCE

INDEX

NOTABLE INNOVATION

Rosenberg, Gili; Poya Haghnegahdar; Goddard, Phil; Lopez de Prado, Marcos; Carr, Peter; Wu, Kesheng

Solving the Optimal Trading Trajectory Problem Using a Quantum Annealer

IEEE Journal of Selected Topics in Signal Processing, 2016. Forthcoming.

JCR (5Y IF = 3.681)

MathSciNet; Zentralblatt MATH

We solve a multi-period portfolio optimization NP-complete problem using D-Wave's quantum annealer. The formulation incorporates transaction costs (including permanent and temporary market impact) and, significantly, the solution does not require the inversion of a covariance matrix. The discrete multi-period portfolio optimization problem we solve is significantly harder than the continuous variable problem. 

Bailey, David H.; Borwein, Jon M.; Lopez de Prado, Marcos; Zhu, Jim

The Probability of Backtest Overfitting

Journal of Computational Finance, 2016. Forthcoming.

JCR (1Y IF = 0.382)

MathSciNet; Zentralblatt MATH

We propose a framework that estimates the probability of backtest overfitting (PBO) specifically in the context of investment simulations, through a numerical method that we call combinatorially symmetric cross-validation (CSCV). We show that CSCV produces accurate estimates of the probability that a particular backtest is overfit. Mathematical Appendices available here.

Lopez de Prado, Marcos

Mathematics and Economics: A reality check

Journal of Portfolio Management, 43(1). Fall 2016.

JCR (5Y IF = 0.562) Economics (and by extension finance) is arguably one of the most mathematical fields of research. However, economists’ choice of math may be inadequate to model the complexity of social institutions. In a constructive spirit, this note offers some advice on how students could increase their chances of having a successful career in 21st century finance. Practitioners seeking to enhance their skillset may also draw some ideas.

Lopez de Prado, Marcos; Rebonato, Riccardo

Kinetic Component Analysis

Journal of Investing, 2016. Forthcoming.

 

We introduce Kinetic Component Analysis (KCA), a state-space application that extracts the signal from a series of noisy measurements by applying a Kalman Filter on a Taylor expansion of a stochastic process. We show that KCA presents several advantages compared to other popular noise-reduction methods such as Fast Fourier Transform (FFT) or Locally Weighted Scatterplot Smoothing (LOWESS).

Lopez de Prado, Marcos

Building Diversified Portfolios that Outperform Out-Of-Sample

Journal of Portfolio Management, 42(4), pp. 59-69. Summer 2016.

JCR (5Y IF = 0.562) HRP portfolios address three major concerns of quadratic optimizers in general and Markowitz’s CLA in particular: Instability, concentration and underperformance. Monte Carlo experiments show that HRP delivers lower out-of-sample variance than CLA, even though minimum-variance is CLA’s optimization objective. HRP also produces less risky portfolios out-of-sample compared to traditional risk parity methods.

Lopez de Prado, Marcos

Algorithmic and High Frequency Trading

Quantitative Finance, 16(8), pp. 1175-1176, 2016.

JCR (5Y IF = 0.957)

MathSciNet; Zentralblatt MATH; Scopus

A review of the monograph recently published by Cambridge University Press.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen

Discerning Information From Trade Data

Journal of Financial Economics, 120(2), pp. 269-286. May 2016.

JCR (5Y IF = 5.876) We examine the accuracy of three methods for classifying trade data: Bulk Volume Classification (BVC), Tick Rule and Aggregated Tick Rule. We develop a Bayesian model of inferring information from trade executions, and show that BVC has the highest explanatory power over the bid-ask spread.

Bailey, David H.; Borwein, Jon M.; Salehipour, Amir;  Lopez de Prado, Marcos; Zhu, Jim

Backtest Overfitting in Financial Markets

Automated Trader, Issue 39. Spring 2016.

  We introduce two online backtest overfitting tools: BODT simulates the overfitting of seasonal strategies (typical of technical analysis), and TMST simulates the overfitting of econometric strategies (typical of academic journals). We show that econometric methods lend themselves to extreme levels of overfitting, casting doubt on most investment strategies published in academic journals.

Bailey, David H.;
Lopez de Prado, Marcos

Stop-Outs Under Serial Correlation and "The Triple Penance Rule"

Journal of Risk, 18(2), pp.61-93. Fall 2015.

JCR (5Y IF = 1.794)

We develop a framework for informing the decision of stopping a portfolio manager or investment strategy once it has reached a loss or time under water limit for a certain confidence level. Under standard portfolio theory assumptions, we show that it takes three times longer to recover from the expected maximum drawdown than the time it takes to produce it, with the same confidence level. Mathematical Appendices available here.

Lopez de Prado, Marcos

Recent Trends in Empirical Finance

Journal of Portfolio Management, 42(1), pp. 29-33. Fall 2015.

JCR (5Y IF = 0.562) Financial economics is a surprisingly prolific, topic redundant, asocial field, where most papers go largely ignored. Author collaboration improves scientific output, and yet financial economics seems to be one of the least cooperative empirical fields. If these trends continue, financial economics may be in the path to become a pathological science, a collection of “cold fusion” claims.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen

Optimal Execution Horizon

Mathematical Finance, 25(3), pp. 640-672. July 2015.

JCR (5Y IF = 1.662)

MathSciNet; Zentralblatt MATH

Execution traders know that market impact greatly depends on whether their orders lean with or against the market. And yet, the literature on optimal execution strategies rarely incorporates order imbalance in the modeling of transaction costs. We introduce the OEH model, which considers this fact when determining the optimal trading horizon for an order, an input required by many sophisticated execution strategies.

Lopez de Prado, Marcos

The Future of Empirical Finance

Journal of Portfolio Management, 41(4), pp. 140-144. Summer 2015.

JCR (5Y IF = 0.562) Empirical Finance is in crisis: Our most important discovery tool is historical simulation, and yet, most backtests and time series analyses published in journals are flawed. The problem is well-known to professional organizations of Statisticians and Mathematicians, who have publicly criticized the misuse of mathematical tools among Finance researchers. In this note I point to three problems and propose four practical solutions. An interview on this research appeared in IIJ's Practical Applications (Winter 2016).

Lopez de Prado, Marcos

Quantitative Meta-Strategies

Practical Applications, Institutional Investor Journals, 2(3), pp. 1-3, Winter 2014.

 

Quantitative Meta-Strategies (QMS) are quantitative strategies designed to manage investment strategies. As a field, QMS can be defined as the mathematical study of the decisions made by the supervisor of a team of investment managers, regardless of whether their investment style is systematic or discretionary.

Bailey, David H.; Borwein, Jon M.; Lopez de Prado, Marcos; Zhu, Jim

Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-Of-Sample Performance

Notices of the American Mathematical Society, 61(5), pp. 458-471. May 2014.

MathSciNet; Zentralblatt MATH

We prove that high simulated performance is easily achievable after backtesting a relatively small number of alternative strategy configurations, a practice we denote “backtest overfitting”. Because financial analysts rarely report the number of configurations tried for a given backtest, investors cannot evaluate the degree of overfitting in most investment proposals. This is one of the first Mathematical Finance papers published in the Notices of the AMS, the official membership journal of the American Mathematical Society.

Lopez de Prado, Marcos;
Foreman, Matthew

A Mixture of Gaussians Approach to Mathematical Portfolio Oversight: The EF3M Algorithm

Quantitative Finance, 14(5), pp. 913-930. 2014.

JCR (5Y IF = 0.957)

MathSciNet; Zentralblatt MATH; Scopus

We solve the "Nonic Polynomial problem" posed by Karl Pearson in the 1894 edition of the Philosophical Transactions of the Royal Society. We apply quantitative methodologies originated in the Mathematical Theory of Evolution to model the dynamics of investment styles within a fund.

Bailey, David H.;
Lopez de Prado, Marcos

The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality

Journal of Portfolio Management, 40 (5), pp. 94-107. 2014 (40th Anniversary Special Issue).

JCR (5Y IF = 0.562) The Deflated Sharpe Ratio (DSR) corrects for two leading sources of performance inflation: Selection bias under multiple testing and non-Normally distributed returns. In this interview, Prof. Bailey speaks about our work. IIJ's Practical Applications (Winter 2015) featured this work.

Other authors in this JPM Special Issue include: Cliff Asness (AQR), John Bogle (Vanguard), Mohamed El-Erian (Allianz), Robert Kapito (BlackRock), Mark Kritzman (Windham), Martin Leibowitz (Morgan Stanley), Burton Malkiel (Princeton) and Marc Reinganum (State Street).

Calkin, Neil; Lopez de Prado, Marcos

Stochastic Flow Diagrams

Algorithmic Finance, 3(1), pp. 21-42. 2014.

We introduce Stochastic Flow Diagrams (SFDs), a new mathematical approach to represent complex dynamic systems into a single weighted digraph. This topological representation provides a way to visualize what otherwise would be a morass of equations in differences.

Proceedings SC'14

Song, Jung H.; Lopez de Prado, Marcos; Simon, Horst D.; Wu, Kesheng

Exploring Irregular Time Series Through the Non-Uniform Fourier Transform Proceedings of the International Conference for High Performance Computing, IEEE, 2014.   We explore the frequency domain of irregular time series by applying a Non-Uniform Fast Fourier Transform (NUFFT) on Natural Gas Futures prices. We show that High-Frequency Traders are responsible for a growing number of cyclical patterns. In particular, we observe the emergence of a new power law in the Fourier spectra in recent years.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen

VPIN and the Flash Crash: A Rejoinder

Journal of Financial Markets, 17(1), pp. 47-52. 2014.

JCR (5Y IF = 1.505)

Discusses implementation cautions with regards to VPIN empirical studies.

Calkin, Neil; Lopez de Prado, Marcos

The Topology of Macro Financial Flows: An Application of Stochastic Flow Diagrams

Algorithmic Finance, 3(1), pp. 43-85. 2014.

We construct a network of financial instruments and show how Stochastic Flow Diagrams (SFDs) allow researchers to monitor the flow of capital across the financial system. Because our approach is dynamic, it models how and for how long a financial shock propagates through the system.

Bailey, David H.;
Lopez de Prado, Marcos

An Open-Source implementation of the Critical-Line Algorithm for Portfolio Optimization

Algorithms, 6(1), pp. 169-196. 2013.

MathSciNet; Zentralblatt MATH; Scopus-SciVerse

We fill a gap in the literature by providing a well-documented, step-by-step open-source implementation of the Critical-Line Algorithm (CLA) in a scientific language. We discuss the logic behind CLA following the algorithm’s decision flow. In addition, we have developed several utilities that facilitate the answering of recurrent practical problems.

Bailey, David H.;
Lopez de Prado, Marcos

The Strategy Approval Decision: A Sharpe Ratio Indifference Curve Approach

Algorithmic Finance, 2(1), pp. 99-109. 2013.

The problem of capital allocation to a set of strategies could be partially avoided, or at least greatly simplified, with an appropriate strategy approval decision process. This paper proposes such procedure, by splitting the capital allocation problem into two sequential stages: Strategy approval and portfolio optimization.

Bailey, David H.;
Lopez de Prado, Marcos

The Sharpe Ratio Efficient Frontier

Journal of Risk, 15(2), pp. 3-44, Winter. 2012.

JCR (5Y IF = 1.794)

Introduced the Probabilistic Sharpe Ratio (PSR), a new uncertainty-adjusted investment skill metric that corrects the inflationary effect that Non-Normality has on Sharpe Ratio estimates. It also determines the Minimum Track Record Length (MinTRL) needed to evidence skill. A Sharpe Ratio Efficient Frontier (SEF) arises, based on return-on-risk rather than return-on-capital.

Bailey, David H.;
Lopez de Prado, Marcos

Balanced Baskets: A new approach to Trading and Hedging Risks

Journal of Investment Strategies, 1(4), pp. 21-62. Fall, 2012.

 

Introduced the notion of Balanced Baskets, which are portfolios of instruments that evenly spread risks or exposures across their constituents without requiring a change of basis, like PCA. It also developed the algorithms needed to compute such baskets in hedging as well as trading applications. Finally, it also contributed a new procedure for covariance clustering.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen

The Volume Clock: Insights into the High Frequency Paradigm

Journal of Portfolio Management, 39(1), pp. 19-29. Fall, 2012.

JCR (5Y IF = 0.562)

This paper has been cited by Market Regulators [1, 2, 3] for deepening their understanding of the phenomenon of High Frequency Trading (HFT), beyond the simple notion of "speed trading". In particular, it argues that at the heart of HFT is a new investment paradigm based on making decisions in Volume Time. IIJ's Practical Applications (Fall 2013) featured this work.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen

Flow Toxicity and Liquidity in a High Frequency World

Review of Financial Studies, 25(5), pp. 1457-1493. 2012.

JCR (5Y IF = 5.367)

Developed a new procedure to estimate the flow toxicity impacting market makers, the Volume Synchronized Probability of Informed Trading (VPIN). This metric has been shown to anticipate liquidity crises (including the Flash Crash) and to be a good predictor of toxicity-induced volatility. CFTC's HFT guidelines cite this publication.

Lopez de Prado, Marcos;
Leinweber, David

Advances in Cointegration and Subset Correlation Hedging Methods

Journal of Investment Strategies, 1(2), pp. 67-115. Spring, 2012.

 

Introduced two new hedging methods, called Dickey-Fuller Optimization (DFO) and Mini-Max Subset Correlation (MMSC). The former is a dynamic, cointegration based method while the latter is a static, balanced-basket method to evenly distribute exposure among portfolio constituents. It also generalized the Box-Tiao Canonical Decomposition (BTCD) method.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen

The Exchange of Flow Toxicity

Journal of Trading, 6(2), pp. 8-13. Spring, 2011.

 

It introduced the concept of "Market Makers' Asymmetric Payoff Dilemma", which characterizes a liquidity provider as the seller of a real-option to be adversely selected. Since that option cannot be dynamically replicated, a new contract is proposed to allow market makers to hedge such risks.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen

The Microstructure of the ‘Flash Crash’: Flow Toxicity, Liquidity Crashes and the Probability of Informed Trading

Journal of Portfolio Management, 37(2), pp. 118-128. Winter, 2011.

JCR (5Y IF = 0.562)

This has become one of the most read papers in Finance, according to SSRN. It analyses the "Flash Crash" from a microstructure perspective, and concludes that it was a liquidity crises which resulted from market makers receiving persistently toxic order flow for at least 2 hours before the crash actually unfolded.

Lopez de Prado, Marcos;
Peijan, Achim

Measuring Loss Potential of Hedge Fund Strategies

Journal of Alternative Investments, 7(1), pp. 7-31. Summer, 2004.

 

It developed a new risk framework for assessing hedge funds' loss potential, considering Non-Normal and Serially-Correlated returns. It shows that the IID Normal assumption, ubiquitous in financial risk modeling, leads to a great underestimation of the loss potential of hedge funds.

 

PEER-REVIEWED ACADEMIC BOOKS

 

AUTHORS

TITLE

REFERENCE

NOTABLE INNOVATION

 

Lopez de Prado, Marcos (Ed.) Special Guide to Applied Data Sciences in Finance Institutional Investor Jourmals, 2016 (Forthcoming). Today, scientists model financial markets as true complex dynamic systems, applying methodologies borrowed from all areas of science and engineering. Whether it is signal processing, network analysis or data visualization, modern methods can help us answer fundamental questions that traditional econometric methods have failed to tackle over decades.

Risk-Based and Factor Investing

Bailey, David H.; Ger, Stephanie; Lopez de Prado, Marcos; Sim, Alexander; Wu, Kesheng Statistical Overfitting and Backtest Performance, in Risk-Based and Factor Investing Quantitative Finance Elsevier, 2016 (Forthcoming). This book (edited by Emmanuel Jurczenko) is a compilation of recent articles written by leading academics and practitioners in the area of risk-based and factor investing (RBFI). The articles are intended to introduce readers to some of the latest, cutting edge research encountered by academics and professionals dealing with RBFI solutions. Together the authors detail both alternative non-return based portfolio construction techniques and investing style risk premia strategies.

Easley, David;
Lopez de Prado, Marcos;
O'Hara, Maureen
(Coord.)

High Frequency Trading: New Realities for Traders, Markets and Regulators

Risk Books,  2013.

An overview of high frequency trading (HFT) strategies, with a particular focus on how low frequency traders can survive in a high frequency world.

Contributors include leading practitioners and academics in this field: Robert Almgren (Quantitative Brokers, New York University), Wes Bethel (Lawrence Berkeley National Laboratory), Ming Gu (Lawrence Berkeley National Laboratory), Terry Hendershott (U.C. Berkeley), Charles Jones (Columbia University), Michael Kearns (S.A.C. Capital, University of Pennsylvania), David Leinweber (Lawrence Berkeley National Laboratory), Oliver Linton (University of Cambridge), Albert Menkveld (University of Amsterdam), Yuryi Nevmyvaka (University of Pennsylvania), Richard Olsen (Olsen Ltd.), Oliver Ruebel (Lawrence Berkeley National Laboratory), George Sofianos (Goldman Sachs), Michael Sotiropoulos (Bank of America Merrill Lynch), Kesheng Wu (Lawrence Berkeley National Laboratory), and Jean-Pierre Zigrand (London School of Economics).

Lopez de Prado, Marcos

Advances in High Frequency Strategies

Complutense University, 2011.

This is the author's second doctoral dissertation. The generalization of electronic markets and ubiquitous automation of financial transactions has rendered many established models and theories obsolete. This work presents a new scientific framework for the study of some of the most relevant questions concerning High Frequency Trading.

Lopez de Prado, Marcos

Invertir en Hedge Funds

Díaz de Santos, 2003.

This is the author's first doctoral dissertation, which dealt with portfolio optimization, risk management and capital allocation to hedge funds. Once hedge funds' hidden risks are taken into account, optimal allocations are much smaller than proposed by the standard Markowitz approach.

 

WORKING PAPERS AND BOOKS

AUTHORS

YEAR

TITLE

NOTABLE INNOVATION

Bailey, David H.; Borwein, Jon M.; Lopez de Prado, Marcos

2016

Stock portfolio design and backtest overfitting

We demonstrate a computer program that designs a portfolio consisting of common securities, such as the constituents of the S&P 500 index, that achieves any desired profile via in-sample backtest optimization. Unfortunately, the program also shows that these portfolios typically perform erratically on more recent, out-of-sample data, which is symptomatic of selection bias. One implication of these results is that so-called smart beta funds, which are designed in-sample to deliver a desirable performance pro file, are likely to disappoint out-of-sample.

Song, Jung H.; Lopez de Prado, Marcos; Simon, Horst D.; Wu, Kesheng

2015

Intraday Patterns in Natural Gas Futures: Extracting Signals from High-Frequency Trading Data

As algorithms replace an increasing number of tasks previously performed by humans, cascading effects similar to the Flash Crash of May 6th 2010 become more likely. A case in point are the impact of weather forecasts on Natural Gas trading. The Fourier components corresponding to high frequencies are becoming more prominent in the recent years and are much stronger than could be expected from the structure of the market.

Bailey, David H.; Borwein, Jon M.; Salehipour, Amir;  Lopez de Prado, Marcos; Zhu, Jim

2015

Backtest Overfitting Demonstration Tool: An Online Interface

In this study we introduce two online tools, the Backtest Overfitting Demonstration Tool, or BODT for short, and the Tenure Maker Simulation Tool, or TMST, which illustrate the impact of backtest overfitting on investment models and strategies. We describe BODT and TSMT, the experiments they perform, together with technical details such as the evaluation metrics and parameters used.

Lopez de Prado, Marcos

2015

Generalized Optimal Trading Trajectories: A Financial Quantum Computing Application

Generalized dynamic portfolio optimization problems have no known closed-form solution. These problems are particularly relevant to large asset managers, as the costs from excessive turnover and implementation shortfall may critically erode the profitability of their investment strategies. In this brief note we demonstrate how this financial problem, intractable to modern supercomputers, can be reformulated as an integer optimization problem. Such representation makes it amenable to quantum computers.

Carr, Peter; Lopez de Prado, Marcos

2014

Determining Optimal Trading Rules without Backtesting

We present empirical evidence of the existence of optimal trading rules (OTRs) for the case of prices following a discrete Ornstein-Uhlenbeck process, and show how they can be computed numerically. Although we do not derive a closed-form solution for the calculation of OTRs, we conjecture its existence on the basis of the empirical evidence presented. Also available in ArXiv.

Lopez de Prado, Marcos; Vince, Ralph; Zhu, Jim

2013

Optimal Risk Budgeting Under a Finite Investment Horizon

Growth Optimal Portfolio (GOP) theory determines the path of bet sizes that maximize long-term wealth. This multi-horizon goal makes it more appealing among practitioners than myopic approaches, like Markowitz's mean-variance or risk parity. The GOP literature typically considers risk-neutral investors with an infinite investment horizon. In this paper, we compute the optimal bet sizes in the more realistic setting of risk-averse investors with finite investment horizons.