Skip to content

Review: Machine Learning

March 27, 2011

Machine Learning: An Algorithmic PerspectiveThe following is a review of Machine Learning: An Algorithmic Perspective by Marsland.

Machine learning (ML) is one of those topics that elicits widely varying responses. Some folks think it’s rubbish for trading, recalling memories of failed early biologically-inspired AI techniques like neural nets. Other folks think it’s the holy grail (it’s not). And yet other folks are uncertain of its applicability, confounded by its multidisciplinary nature and corresponding lack of historic pedagogical coherence.

All of these perspectives are confounded by the fairly sophisticated mathematical tenor of modern ML books (e.g. Hastie et al. and Bishop; see How to Learn Algorithmic Trading: Part 2, for more details). This style causes two practical problems:

  • Cognitive burden: fairly heavy cognitive burden to deeply understand the methods. This is a huge issue vis-à-vis trading, as using these methods require understanding their intuition and heavily adapting the algorithms to solve specific trading problems (filtering, prediction, signaling, etc.)
  • Practical gap: significant gap between theory and working trading code; going from integrals to for-loops, no matter what language, simply takes work. This is magnified as ML often provides the biggest bang for the buck when sequentially combined with other quant building blocks

Given this context, the text provides a refreshing introduction to ML through a practical algorithmic lens: simply unlike other texts on ML, really shining at building conceptual intuition and providing intellectual breadcrumbs from idea to code for key ML themes. If you think in programmatic code and prefer lightweight mathematical machinery, this is a strong introduction text to ML.

A few spots in the text are exceptions to the generally strong tenor.

First, use of multi-layer perceptron as a motivating model feels dated (chapter 3). Perhaps this reflects stylistic bias in how to motivate ML: older texts derive motivation from classic AI (e.g. Russell and Norvig), while an increasing number of more recent texts derive motivation from statistical learning (i.e. regression, ranking, and classification). Both are valid perspectives, but their precedent insights are quite different.

Second, coverage of MCMC and graphical methods (chapters 14 and 15), are particularly rough spots and suffer from several problems:

  • Topical selection: idiosyncratic sliver of their respective domains, whose comprehension and effective application depends upon understanding the larger context
  • Mathematical sophistication: fairly heavy mathematical intuition and machinery required for these topics are less amendable to the prose and algorithmic style adopted by the text, and thus the provided examples and code do not do equal illustrative justice as in previous chapters
  • Prematurity: both topics currently have active research literatures for which brief summary may perhaps be premature

Consider Probabilistic Graphical Models by Koller and Friedman as an alternate text for graphical methods, albeit in a totally different prose style than this text.

About these ads
5 Comments leave one →
  1. andy permalink
    March 28, 2011 12:56 am

    Could you expand a little more on the sentence “…ML often provides the biggest bang for the buck when sequentially combined with other quant building blocks”? ML could be used for many of the various ‘quant building blocks’ discussed on this site. Where do you think it provides the best bang for the buck?

    • quantivity permalink*
      March 28, 2011 1:38 pm

      @andy: first inclination of many folks is to try and solve directional prediction using ML methods (e.g. applying particle filtering to prices). This inclination seems motivated by research papers which introduce a new ML technique and justify it via application to a prediction problem. In other words, everything looks like a nail when you are talking about a shiny new hammer.

      In contrast, a different way to consider ML is within context of solving a particular trading problem. For example, consider dataflow from ticker plant to analytic engine. Within this flow, filtering is a common requirement for tolerating dirty incoming data in both off- and online use cases. ML may be useful in this context (depending on the data), as this problem is more tricky in practice than it first appears.

  2. May 6, 2011 6:25 pm

    interesting review.

    I also don´t think the great strenght of ML algos are on the prediction side.

    Im constantly developing new algos for my trading, and almost none of them are specifically related to predicting prices.

  3. January 28, 2013 6:57 am

    @Dalton so what do you attempt to predict and how successful you are in doing it?

Trackbacks

  1. [Review] Machine Learning: An Algorithmic Perspective | [Caring is sharing]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 230 other followers

%d bloggers like this: