Machine learning glossary

, ,
3 min.

Welcome!

This is a collection of posts about general machine learning concepts that I use as a my personal ML glossary.

What

I will try to summarize important terms and concepts of machine learning. The focus is on intuition but there will also be practical and theoretical notes.

Target Audience, this is for you if you:

  • Have a decent understanding of a concept but want more intuition.
  • Knew a term, but want to refresh your knowledge as it's hard to remember everything (that's me :sweat_smile: ).
  • Need to learn the important concepts in an efficient way. Students cramming for an exam: that's for you :four_leaf_clover: !

Why

Having a bad memory but being (at least considering myself to be :sweat_smile: ) a philomath who loves machine learning, I developed the habit of taking notes, then summarizing and finally making a cheat sheet for every new ML domain I encounter. There are multiple reasons I want to switch to a web-page:

  • Paper is not practical and prone to loss.
  • Thinking that someone I don't know (I'm talking about you :raising_hand: ) might read this post makes me write higher quality notes .
  • I'm forever grateful to people that spend time on forums and open source projects. I now want to give back to the community (The contribution isn't comparable, but I have to start somewhere :innocent: ).
  • Taking notes on a computer is a necessary step for my migration to CS :sweat_smile: .
  • As a wise man once said:
    You do not really understand something unless you can explain it to your grandmother. - Albert Einstein
    My grandma's are awesome :heart: but not really into ML (yet). You have thus been designated "volunteer" to temporarily replace them.

Notation

Keeping a uniform notation in machine learning is not easy as many sub-domains use different notations due to historical reasons. I will try using a uniform notation for all the glossary:

  • Lower case letters (a,b,c,…): scalars and functions.
  • Bold capital letters (A,B,C …): matrices.
  • Bold lower case letters (a,b,c …): vectors.
  • Capital letters (X,Y,Z…): either a random variable or a number of values an index can take (e.g. $k=1 \ldots K$)
  • Superscripts with $(n)$ (e.g. $x^{(n)}$) are used to denote one of the $T$ training examples. Superscripts with $(t)$ denote one of the $T$ test examples.

To make it easier to search the relevant information in the Glossary here is the color coding I will be using:

  • :bulb: Intuition
  • :wrench: Practical
  • :x: Disadvantage
  • :white_check_mark: Advantage
  • :school_satchel: Example
  • :mag: Side Notes
  • :wavy_dash: Compare to
  • :information_source: Resources

Disclaimer:

  • This is my first post ever :bowtie:, I would love to get your feedback.
  • I'm bad at spelling: Apologies in advance (feel free to correct me).
  • Check out the resources from where I got most of this information.
  • ML sub-domains overlap A LOT. I will try not to make the separations too artificial. Any suggestions would be appreciated :relaxed: . Note that I separate domains both by learning style and by algorithm similarity.
  • Many parts are still missing, come back soon for those.

Enough talking: prepare your popcorn and let us get going :clapper: !