Full Cycle of a ML project

1. define project
2. define and collect data (Data Sampling) + Data Preprocessing & Feature Engineering
3. train model: training, Error analysis & iterative improvement -> loop between 2 and 3 until your model is done
4. deploy in production (Machine Learning Systems Design): deploy, monitor, and maintain system -> back to 3 and/or 2 if needed

ML Algorithms Cheat Sheet

ML+Algorithms+Cheat+Sheet.pdf

Learning Paradigms

Where does feedback come from?

Supervised Learning

Classification

• linear
  • Logistic Regression
  • Support Vector Machine SVM: Support Vector X#Support Vector Machine SVM
• non-linear
  • Kernel SVM: Support Vector X#Kernels SVM
  • K-Nearest Neighbor (k-NN)
  • Naive Bayes
  • Decision Tree Classification: Decision Tree & Random Forest#Decision Tree
  • Random Forest Classification: Decision Tree & Random Forest#Random Forest
• Pros and Cons
  
• Multi-class vs. Multi-label Classification

Regression

• Types
  • Linear Regression
  • Polynomial Regression
  • Regularized regression
    • Lasso regression: Regression#Lasso Regression
    • Ridge regression: Regression#Ridge Regression
  • Support Vector Regression (SVR) Support Vector X#^46e7f9
  • Decision Tree Regression: Decision Tree & Random Forest#Decision Tree
  • Random Forest Regression: Decision Tree & Random Forest#Random Forest
• Pros and Cons
  

Unsupervised Learning

Clustering

• Types
  • Centroid-based Clustering: K-Means Clustering
  • Connectivity-based Clustering: Hierarchical Clustering
  • Density-based Clustering: DBSCAN
  • Graph-based Clustering: Affinity Propagation
  • Distribution-based Clustering: Gaussian Mixture Model
  • Compression-based Clustering: Spectral Clustering
• Pros and Cons

• Clustering Evaluation Metrics

Dimensionality Reduction

• Dimensionality Reduction#Feature Selection
• Dimensionality Reduction#Feature Extraction
  • Principal Component Analysis (PCA)
  • Linear Discriminant Analysis (LDA)
  • Kernel PCA
  • Quadratic Discriminant Analysis (QDA)
  • T-Distributed Stochastic Neighbor Embedding (t-SNE)
  • Uniform manifold approximation and projection (UMAP)
  • Autoencoders

Density Estimation

see Density Estimation

Anomaly Detection

see Outlier & Anomaly Detection

Association Rule Learning

• Apriori: Association Rule Learning#Apriori
• Eclat: Association Rule Learning#Eclat

Semi-Supervised Learning

• dataset: partially labeled, with some data points having labels and others being unlabeled
• how it works
  • use the labeled data to learn patterns and then generalize those patterns to the unlabeled data
  • minimizes the difference in predictions between similar training examples
• approaches
  • self-training
  • co-training
  • multi-view learning

Self-Supervised Learning

• algorithm learns from the data without explicit human annotations

• approaches
  • Contrastive Learning
  • Pretext-task Learning

Reinforcement Learning

Decision making

• Q-Learning: Reinforcement Learning#Q-Learning
• R Learning
• TD Learning

Upper Confidence Bound

see Reinforcement Learning#Upper Confidence Bound

Thompson Sampling

Reinforcement Learning#Thompson Sampling

Learning Settings

How is data received or labeled?

• Batch Learning
• Online Learning
• Active Learning
• Continual Learning

Generalization Strategies

How does knowledge move across tasks?

• Inductive Learning
• Transductive Learning
• Deductive Inference
• Transfer Learning
• Multi-Task Learning
• Few-Shot Learning
• Zero-Shot Learning
• Multi-Instance Learning

Modeling Frameworks

• Discriminative Learning
• Generative Learning
• Representation Learning
• Bayesian Learning
• Hebbian Learning

Model Families

Classical ML Models

• Linear models
• Trees
• SVM
• Naive Bayes
• k-NN

Ensemble Methods

• Bagging: Ensemble Learning#Bagging (Bootstrap Aggregating)
• Boosting: Ensemble Learning#Boosting
• Random Forest: Decision Tree & Random Forest#Random Forest
• Gradient Boosting

Deep Learning

• Overview: Key Components of Deep Learning
• Artificial Neural Networks
• Convolutional Neural Networks (CNN)
• Recurrent Neural Networks (RNN)
• Transformer
• Autoencoders
• Generative Adversarial Networks (GANs)
• Variational Autoencoders (VAE)

Model Selection & Improving

• Cross-Validation
• Hyperparameter Tuning
  • randomized search
  • grid search
• Error analysis
• Error Metrics
• Bias-Variance Tradeoff
• Underfitting vs. Overfitting
• Data Leakage

ML Model into Production

see Machine Learning Systems Design

• ML Model Deployment
• ML System Monitoring and Continual learning

Best Practice for ML

• Best Practices & Common Pitfalls in Machine Learning