Eduzan / AI & Machine Learning
AI and ML in Practice
1. Model Selection:
- Definition: The process of choosing the most suitable machine learning model for a given dataset and problem.
- Purpose: Different models have different strengths, weaknesses, and assumptions. Selecting the right model helps in achieving better performance.
- Example: Deciding between a decision tree, support vector machine (SVM), or a neural network for a classification task.
2. Hyperparameter Tuning:
- Definition: The process of optimizing the hyperparameters of a machine learning model to improve its performance.
- Purpose: Hyperparameters control the behavior of the training algorithm and model complexity. Proper tuning can significantly enhance model accuracy and generalization.
- Techniques:
- Grid Search: Exhaustive search over a specified parameter grid.
- Random Search: Randomly sampling hyperparameters from a specified distribution.
- Bayesian Optimization: A probabilistic model-based optimization approach.
- Example using Scikit-learn:
from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier
# Define the model and hyperparameters to tune
model = RandomForestClassifier()
param_grid = {
'n_estimators': [100, 200, 300],
'max_depth': [None, 10, 20, 30],
'min_samples_split': [2, 5, 10]
}
# Perform grid search
grid_search = GridSearchCV(estimator=model, param_grid=param_grid, cv=5)
grid_search.fit(X_train, y_train)
# Best hyperparameters
print("Best parameters found: ", grid_search.best_params_)Cross-Validation:
- Definition: A technique for assessing how a machine learning model generalizes to an independent dataset. It involves splitting the data into multiple folds and training/evaluating the model on each fold.
- Types:
- K-Fold Cross-Validation: Divides the data into
ksubsets (folds) and trains the modelktimes, each time using a different fold as the validation set. - Stratified K-Fold: Ensures that each fold has a representative distribution of the target variable.
- Leave-One-Out Cross-Validation (LOOCV): A special case where
kequals the number of data points.
- K-Fold Cross-Validation: Divides the data into
- Example:
from sklearn.model_selection import cross_val_score
from sklearn.svm import SVC
model = SVC(kernel='linear')
scores = cross_val_score(model, X, y, cv=5)
print("Cross-validation scores: ", scores)2. Model Evaluation Techniques:
- Definition: Methods used to assess the performance of a model on unseen data.
- Common Metrics:
- Accuracy: Proportion of correctly predicted instances.
- Precision, Recall, F1-Score: Useful for imbalanced datasets.
- ROC-AUC: Area under the Receiver Operating Characteristic curve, useful for binary classification.
- Mean Absolute Error (MAE), Mean Squared Error (MSE): Used for regression tasks.
1. Cloud Deployment:
- Definition: Hosting machine learning models on cloud platforms like AWS, Google Cloud, or Azure.
- Use Case: Scalable solutions where the model can be accessed via APIs for real-time predictions.
- Example: Deploying a TensorFlow model on AWS Sagemaker.
2. Edge Computing:
- Definition: Deploying machine learning models on edge devices like smartphones, IoT devices, or embedded systems.
- Use Case: Real-time predictions in environments with limited or no internet connectivity, like self-driving cars or smart cameras.
- Example: Deploying a TensorFlow Lite model on a Raspberry Pi for image classification.
1. TensorFlow:
- Definition: An open-source machine learning framework developed by Google, widely used for building and deploying neural networks.
- Features: Supports deep learning, distributed training, and deployment on various platforms including cloud, mobile, and edge devices.
- Example:
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
model = Sequential([
Dense(128, activation='relu', input_shape=(784,)),
Dense(64, activation='relu'),
Dense(10, activation='softmax')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=10)2. PyTorch:
- Definition: An open-source machine learning framework developed by Facebook, known for its dynamic computational graph and ease of use in research.
- Features: Ideal for building deep learning models, especially in NLP and computer vision.
- Example
import torch
import torch.nn as nn
import torch.optim as optim
class SimpleNN(nn.Module):
def __init__(self):
super(SimpleNN, self).__init__()
self.fc1 = nn.Linear(784, 128)
self.fc2 = nn.Linear(128, 64)
self.fc3 = nn.Linear(64, 10)
def forward(self, x):
x = torch.relu(self.fc1(x))
x = torch.relu(self.fc2(x))
x = torch.softmax(self.fc3(x), dim=1)
return x
model = SimpleNN()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters())
# Training loop
for epoch in range(10):
optimizer.zero_grad()
output = model(X_train)
loss = criterion(output, y_train)
loss.backward()
optimizer.step()3. Scikit-learn:
- Definition: A simple and efficient tool for data mining and data analysis in Python, built on NumPy, SciPy, and matplotlib.
- Features: Provides a range of tools for model selection, evaluation, and preprocessing.
- Example:
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
# Load dataset
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.3)
# Train model
model = RandomForestClassifier()
model.fit(X_train, y_train)
# Evaluate
accuracy = model.score(X_test, y_test)
print("Model accuracy:", accuracy)End of lesson.