EZ

Eduzan

Learning Hub

Tutorials
Eduzan
Eduzan / Data Science

Advanced Machine Learning

Random Forests

Random forests are an ensemble learning method that builds multiple decision trees during training and outputs the mode of the classes (classification) or mean prediction (regression) of the individual trees. This helps reduce overfitting and improves the model’s accuracy and robustness.

  • Key Idea: Combines the output of multiple decision trees to produce a final prediction.
  • Advantages: Handles large datasets well, reduces overfitting, and provides feature importance.

Boosting

Boosting is an ensemble technique that combines the predictions of several weak learners (typically decision trees) to form a strong learner. Unlike random forests, where trees are built independently, boosting builds trees sequentially, with each tree trying to correct the errors of the previous ones.

  • Key Idea: Sequentially combines weak models to correct errors and improve performance.
  • Popular Algorithms: AdaBoost, Gradient Boosting, XGBoost, LightGBM.
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load dataset
iris = load_iris()
X, y = iris.data, iris.target

# Split data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Create and train the Random Forest model
rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)

# Predict and evaluate the model
y_pred = rf_model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print(f"Random Forest Accuracy: {accuracy:.2f}")
End of lesson.
Back to Data Science