Loading


Machine Learning

  • Home
  • Machine Learning
  • ML | Voting Classifier using Sklearn. The Complete Machine Learning Developer Course 2024.
Machine Learning

ML | Voting Classifier using Sklearn. The Complete Machine Learning Developer Course 2023 [Videos].

By admin / 18-jan-2022 / in Blog, Machine Learning / 146 Comments / 10 Likes / Share This Post

A Voting Classifier is a machine learning model that trains on an ensemble of numerous models and predicts an output (class) based on their highest probability of chosen class as the output.

It simply aggregates the findings of each classifier passed into Voting Classifier and predicts the output class based on the highest majority of voting. The idea is instead of creating separate dedicated models and finding the accuracy for each them, we create a single model which trains by these models and predicts output based on their combined majority of voting for each output class.

Voting Classifier supports two types of votings.

  1. Hard Voting: In hard voting, the predicted output class is a class with the highest majority of votes i.e the class which had the highest probability of being predicted by each of the classifiers. Suppose three classifiers predicted the output class(A, A, B), so here the majority predicted A as output. Hence A will be the final prediction.
  2. Soft Voting: In soft voting, the output class is the prediction based on the average of probability given to that class. Suppose given some input to three models, the prediction probability for class A = (0.30, 0.47, 0.53) and B = (0.20, 0.32, 0.40). So the average for class A is 0.4333 and B is 0.3067, the winner is clearly class A because it had the highest probability averaged by each classifier.

Note: Make sure to include a variety of models to feed a Voting Classifier to be sure that the error made by one might be resolved by the other.
Code : Python code to implement Voting Classifier

# importing libraries
from sklearn.ensemble import VotingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
  
# loading iris dataset
iris = load_iris()
X = iris.data[:, :4]
Y = iris.target
  
# train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, 
                                                    Y, 
                                                    test_size = 0.20, 
                                                    random_state = 42)
  
# group / ensemble of models
estimator = []
style="color:#fff; text-decoration:underline;" estimator.append(("LR", LogisticRegression(solver ="lbfgs", multi_class ="multinomial", max_iter = 200))) estimator.append(("SVC", SVC(gamma ="auto", probability = True))) estimator.append(("DTC", DecisionTreeClassifier())) # Voting Classifier with hard voting vot_hard = VotingClassifier(estimators = estimator, voting ="hard") vot_hard.fit(X_train, y_train) y_pred = vot_hard.predict(X_test) # using accuracy_score metric to predict accuracy score = accuracy_score(y_test, y_pred) print("Hard Voting Score % d" % score) # Voting Classifier with soft voting vot_soft = VotingClassifier(estimators = estimator, voting ="soft") vot_soft.fit(X_train, y_train) y_pred = vot_soft.predict(X_test) # using accuracy_score score = accuracy_score(y_test, y_pred) print("Soft Voting Score % d" % score) Output : Hard Voting Score 1 Soft Voting Score 1 Examples:estimator.append(("LR", LogisticRegression(solver ="lbfgs", multi_class ="multinomial", max_iter = 200))) estimator.append(("SVC", SVC(gamma ="auto", probability = True))) estimator.append(("DTC", DecisionTreeClassifier())) # Voting Classifier with hard voting vot_hard = VotingClassifier(estimators = estimator, voting ="hard") vot_hard.fit(X_train, y_train) y_pred = vot_hard.predict(X_test) # using accuracy_score metric to predict accuracy score = accuracy_score(y_test, y_pred) print("Hard Voting Score % d" % score) # Voting Classifier with soft voting vot_soft = VotingClassifier(estimators = estimator, voting ="soft") vot_soft.fit(X_train, y_train) y_pred = vot_soft.predict(X_test) # using accuracy_score score = accuracy_score(y_test, y_pred) print("Soft Voting Score % d" % score) Output : Hard Voting Score 1 Soft Voting Score 1 Examples:
Input  :4.7, 3.2, 1.3, 0.2 
Output :Iris Setosa

In practical the output accuracy will be more for soft voting as it is the average probability of the all estimators combined, as for our basic iris dataset we are already overfitting, so there wont be much difference in output.

Read more
See All

Comments (146 Comments)

Submit Your Comment

See All Posts

Related Posts

Machine Learning / Youtube

What is machine learning in simple words?

Learning means the acquisition of knowledge or skills through study or experience. Based on this, we can define machine learning (ML) as follows: It may be defined as the field of computer science, more specifically an application of artificial intelligence, which provides computer systems the ability to learn with data and improve from experience without being explicitly programmed. Basically, the main focus of machine learning is to allow the computers learn automatically without human intervention. Machine learning is a subfield of artificial intelligence, which is broadly defined as the capability of a machine to imitate intelligent human behavior. Artificial intelligence systems are used to perform complex tasks in a way that is similar to how humans solve problems.
27-jan-2021 /10 /146

Machine Learning / Youtube

What is sequence data in machine learning?

Sequence Modeling is the task of predicting what word/letter comes next. Unlike the FNN and CNN, in sequence modeling, the current output is dependent on the previous input and the length of the input is not fixed. In this section, we will discuss some of the practical applications of sequence modeling.
3-jan-2022 /10 /146

Machine Learning / Youtube

What is descriptive statistics in machine learning?

DESCRIPTIVE STATISTICS : Descriptive Statistics is a statistics or a measure that describes the data. INFERENTIAL STATISTICS : Using a random sample of data taken from a population to describe and make inferences about the population is called Inferential Statistics.
3-jan-2022 /10 /146