Naive Bayes is a very popular Supervised Classification algorithm. This algorithm is called “Naive” because it makes a naive assumption that each feature is independent of other features. It is near to impossible to find such data sets in real life. Bayes’ theorem is the base for Naive Bayes Algorithm.
The essence of the Bayes theorem is conditional probability. Conditional probability is the probability that something will happen, given that something else has already occurred.
By using conditional probability, we can find out the probability of an event that will occur given the knowledge of the previous event.
- P(A|B) = Posterior Probability, Probability of A given Value of B.
- P (B|A) = Likelihood of B given A is True.
- P (A) = Prior Probability, Probability of event A.
- P (B) = Marginal Probability, Probability of event B.
By using the basis of the Bayes theorem, the Naive Bayes Classifier formula can be written as follows :
Pros and Cons of Naive Bayes Algorithm:
- It is easy to understand.
- It can also be trained on a small dataset.
- It has a ‘Zero conditional probability Problem’.That is for features having zero frequency the total probability becomes zero. There are several sample correction techniques to fix this problem such as “Laplacian Correction.”
- Another disadvantage is the very strong assumption of independence class features that it makes.
- Real-time Prediction: Naive Bayes is an eager learning classifier and it is sure fast. Thus, it could be used for making predictions in real-time.
- Multi-class Prediction: This algorithm is also well known for its multi-class prediction feature. Here we can predict the probability of multiple classes of the target variables.
- Text classification/ Spam Filtering/ Sentiment Analysis: Naive Bayes classifiers mostly used in text classification. Due to better results in multi-class problems and independence rule, it has a higher success rate as compared to other algorithms. So it is widely used in Spam filtering and Sentiment Analysis.
- Recommendation System: Naive Bayes Classifier and Collaborative Filtering together build a Recommendation System that uses machine learning and data mining techniques to filter unseen information and predict whether a user would like a given resource or not.