📘 Data Mining & Warehousing Unit 4

🎯 Unit 4 Overview

Unit 4 covers supervised learning and classification techniques used in data mining. Classification is used to assign data objects into predefined classes using different algorithms such as statistical-based, distance-based, decision tree, neural network, rule-based and probabilistic classifiers.

🤖 Supervised Learning

Supervised learning is a machine learning technique where the model is trained using labeled data. Labeled data means input data already contains correct output or class label.

Example

If student data contains attendance, marks and result as Pass/Fail, then result is the class label. The model learns from this data and predicts result for new students.

Applications

• Email spam detection
• Medical disease prediction
• Loan approval prediction
• Customer churn prediction
• Student performance prediction

🏷️ Classification

Classification is a supervised learning task in which data objects are assigned to predefined classes.

Basic Steps of Classification

1. Collect labeled training data.
2. Train classification model.
3. Test model using test data.
4. Evaluate accuracy.
5. Use model for prediction on new data.

⚖️ Classification vs Regression

📊 Statistical-Based Algorithms

Statistical-based classification algorithms use statistical methods and mathematical models to classify data.

Characteristics

• Based on probability and statistics
• Uses data distribution
• Works well when data follows assumptions
• Useful for prediction and classification

Examples

• Naive Bayes classifier
• Logistic regression
• Linear discriminant analysis

📏 Distance-Based Algorithms

Distance-based algorithms classify data based on distance or similarity between data points.

K-Nearest Neighbor Algorithm

KNN classifies a new data point based on the majority class among its nearest neighbors.

Steps of KNN

1. Select value of K.
2. Calculate distance between new point and training points.
3. Find K nearest neighbors.
4. Check majority class among neighbors.
5. Assign that class to new point.

Common Distance Measures

• Euclidean distance
• Manhattan distance
• Cosine similarity

🌳 Decision Tree-Based Algorithms

Decision tree is a classification technique that represents decisions in the form of a tree. Each internal node represents a test on attribute, each branch represents result of test, and each leaf node represents class label.

Important Terms

• Root Node: Starting node of tree.
• Internal Node: Represents condition or test.
• Branch: Represents outcome of test.
• Leaf Node: Represents final class label.

Advantages

• Easy to understand
• Easy to represent using diagram
• Handles categorical and numerical data
• Useful for classification rules

Examples

• ID3 algorithm
• C4.5 algorithm
• CART algorithm

🧠 Neural Network-Based Algorithms

Neural networks are inspired by the human brain. They consist of interconnected nodes called neurons. Neural networks are useful for complex classification problems.

Layers in Neural Network

• Input Layer: Receives input features.
• Hidden Layer: Processes information.
• Output Layer: Produces final class prediction.

Advantages

• Can handle complex patterns
• Useful for large datasets
• Good for image, speech and pattern recognition
• Learns automatically from data

Limitations

• Requires large training data
• Training may take more time
• Difficult to interpret compared to decision tree

📜 Rule-Based Algorithms

Rule-based classification uses IF-THEN rules to classify data.

Example Rule

IF attendance > 75% AND marks > 40 THEN class = Pass

Advantages

• Easy to understand
• Rules are simple to explain
• Useful for expert systems
• Can be generated from decision trees

Limitations

• Too many rules may become complex
• Rules may conflict
• Not suitable for very noisy data

🎲 Probabilistic Classifiers

Probabilistic classifiers classify data based on probability. They calculate the probability of each class and assign the class with highest probability.

Naive Bayes Classifier

Naive Bayes is a probabilistic classifier based on Bayes theorem. It assumes that features are independent.

Features

• Simple and fast
• Works well for text classification
• Used in spam detection
• Requires less training data

Applications

• Email spam filtering
• Sentiment analysis
• Document classification
• Disease prediction

📌 Classification Algorithm Summary

⚖️ Decision Tree vs Neural Network

⭐ Important Questions

1. Define supervised learning with example.
2. Explain classification and its steps.
3. Differentiate between classification and regression.
4. Explain statistical-based classification algorithms.
5. Explain distance-based classification algorithm KNN.
6. Explain decision tree classifier with diagram.
7. Explain neural network-based classification.
8. Explain rule-based classifier with example.
9. Explain probabilistic classifier and Naive Bayes.
10. Compare decision tree and neural network classifier.

🔥 Last Minute Revision

• Supervised learning uses labeled data.
• Classification predicts category/class.
• KNN is distance-based classifier.
• Decision tree has root, internal and leaf nodes.
• Neural network has input, hidden and output layers.
• Rule-based classifier uses IF-THEN rules.
• Naive Bayes is probabilistic classifier.
• Decision tree is easy to interpret, neural network handles complex patterns.

🔗 Related Links