Linear Classifiers

Home 📖 Read Deep Dive Next: Perceptron →

Core Concept

Separates data into classes using a straight line (2D) or hyperplane (higher dimensions). Think of organizing a desk: papers left, mug right.

Decision Function

Components:

• x: Input feature vector (d-dimensional)
• θ: Weight vector (perpendicular to decision boundary)
• θ₀: Bias term (offset from origin)
• sign(·): Returns +1 or -1

Decision Boundary

Defined by: θ^Tx + θ₀ = 0

• Points where θ^Tx + θ₀ > 0 → Class +1
• Points where θ^Tx + θ₀ < 0 → Class -1
• θ points toward positive class

Key Characteristics

Strengths

• Fast computation
• Interpretable results
• Mathematically elegant

Limitations

• Only works for linearly separable data
• Cannot handle complex, curved boundaries

Learning Algorithms

Two classic approaches to find θ and θ₀:

• Perceptron: Iterative error correction
• Support Vector Machines (SVM): Maximize margin between classes

Quick Facts

• Building block of neural networks
• Used to interpret complex models
• Efficient for real-time systems
• Still relevant despite AI advances

Home 📖 Read Deep Dive Next: Perceptron →