In many applications, classical machine learning models remain the best choice, yet their internal workings are often misunderstood or unknown. In this edition, we explain how they work, focusing on SVM, the importance of manual feature engineering, and how tools like correlation matrices help optimize this process. We also contrast with deep learning to clarify key differences.
A classical model like SVM (Support Vector Machine) learns to classify data based on a set of numerical features, which are measurements or filters manually applied to the original data. For example, in a system categorizing clients or products, an engineer defines and calculates these features based on domain knowledge (such as Sobel filters for edges in images or statistical measures in tabular data).
The SVM seeks a hyperplane in this feature space that maximizes the distance (margin) between classes—that is, the boundary that best separates the data without the closest examples from each class overlapping. This margin is key for the model to generalize well to new data.
Feature selection is critical. This is where a correlation matrix helps: it identifies useful variables and discards redundant or irrelevant ones, simplifying the model and reducing computational and labeling costs.
In contrast, deep learning networks automatically learn relevant features and classification functions simultaneously, adjusting filters and weights during training. This requires more data and computing power but can capture more complex patterns without manual intervention.
Without good feature selection and definition, SVM and other classical models perform poorly. Many companies underestimate the effort needed at this stage and expect the model to "learn everything by itself," which is not the case in classical ML.
To successfully apply classical ML:
- Invest time in understanding and selecting relevant variables using statistical analyses like correlation matrices. - Design filters or transformations that extract meaningful information from data, based on domain knowledge. - Use SVM to find the hyperplane that maximizes the margin between classes, ensuring good generalization. - Be clear that this approach works well in problems with labeled data and defined features, at a lower cost than deep learning.
This knowledge helps optimize resources and avoid overcomplicating solutions when unnecessary.