Machine Learning Done Wrong

Abstraction: Seven common practitioner mistakes in applied machine learning modeling

Key points:

• Default loss functions (e.g., squared error) rarely align with business objectives — fraud detection needs asymmetric false-negative penalties proportional to dollar amount
• Logistic regression is linear; non-linear feature interactions (e.g., "billing = shipping AND amount < $50") must be manually encoded or use kernel/tree models
• High-variance models (SVM with kernel) overfit when n<<p (samples far fewer than features), common in medical data
• L1/L2 regularization without prior feature standardization biases against large-scale features
• Multicollinear predictors make linear model coefficients ill-conditioned and uninterpretable as feature importance measures
• AdaBoost is more sensitive to outliers than decision trees; robust models or outlier filtering needed when outliers are non-systematic

Connections: Machine Learning · Regularization · Overfitting · Feature Engineering

Source: http://ml.posthaven.com/machine-learning-done-wrong