DZone: Programming & DevOps news, tutorials & tools

Abstraction: Databricks Feature Store eliminates training-serving skew via unified feature pipelines

Key points:

• Training-serving skew (different feature logic at training vs. inference time) is identified as the primary silent failure mode in production ML; Databricks Feature Store solves this by encoding computation logic once for both paths.
• Point-in-time correct joins using timestamp_lookup_key are required for time-series label datasets to prevent data leakage — a missing event_timestamp is described as a data leakage bug.
• fe.log_model() (not mlflow.sklearn.log_model()) records feature lineage and enables automatic feature lookup at serving time; model versions are bound to the exact training set used.
• Online tables (backed by DynamoDB/Cosmos DB) are needed only for sub-100ms SLAs; batch scoring via fe.score_batch() reads directly from offline Delta tables for periodic jobs.
• Structured Streaming watermarks are mandatory for stateful aggregations — omitting them causes unbounded state accumulation and eventual OOM.
• Streaming features are ~10× more expensive than batch; only promote to streaming after ablation study confirms meaningful degradation with staleness.

Connections: Databricks · Apache Spark · Feature Store · Feature Engineering · Machine Learning Pipelines · Training Serving Skew

Source: http://refcardz.dzone.com/