21st-Century Analytics: New Technologies and New Rules

data-analyticsdata-warehousinghigher-educationdataopsstreaming

Abstraction: Higher-education analytics modernization arguing against 30-year-old Kimball data warehouse mental models

Key points:

Diagnosing higher-ed analytics dysfunction: political silos, flat-file data movement still dominant, Kimball star/snowflake warehouses requiring add-ons; designers using 30-year-old scarcity-of-compute mental models
Five technologies bending the cost-benefit curve: streaming (Kafka, NiFi), in-memory analytics (SAP HANA), big data storage (Hadoop, Redshift, BigQuery), AI/neural network resurgence including ASICs/FPGAs/TPUs, hyperscale cloud
"Eventual consistency data integrity model" enables low-cost, high-speed data integration with idempotent replayable logs
UC San Diego implementation: activity log data lake with curated views (analytics vignettes) layered atop; one domain brings in 5,000+ fields, surfaces 800 initially
Old-to-new concept table maps 12 transitions (e.g., ETL to streaming, dimensional modeling to multiple techniques, statistical analysis to ML/AI, individual to team accountability)
ML model management framed as "tires": many in service simultaneously, requiring operational monitoring, runway testing, and seamless rotation

Connections: Data Analytics · Data Engineering · Data Warehousing · Data Democratization