Business Technology Trends and Predictions 2019

data-analyticsdata-engineeringdataopsdata-warehousinganalytics-strategy

Abstraction: Six new rules for 21st-century analytics enabled by streaming and cloud technologies

Key points:

Five enabling technologies: low-cost streaming (Kafka, NiFi, Kinesis), in-memory analytics (SAP HANA), big data environments (Hadoop, Redshift, BigQuery), AI/neural network renaissance, hyperscale cloud (AWS, Azure, GCP)
Rule 1: "Everything is a verb" — model events (verbs) in replayable activity logs rather than entity-relationship nouns
Rule 3: "Build provisionally" — curated views with 3-4 hierarchy levels layered atop a data lake; ephemeral but richly structured
Rule 4: "Design for speed of thought" — sub-second responses even for aggregations across 500M rows; ML models managed dynamically at scale
Rule 5: "Waste is good" — a class schedule exploded to room-minute granularity yields 2.3B rows but trivial visualization
Rule 6: "Democratize it" — analytics as a team sport; information sharing replaces information hoarding as organizational power

Connections: Data Analytics · Data Engineering · Data Democratization