Distributed Computing

concepts · 5 notes linked

Related: Big Data · Hadoop · Data Engineering · Mapreduce · Coiled · Data Science · Pandas · Pachyderm

Notes

• Adam Drake — Command-line Tools can be 235x Faster than your Hadoop Cluster — Shell pipelines outperform Hadoop by 235x on single-machine data tasks
• Distributed Computing for Data Scientists — Organizational and technical challenges of scaling Python data science to clusters
• Don't use Hadoop - your data isn't that big — When Hadoop is overkill versus simpler data tools
• When Grandmasters Blunder — MapReduce analysis of chess blunder rates across rating levels
• Writing An Hadoop MapReduce Program In Python — Tutorial for writing Hadoop MapReduce word-count jobs in Python via Streaming API