Library / Case Studies
TensorFlowOnSpark: Distributed Deep Learning on Big Data Clusters at Yahoo
yahoohadoop: By Lee Yang, Jun Shi, Bobbie Chern, and Andy Feng (@afeng76), Yahoo Big ML team Introduction Today, we are pleased to offer TensorFlowOnSpark to the community, our latest open source framework for distributed deep learning on big-data clusters. Deep learning (DL) has evolved significantly in recent…
By Yahoo engineers · Yahoo · Added 2026-10-12
All content belongs to Yahoo engineers / Yahoo. We only link to it and give credit. Report a problem
Learn this case study
About 37 min left. Your work is saved as you go.
What you are about to study
Yahoo explains a real system. You should know the basics: caching, queues and replication.
Ideas you will meet
- Distributed Systems
- Consensus, time, failure and the fundamentals of scale-out.
- Machine Learning Systems
- Recommendations, ranking, feature stores and serving models at scale.
Go deeper on this
A path through videos, articles, other case studies and code on the same ideas as TensorFlowOnSpark: Distributed Deep Learning on Big Data Clusters at Yahoo.
Warm upGet the vocabulary first.
Books · Free from the author
Concurrent and Distributed Systems (Cambridge lecture notes)
Eight lectures on time, broadcast, replication, consensus and transactions, with notes, slides and video. Free from the author under CC BY-SA.
University of CambridgeIntermediate
Books · Free from the author
Computer Networks: A Systems Approach
How networks are built and why: protocols, routing, congestion control and the internet's architecture. The authors publish it free online under CC BY 4.0.
Systems ApproachIntermediate
WatchHow others solve the same problem, on video.
Videos belong to their creators and play from YouTube. More on YouTube (opens in a new tab)
ReadEngineers explain it in their own words.
Blogs
Amazon Science
Technical deep-dives and perspectives from our scientists who are pushing boundaries across multiple scientific disciplines while solving real-world challenges. Explore how our teams combine research excellence, world...
AmazonIntermediate
Papers
Atom Smashing using Machine Learning at CERN
A real-world engineering write-up from CERN on distributed Machine Learning.
CERNIntermediate
See it in another companyThe same ideas, a different system.
Case Studies · Read here
Aroma: Using ML for Code Recommendation at Facebook
Thousands of engineers write the code to create our apps, which serve billions of people worldwide. This is no trivial task—our services have grown so diverse and complex that the codebase contains…
FacebookIntermediate
Case Studies · Read here
Concourse: Generating Personalized Content Notifications in Near-Real-Time at LinkedIn
Social media are computer-mediated platforms that facilitate creation and sharing of information, ideas, career interests, and other forms of expression via virtual communities and networks. In the last few years, not...
LinkedInIntermediate
Read the codeSee how it looks when it is real.
GitHub
Engineering Blogs
A huge directory of company engineering blogs to learn from real production systems.
kilimchoiIntermediate
GitHub
Awesome Scalability
Curated reading list of articles on the architecture of scalable, reliable systems.
binhnguyennusIntermediate
Keep learning
Loading community…