This session, led by Anton McGonnell, Head of Software Products at SambaNova Systems, presents an in-depth exploration of the Composition of Experts approach in AI, and in particular the benefits to individual developers. Instead of building one monolithic model, this strategy composes workflows using many specialised, independent "expert" models. This proves to be more efficient, modular, and extensible.

The talk will provide a deep dive into the advantages of this technique, which promotes reusability, saves on alignment tax, and enables granular access control. By detailing real-world implementations, attendees will learn best practices for breaking down problems, assigning appropriate experts, and orchestrating data flow between them. McGonnell will discuss how SambaNova's architecture, with its large memory capacity, enables this approach by hosting and serving thousands of models in parallel on a single node without performance degradation. This makes the Composition of Experts approach highly scalable and reduces the total cost of ownership for inference.