The Latent Space Live conference at NeurIPS 2024 highlighted advancements in AI architectures, particularly focusing on alternatives to traditional transformer models. Keynote speakers Dan Few and Eugene Cheer discussed the evolution of post-transformer architectures, emphasizing the importance of scaling models efficiently in terms of parameter size and context length. They explored the challenges of quadratic scaling in attention mechanisms and introduced linear attention as a more computationally efficient alternative. The conference also covered state-space models, which use principles from signal processing to improve quality and efficiency in sequence modeling. These models have shown promise in various benchmarks and applications, including language modeling and time series analysis. The speakers highlighted the importance of hardware and kernel support for these new architectures to ensure practical implementation and efficiency. Additionally, they discussed the potential of hybrid models that combine different architectural elements to achieve better performance than traditional models. The conference concluded with discussions on future directions, including the integration of efficient architectures with new applications like video generation and long-context processing.