Steering LLMs and Embeddings with SAEs

In this demo, we’ll explore how Sparse Autoencoders (SAEs) can be utilized to steer Large Language Models (LLMs) and their embeddings. By leveraging tools like Goodfire’s API and Google’s Gemma Scope, we’ll demonstrate how SAEs can interpret and modify the internal activations of LLMs, enabling precise control over their knowledge selection behaviors. This approach addresses challenges such as context-memory knowledge conflicts, enhancing the reliability and accuracy of AI-generated outputs.