The AI stack is still developing. Different companies experiment with various approaches, tools, and architectures as they figure out what works at scale.
The complication is that patterns are beginning to coalesce around a clear chain that multiple enterprises have independently discovered. I’ve observed this same architecture at Uber, Airbnb, the largest bank in Brazil, Coca-Cola, HubSpot, and several European airlines.
Why is this chain structured this way? The architecture follows a specific flow: Data → Vector Database → Context Database → LLM → DSPy/GEPA/Evals/Experimentation → RL
Raw data forms the foundation of any AI system. This includes structured databases, unstructured documents, real-time streams, and historical archives that contain the information the AI system needs to understand and act upon.
Vector databases transform this raw data into mathematical representations that AI models can efficiently process. They convert text, images, and other data types into high-dimensional vectors that capture semantic meaning, enabling fast similarity searches and retrieval.
Context databases store unstructured institutional knowledge that was previously trapped in people’s heads. Andy Triedman explores this concept in his analysis of the business context layer. These databases provide crucial business context, historical decisions, and domain expertise that inform AI responses.
Large Language Models process the vector representations and contextual information to generate responses. They serve as the reasoning engine that transforms inputs into coherent outputs based on their training and the provided context.
DSPy and GEPA represent the experimentation layer where models are optimized and refined. DSPy provides a framework for systematic prompt engineering, while GEPA enables multi-objective optimization of AI systems.
Evaluations and experimentation create feedback loops for continuous improvement. Teams test different approaches, measure performance across multiple metrics, and iterate on model behavior to achieve better results.
Reinforcement Learning closes the loop by using real-world feedback to further refine model behavior. It enables systems to learn from deployment experience and adapt to changing requirements over time.
This structure emerged because each component solves a specific problem that enterprises encountered when deploying AI at scale. The linear flow ensures data flows efficiently from source to application while maintaining quality, context, and continuous improvement throughout the pipeline.