Three Phases of Generative AI Development
genai
Given this is proving to be the path to production - organisations need to first settle on the idea of a platform. With so many aspects of the development changing - organisations need to identify in the GenAI landscape what aspects of the SDLC remains the same as the patterns, models and libraries evolve. Here are a list of of areas that are worth considering to standardize on.
- Access to Models - Access to model card, visibility of data used for training, license details, support
- Prompt Lifecycle Management - Testing, Tuning, Version Management, Regression Testing, Migration between models
- Model Comparison - Golden Dataset, method of evaluation
- LLM Hosting - Self Managed vs. Managed service, optimisations to match your load pattern
- Agent Hosting - Security, logging, observability, version management (see reasoning engine from gcp for e.g. )
- LLM Observability - Usage and Pricing Transparency (e.g. see genkit from firebase)
By taking a platform first approach, organisations can adapt to the evolving GenAI landscape while minimizing evaluation fatigue, security risks and technical debt.