Pluggable Architecture: Swapping Components
Zyrabit SLM is built on a Hexagonal Architecture (Ports and Adapters). This design pattern ensures that the core logicβthe api-ragβis completely decoupled from infrastructure dependencies.
You can swap your inference engine (e.g., from Ollama to Anthropic) or your vector database (e.g., from ChromaDB to Postgres + pgvector) without modifying the application core.
1. Swapping LLM Providers (Inference)β
The system uses an Inference Port to communicate with language models. To use an external API like OpenAI or Anthropic instead of a local Ollama instance, you must implement or configure a new adapter.
Supported Adaptersβ
- Ollama (Default): Local, sovereign inference.
- Gemini: Google's generative AI API.
Adding a New Provider (e.g., Anthropic/OpenAI)β
To integrate a new provider:
- Implement the Adapter: Create a new file in
app/infrastructure/inference/(e.g.,anthropic_adapter.py) implementing theInferenceProviderPort. - Update the Factory: Register the new adapter in
app/inference_factory.py. - Configure Environment: Set the
INFERENCE_PROVIDERvariable in your.envfile.
# Example: Switching to a cloud provider
INFERENCE_PROVIDER="anthropic"
ANTHROPIC_API_KEY="sk-ant-..."
2. Swapping Vector Databases (Persistence)β
The RAG engine interacts with the vector database through a Persistence Port. While Zyrabit ships with ChromaDB as the default for its simplicity and local-first nature, it can be replaced with enterprise-grade solutions like Postgres with pgvector.
Migration to Postgres + pgvectorβ
To swap the vector store:
- Add the Postgres Adapter: Implement the
VectorStorePortin a newapp/infrastructure/persistence/postgres_adapter.py. - Database Dependencies: Ensure the
postgresservice is added to yourdocker-compose.ymlwith thepgvectorextension enabled. - Update Dependency Injection: Configure the system to inject the
PostgresAdapterinstead of theChromaAdapterin the infrastructure layer.
3. Benefits of Hexagonal Designβ
- Technology Independence: Swap components as the AI landscape evolves.
- Testability: Mock external providers easily for unit and integration testing.
- Data Sovereignty Flexibility: Switch between 100% local stacks (Ollama + Chroma) and hybrid stacks (OpenAI + Local DB) depending on your organization's privacy requirements.