The £20/Month AI Software Factory: Part 2 — Architecture & LLM Router

Whitepaper Callout: Download the Full 12-Page Architecture PDF Whitepaper

1. 3-Tier Model Dispatch Topology

In Part 1: Vision & The Financial Trap, we established why flat-rate SaaS seats fail under autonomous agent workloads and introduced the 80/15/5 Pareto principle. To operationalize this allocation in production, we implement a 3-Tier Model Dispatch Topology.

Rather than sending every developer request or agent tool call to a single frontier model, incoming tasks pass through an intelligent classifier that routes the payload to the optimal execution tier based on required intelligence, latency tolerances, and budget constraints.

                                  3-TIER DISPATCH TOPOLOGY SCHEMATIC
  
                                       +-----------------------+
                                       | Incoming Dev / Agent  |
                                       |     Task Payload      |
                                       +-----------+-----------+
                                                   |
                                                   v
                                       +-----------------------+
                                       | Dynamic AST Classifier|
                                       | Complexity Score (S)  |
                                       +-----------+-----------+
                                                   |
                   +-------------------------------+-------------------------------+
                   | (S < 0.35)                    | (0.35 <= S < 0.75)            | (S >= 0.75)
                   v                               v                               v
       +-----------------------+       +-----------------------+       +-----------------------+
       |     TIER 0 (LOCAL)    |       |    TIER 1 (FLASH API) |       |  TIER 2 (FRONTIER API)|
       |   Qwen 2.5 Coder 14B  |       |    Gemini 1.5 Flash   |       |    Claude 3.5 Sonnet  |
       |  (Hetzner VPS / Ollama|       |   (Claude 3.5 Haiku)  |       |   (DeepSeek-R1 / o3)  |
       |   Cost: £0.00 Var.    |       |   Cost: ~£0.001/task  |       |   Cost: ~£0.080/task  |
       +-----------+-----------+       +-----------+-----------+       +-----------+-----------+
                   |                               |                               |
                   +-------------------------------+-------------------------------+
                                                   |
                                                   v
                                       +-----------------------+
                                       | Quality Verification  |
                                       |   Escalation Ladder   |
                                       +-----------------------+

The Three Execution Tiers

1. Tier 0: Local VPS (Open-Weights Inference Engine)

   • Models: Qwen 2.5 Coder 14B, Llama 3.1 8B, DeepSeek-Coder-V2-Lite served via Ollama / vLLM on a dedicated Hetzner VPS instance.
   • Role: Handles deterministic, structural, and low-complexity tasks (formatting, boilerplate creation, unit test template generation, syntax validation).
   • Economics: Fixed server rental (£13/month). Unlimited token processing with £0.00 variable cost.

2. Tier 1: High-Efficiency Flash API

   • Models: Google Gemini 1.5 Flash, Anthropic Claude 3.5 Haiku.
   • Role: Processes routine function modifications, multi-file refactoring, API integration glue, and standard bug fixes.
   • Economics: Extremely low API rates ($0.075 / 1M input tokens). Average cost ~£0.001 per execution.

3. Tier 2: Frontier Reasoning API

   • Models: Anthropic Claude 3.5 Sonnet, DeepSeek-R1, OpenAI o3-mini.
   • Role: Reserved exclusively for high-ambiguity architectural design, cross-service refactoring, security audits, and complex root-cause failure isolation.
   • Economics: Premium API rates ($3.00 / 1M input tokens). Average cost ~£0.080 per execution.

2. Dynamic Prompt Classification & Routing Algorithm

At the core of the AI Software Factory is a high-speed, lightweight classification engine that evaluates every task payload prior to LLM invocation.

Metric Vector & Complexity Equation

The router evaluates incoming tasks across a 4-dimensional Metric Vector:

$$V = \begin{pmatrix} L_{\text{norm}} \ C_{\text{ast}} \ R \ A \end{pmatrix}$$

Where:

• $L_{\text{norm}} \in [0, 1]$: Normalized token length of prompt and attached file context.
• $C_{\text{ast}} \in [0, 1]$: AST structural complexity (cyclomatic complexity score, nesting depth, total functions affected).
• $R \in [0, 1]$: Reasoning requirement index (detected keywords indicating algorithmic design, concurrency, or state management).
• $A \in [0, 1]$: Architectural ambiguity rating (high impact on multi-module contracts vs. isolated local changes).

The total Complexity Score ($S$) is calculated as a weighted linear combination:

$$S = 0.25 L_{\text{norm}} + 0.35 C_{\text{ast}} + 0.25 R + 0.15 A$$

Routing Decision Thresholds

$$S_{\text{target}} = \begin{cases} \text{Tier 0 (Local VPS)}, & \text{if } S < 0.35 \ \text{Tier 1 (Flash API)}, & \text{if } 0.35 \le S < 0.75 \ \text{Tier 2 (Frontier API)}, & \text{if } S \ge 0.75 \end{cases}$$

TypeScript Implementation

Below is the production TypeScript implementation of the dynamic routing engine:

export interface TaskContext {
  promptText: string;
  filesModifiedCount: number;
  astDepth: number;
  isArchitecturalChange: boolean;
}

export interface RouteDecision {
  tierNumber: 0 | 1 | 2;
  tierName: "Tier 0 (Local VPS)" | "Tier 1 (Flash API)" | "Tier 2 (Frontier API)";
  targetModel: string;
  complexityScore: number;
  estimatedCostUSD: number;
  reasoning: string;
}

export function classifyAndRoutePrompt(
  prompt: string,
  context: TaskContext
): RouteDecision {
  const tokenEst = Math.ceil(prompt.length / 4);
  const L_norm = Math.min(1.0, tokenEst / 20000);

  // Calculate AST complexity ratio
  const C_ast = Math.min(1.0, (context.astDepth * 0.1) + (context.filesModifiedCount * 0.15));

  // Inspect reasoning keywords
  const reasoningRegex = /(architect|concurrency|race condition|saga|state machine|security|crypto|distributed)/i;
  const R = reasoningRegex.test(prompt) ? 0.9 : 0.2;

  // Ambiguity score
  const A = context.isArchitecturalChange ? 0.95 : 0.15;

  // Compute weighted complexity score S
  const S = 0.25 * L_norm + 0.35 * C_ast + 0.25 * R + 0.15 * A;
  const complexityScore = Math.round(S * 100);

  if (S < 0.35) {
    return {
      tierNumber: 0,
      tierName: "Tier 0 (Local VPS)",
      targetModel: "Qwen 2.5 Coder 14B (Local VPS)",
      complexityScore,
      estimatedCostUSD: 0.00,
      reasoning: "Low AST complexity & short context. Assigned to Local VPS for zero-variable cost execution."
    };
  } else if (S < 0.75) {
    return {
      tierNumber: 1,
      tierName: "Tier 1 (Flash API)",
      targetModel: "Gemini 1.5 Flash / Claude 3.5 Haiku",
      complexityScore,
      estimatedCostUSD: (tokenEst / 1_000_000) * 0.075,
      reasoning: "Moderate code modifications required. Assigned to high-throughput Tier 1 Flash API."
    };
  } else {
    return {
      tierNumber: 2,
      tierName: "Tier 2 (Frontier API)",
      targetModel: "Claude 3.5 Sonnet / DeepSeek-R1",
      complexityScore,
      estimatedCostUSD: (tokenEst / 1_000_000) * 3.0,
      reasoning: "High architectural impact & deep reasoning needed. Scaled to Tier 2 Frontier API."
    };
  }
}

3. Embedded Interactive Simulator

Test the routing logic and model allocation ratios live using the interactive calculator below. Adjust your team's daily task volume, average token size, and tier allocation sliders to simulate monthly savings versus raw frontier APIs and traditional enterprise SaaS seat pricing.

4. Quality Verification & Automated Fallback Escalation Ladder

Relying on lower-tier models (Tier 0 and Tier 1) introduces the risk of subtle syntax errors, hallucinated variable names, or incomplete logic. To guarantee code quality without manual oversight, the AI Software Factory wraps model execution in a 5-Step Quality Escalation Ladder.

                   QUALITY VERIFICATION & ESCALATION LADDER
  
  [Step 1: ESLint / Format]  ---> (Fail?) ---> Auto-Fix via Regex
           | (Pass)
           v
  [Step 2: AST & Type Check] ---> (Fail?) ---> Retry Prompt with Compiler Output (Tier 0)
           | (Pass)
           v
  [Step 3: Unit & Integration Test] ---> (Fail) ---> Escalate to Tier 1 Flash API
           | (Pass)
           v
  [Step 4: Tier Escalation Retry] ---> (Fail x2) ---> Escalate to Tier 2 Frontier API
           | (Pass)
           v
  [Step 5: HITL Alerting] ----> Flag to Human Developer with Full Diagnostic Stack

The 5 Verification Steps

1. Step 1: Syntax & Style Linting (ESLint / Prettier) The generated code is immediately checked against ESLint rules. Trivial errors (missing semicolons, unused imports, indentation) are auto-corrected using zero-shot regex rules without LLM re-invocation.

2. Step 2: AST & Static Type Validation (tsc / pyright) The codebase undergoes AST parsing and type checking. If type errors occur, the compiler diagnostic output is appended to the context, and Tier 0 retries code generation.

3. Step 3: Automated Unit & Integration Tests (vitest / pytest) The automated test suite runs against the modified module. If unit tests fail, the exact test failure stack trace is fed back into the pipeline.

4. Step 4: Tier Escalation Ladder (Automatic Model Promotion)

   • If Tier 0 (Local VPS) fails verification twice, the task automatically escalates to Tier 1 (Gemini Flash) along with the compiler/test error logs.
   • If Tier 1 fails to pass tests after 1 attempt, the task escalates to Tier 2 (Claude 3.5 Sonnet).

5. Step 5: Human-in-the-Loop (HITL) Alerting If Tier 2 fails after 2 retry attempts, the pipeline halts, isolates the breaking commit in a sandbox git branch, and sends a notification with complete diagnostic stack traces to the human developer.

This escalation ladder guarantees that expensive Tier 2 models are invoked only when simpler models hit verifiable logic ceilings, preserving both software reliability and financial efficiency.

5. Conclusion & Download PDF Whitepaper

By building a self-hosted AI Software Factory governed by a 3-tier routing topology and automated quality verification ladder, software teams can break free from SaaS seat restrictions and avoid runaway API bills. On a fixed £20/month budget, your organization can process thousands of background agent tasks per month with production-grade code quality.

Whitepaper Callout: Download the Full 12-Page Architecture PDF Whitepaper