Cost Optimization AI

AI API mahal jika tidak dikelola. Satu user bisa habiskan $0.10 per percakapan. 10K user aktif = $1000/hari. Optimasi cost penting sejak awal.

Strategi

1. Model Selection

// Route berdasarkan complexity
function selectModel(task) {
  if (task.type === "simple_qa") return "gpt-4o-mini"; // $0.15/1M tokens
  if (task.type === "complex_reasoning") return "gpt-4o"; // $2.50/1M tokens
  if (task.type === "code_generation") return "claude-sonnet"; // varies
}

2. Caching

// Semantic cache — cache by meaning, not exact match
async function cachedAI(query) {
  // 1. Embed query
  const embedding = await embed(query);

  // 2. Check cache (cosine similarity > 0.95)
  const cached = await vectorCache.query({ vector: embedding, topK: 1 });
  if (cached.matches[0]?.score > 0.95) {
    return cached.matches[0].metadata.response; // Cache hit!
  }

  // 3. Cache miss — call AI
  const response = await openai.chat.completions.create({...});

  // 4. Store in cache
  await vectorCache.upsert([{
    id: uuid(),
    values: embedding,
    metadata: { query, response: response.choices[0].message.content },
  }]);

  return response.choices[0].message.content;
}

3. Prompt Optimization

• Shorter system prompts = fewer tokens per request
• Summarize conversation history instead of sending full history
• Use max_tokens to limit output length

4. Rate Limiting

// Per-user limits
const DAILY_LIMIT = 50; // messages per day
const userUsage = await redis.incr(`ai:usage:${userId}:${today}`);
if (userUsage > DAILY_LIMIT) {
  throw new Error("Batas harian tercapai");
}