AI in the System: Multi-Provider Strategy

This article explains how we use AI across the website for content generation, translation, and chatbot responses.

The Problem: Scaling Content Creation

Creating content manually doesn't scale:

  • Products(5k+) need descriptions

  • Query pages(65k+) need unique content

  • Multiple languages(10) need translations

  • Customer questions need instant answers

Writing this manually would take years. We need AI automation.

The Solution: Multi-Provider AI Strategy

We use three AI providers, each optimized for specific tasks:

  • DeepSeek: Product descriptions, query pages, AI-powered search (cost-effective, high quality)

  • Amazon Bedrock: Chatbot responses, internal tools, summarization (AWS-integrated, multiple models)

  • Gemini: Content generation fallback (fast)

AI Providers

DeepSeek

DeepSeek is our primary provider for content generation due to cost-effectiveness and quality:

  • Use cases: Product descriptions, query page content, AI-powered semantic search

  • Fallback: Together.ai provides automatic failover using the same model family

  • Caching: System prompts enable prompt caching - repeated requests with the same system prompt benefit from reduced token costs

Amazon Bedrock

Amazon Bedrock provides access to multiple foundation models through AWS:

  • Chatbot: Powers both public website chatbot and internal employee chat

  • Summarization: Update message summarization

  • Analysis: Git commit analysis, file classification

  • SEO Tasks: Advanced parsing using Claude models

Gemini

Google Gemini serves as an alternative provider:

  • Use cases: Content generation when DeepSeek is unavailable

  • Integration: Direct API via Google's generative AI SDK

AI Use Cases

Product Descriptions

We use DeepSeek to generate product descriptions:

  1. Build product name and feature list from the product database
  2. Construct a system prompt describing the expected output format
  3. Send to DeepSeek API (system prompt enables caching for repeated requests)
  4. Parse response into tagline and body content
  5. Store in database for multilingual support

Query Page Content

We use DeepSeek to generate query page content:

  1. Extract query text from cluster data
  2. Build product context from matching products
  3. Send to DeepSeek with structured system prompt
  4. Parse response into page components (tagline and body)
  5. Store for multilingual serving

Translation

We use DeepSeek for AI-powered translations:

  1. Queue missing translations as they're encountered
  2. Process queued strings in batches (with Together.ai fallback)
  3. Store translations in phrase tables for lookup
  4. Subsequent requests use cached translations

Search Suggestions

We use Sentence Transformers for semantic similarity:

  • Generate query embeddings for similarity comparison

  • Store embeddings in Valkey for vector search

  • Find related queries based on embedding distance

Chatbot

Amazon Bedrock powers the chatbot:

  • Public chatbot: Answers product questions, provides recommendations

  • Internal chatbot: Assists employees with system queries, includes tool calls for data lookup

Prompt Engineering

System Prompts

System prompts enable caching efficiency with DeepSeek:

  • Cache hits: When the same system prompt is reused, DeepSeek caches the prompt tokens, reducing cost and latency

  • Structure: System prompt defines output format and constraints; user prompt contains the specific request

  • Logging: The system logs cache hit/miss statistics for monitoring

Temperature Settings

Different tasks use different temperature settings:

  • Product descriptions: Higher temperature for creative, varied output

  • Search interpretation: Lower temperature for consistent, predictable filter extraction

  • Chatbot: Moderate temperature for balanced conversational responses

  • Summarization: Lower temperature for accurate, focused summaries

Token Optimization

We optimize token usage through:

  • Prompt caching: DeepSeek's cache reduces repeated system prompt costs

  • Structured outputs: Request specific JSON formats to minimize parsing overhead

  • Fallback handling: Automatic failover to Together.ai maintains availability without manual intervention

References

AI Provider Documentation

Related Articles

Summary

Our AI system provides:

  • Multiple providers: DeepSeek for content generation, Bedrock for chatbots and internal tools, Gemini as fallback

  • Optimized prompts: System prompts enable caching for cost reduction

  • Task-specific tuning: Temperature and model selection matched to use case requirements

  • Reliability: Automatic fallback mechanisms maintain availability