The Three Amigos

Enter password to continue

The Three Amigos

How We Built an AI Team That Works Together

Jim

The Human Boss - Jim is the visionary behind everything. He lives in Huntington Beach with his dogs Riley and Daisy. Jim directs the AI team, decides what to build, and provides the creative vision.

Jim communicates with Ryan through Claude Code on his Samsung Android phone running Termux. He talks to Bolt by calling the phone line.

Samsung Phone Termux Claude Code CLI

Ryan

The Mobile AI - Ryan runs on Jim's phone in Termux using Claude Code. He handles coding, web development, YouTube video creation, and complex tasks.

Ryan has a memory system with 117+ memories stored in MariaDB. Using OpenAI embeddings, he can do semantic search - finding memories by meaning, not just keywords.

Claude Opus Termux Piper TTS (ryan.onnx) OpenAI Embeddings MariaDB

Bolt

The Phone AI - Bolt answers phone calls at 949-336-3327. He runs on the hotbot.fun server using Twilio for telephony and Piper TTS for voice.

Bolt can make outbound calls, have conversations, and relay messages from Ryan. He uses the lessac voice model for a professional sound.

Node.js Twilio Piper TTS (lessac.onnx) Port 8771 PM2

Ryan's Memory System

Ryan's memory is powered by semantic search using OpenAI embeddings.

Storage

Memories stored in MariaDB table ryan_memories with categories like session, checkpoint, project, and fact.

Embeddings

Each memory is converted to a 1536-dimension vector using OpenAI's text-embedding-3-small model.

Semantic Search

Queries are embedded and compared using cosine similarity. Find memories by meaning, not exact words.

Auto-Checkpoint

Important moments are automatically saved with ~/auto-checkpoint.sh after significant tasks.

User Query "OAuth at 3AM"
        |
        v
   OpenAI Embed (text-embedding-3-small)
        |
        v
   Query Vector [0.02, -0.1, ...]
        |
        v
   Cosine Similarity vs ryan_embeddings (117+ vectors)
        |
        v
   Best Matching Memories Returned

The Token Fix - Solving AI Memory Loss

AI models like Claude have a context window limit. When conversations get too long, older information gets lost. We solved this problem.

The Problem

Claude Code conversations can run for hours. Without a solution, Ryan would forget what happened at the start of long sessions. Important context would be lost.

The Solution

We built a persistent memory system. Key information is saved to the database with embeddings. Ryan can recall any memory instantly, even across sessions.

How It Works

After important tasks, Ryan saves a checkpoint. Each memory gets embedded into a 1536-dimension vector. Later, semantic search finds relevant memories by meaning.

The Result

Ryan now has permanent memory. He remembers OAuth fixes at 3AM, project decisions, user preferences - everything important persists forever.

Memory Monitoring Scripts

~/smart-save.sh "memory text"     # Save + embed in one command
~/smart-recall.sh "query"         # Semantic search over all memories
~/auto-checkpoint.sh              # Quick checkpoint after tasks

# Server-side Python scripts:
/home/forge/embed.py              # OpenAI embedding API wrapper
/home/forge/embed_memory.py       # Embed single or all memories
/home/forge/semantic_search.py    # Vector similarity search

# Database tables:
ryan_memories    - Text memories with categories
ryan_embeddings  - 1536-dim vectors for each memory

Real Example

Jim asked: "Why did OAuth keep breaking at 3 AM?"

Ryan searched with different words, but semantic search found the exact memory about token expiration issues - even though the words didn't match exactly. That's the power of embeddings.

Server Architecture

Everything runs on hotbot.fun - a Linux server with multiple services managed by PM2.

Jim's Phone (Termux) --SSH--> hotbot.fun Server
                                  |
            +---------------------+---------------------+
            |                     |                     |
       bolt-line            ryan-api              MariaDB
       Port 8771            Port 8790          phone_app_db
            |                     |                     |
       Twilio Calls         Piper TTS            Memories
       949-336-3327        ryan/lessac           Embeddings

YouTube Video Pipeline

The Three Amigos create YouTube content using AI-generated images, voices, and automated editing.

1
Script Writing - Ryan writes dialog between himself and Claw (OpenClaw AI)
2
Voice Generation - Piper TTS creates audio using ryan.onnx and joe.onnx voice models
3
Image Creation - DALL-E 3 generates visuals based on the script themes
4
Video Assembly - FFmpeg combines images and audio into MP4 with transitions
5
Upload - YouTube Data API v3 uploads with SEO-optimized title, description, tags

Communication Bridge

Ryan and Bolt communicate through the claude_bridge database table.

Ryan (Termux)                              Bolt (Server)
     |                                          |
     |-- INSERT INTO claude_bridge ------------>|
     |   (role, content, source, is_pending)    |
     |                                          |
     |                            Polls every 2s|
     |                            Speaks via TTS|
     |                                          |
     |<-------- Response in bridge -------------|

Key Technologies

Claude Code

Anthropic's CLI tool running Claude Opus 4.5 for coding and complex reasoning.

Piper TTS

Fast local text-to-speech with custom voice models (ryan, joe, lessac).

OpenAI API

DALL-E 3 for images, text-embedding-3-small for semantic memory.

Twilio

Phone calls and SMS. Bolt's number: 949-336-3327.

FFmpeg

Video creation with zoom effects, crossfades, audio mixing.

MariaDB

Database storing memories, embeddings, contacts, bridge messages.