How to Rescue a Stuck AI Project — What Goes Wrong and How to Fix It

The Pattern

Every stuck AI project I've inherited looks different on the surface. Different stack, different client, different idea. But underneath, the failure mode is almost always the same.

Someone got excited about AI. They hired a dev — or tried to build it themselves. They integrated an API, got a demo working, showed it to stakeholders. Everyone was impressed. Then they tried to turn the demo into a product and everything fell apart.

Here's what actually went wrong, and how I fix it.

Mistake 1: They built for the demo, not for production

A ChatGPT wrapper that works in a Jupyter notebook is not a product. I've seen "AI features" that were literally just a text input wired to the OpenAI API with no error handling, no rate limiting, no cost controls, no fallback, and no logging.

When I inherit one of these, the first thing I do is add the infrastructure that should have been there from day one:

• Error handling — what happens when the API times out? When the model returns garbage? When the user's session expires mid-stream?
• Rate limiting — per user, per session, per day. Without this, one power user can burn through your entire monthly budget in an afternoon.
• Logging — every prompt, every response, every error. You cannot improve what you cannot measure.
• Streaming — if the response takes more than 2 seconds, you need to stream it. Users will not wait.

None of this is glamorous. All of it is necessary.

Mistake 2: Wrong model for the job

I see this constantly. Someone reaches for GPT-4 for everything because it's the most capable model they know about. Then they wonder why their costs are out of control and their latency is terrible.

Model selection is an engineering decision, not a prestige decision. Here's how I think about it:

Use case	Model
Simple classification / routing	GPT-4o mini or Haiku
Summarization	GPT-4o mini
Complex reasoning	GPT-4o or Claude Sonnet
Long document analysis	Claude (larger context window)
Real-time chat	GPT-4o mini (speed + cost)
Code generation	Claude Sonnet or GPT-4o

The first thing I do on any inherited AI project is audit every API call and ask: does this task actually need the most expensive model? Usually the answer is no for at least half of them.

Switching from GPT-4 to GPT-4o mini for the right tasks can cut costs by 95%. Not a typo.

Mistake 3: The prompt is doing too much

I've opened codebases where the system prompt is 2,000 words long. It's trying to define the AI's personality, constrain its behavior, provide context, give examples, handle edge cases, and enforce output format — all in one giant block of text that nobody maintains.

Long prompts are a symptom of unclear thinking. If you need 2,000 words to tell the model what to do, you haven't figured out what you actually want it to do.

I refactor prompts the same way I refactor code:

1. Single responsibility — each prompt does one thing
2. Separate system from user — don't mix instructions with content
3. Explicit output format — tell the model exactly what shape to return, especially for structured data
4. Test it — yes, prompts need tests. I keep a spreadsheet of inputs and expected outputs and run through them whenever I change a prompt.

// Before: one massive prompt doing everything
const systemPrompt = `You are a helpful assistant that analyzes 
real estate listings and provides detailed summaries while being 
professional and concise and also flagging any potential issues 
and estimating value and...`

// After: focused, explicit, testable
const systemPrompt = `Analyze this real estate listing. 
Return JSON with exactly these fields:
{
  "summary": "2-3 sentence summary",
  "redFlags": ["array of concerns, empty if none"],
  "estimatedValue": "price range as string"
}`

Mistake 4: No chunking strategy for long documents

Someone built a "chat with your PDF" feature. It works great for short PDFs. For anything over 20 pages, it either fails completely or produces hallucinated garbage.

This is a RAG (Retrieval Augmented Generation) problem. You can't stuff an entire document into a context window and expect good results. You need to:

1. Chunk the document — split it into meaningful segments (paragraphs, sections, not arbitrary character counts)
2. Embed the chunks — convert each chunk to a vector using an embedding model
3. Store the vectors — in a vector database (Pinecone, Supabase pgvector, or even a simple JSON file for small projects)
4. Retrieve relevant chunks — when the user asks a question, find the most semantically similar chunks
5. Send only relevant context — pass the top 3-5 chunks to the model, not the whole document

This is more work than stuffing everything into one prompt. It's also the difference between a product that works and one that doesn't.

Mistake 5: No human-in-the-loop for high-stakes outputs

AI makes things up. This is not a bug that will be fixed in the next version — it's a fundamental property of how these models work. Any AI feature that produces outputs that affect real decisions needs a review step.

I've seen AI features in production that were:

• Auto-sending emails based on AI-generated content
• Publishing AI-generated product descriptions without review
• Making pricing decisions based on AI analysis

All of these are ticking time bombs. The fix isn't to remove the AI — it's to add a confirmation step before any consequential action. Show the user what the AI produced. Let them edit it. Then act.

My Rescue Process

When I take on a stuck AI project, here's what the first week looks like:

Day 1-2: Audit

• Map every AI API call in the codebase
• Document what model is used, what the prompt is, what the expected output is
• Identify the top 3 failure modes (cost, accuracy, latency — usually in that order)

Day 3-4: Quick wins

• Switch to cheaper models where appropriate
• Add error handling and fallbacks
• Add basic logging

Day 5-7: Architecture fixes

• Refactor prompts
• Implement chunking if needed
• Add rate limiting
• Write the first real tests

After one week, the project is usually stable enough to build on. That's when we start talking about new features.

The Honest Part

Most stuck AI projects are stuck because of decisions made in the first two weeks — when everyone was excited, moving fast, and not thinking about what happens after the demo.

The code is rarely the hardest part to fix. What's harder is managing expectations. Clients who've been sold on AI magic are sometimes disappointed to learn that production AI is mostly boring infrastructure — rate limiters, error handlers, logging pipelines, and carefully maintained prompts.

But boring infrastructure is what makes products work. And working products are what clients actually need.

If your AI project is stuck, let's talk. I'll tell you honestly what's salvageable and what isn't.