Most developers use AI to write functions. Some use it to write features. Very few use it where it might be most valuable: system design.

We’re not talking about asking ChatGPT to draw a box diagram. We’re talking about using AI as a thinking partner for the hardest decisions in software engineering — the ones where there’s no right answer, just trade-offs.

The function trap

There’s a gravitational pull toward using AI at the lowest level of abstraction. Write a function. Generate a component. Create an endpoint. It feels productive because you see immediate output.

But here’s the uncomfortable truth: the most expensive mistakes in software happen at the architecture level, not the function level. Choosing the wrong database costs you six months. A bad service boundary creates a year of integration pain. An auth architecture that doesn’t scale means a rewrite when you have 10,000 users instead of 100.

Functions are cheap. Architecture decisions compound.

So why aren’t more developers using AI for architecture? Because it requires a different kind of interaction.

How to use AI for system design

1. Trade-off analysis

This is the single most valuable architectural use of AI. You’re choosing between two (or five) approaches and you need to think through the implications.

I'm building a multi-tenant SaaS application. I need to 
decide between:

A) Shared database, tenant_id column on every table
B) Schema-per-tenant in a shared PostgreSQL instance  
C) Separate database per tenant

Our context:
- Expected scale: 50-500 tenants, 10-50K rows per tenant
- Team size: 4 developers
- Compliance: SOC 2, some tenants will ask about data isolation
- Budget: startup, cost matters

Analyze the trade-offs across: data isolation, operational 
complexity, query performance, migration difficulty, and cost.

The AI won’t make the decision for you. It will lay out the trade-offs more thoroughly than you’d do on a whiteboard in an hour. It’ll surface considerations you forgot — like how schema-per-tenant interacts with connection pooling, or how shared-database makes tenant data export for compliance requests harder.

You still decide. But you decide with better information.

2. Architecture review

Hand the AI your existing architecture and ask it to poke holes:

Here's our current architecture:
- React frontend → API Gateway → 3 microservices
- Services communicate via REST
- Shared PostgreSQL database
- Redis for caching
- Deployed on AWS ECS

We're seeing increasing latency as our data grows. 
Review this architecture and identify potential bottlenecks, 
scaling concerns, and architectural risks.

A good AI will identify issues like:

It’s like having a senior architect review your system — except it’s available at 2 AM and doesn’t charge consulting rates.

3. Technology selection

“Should we use Kafka or RabbitMQ?” is a question that launches a thousand Reddit debates. AI can cut through the noise:

We need an event streaming solution for order processing.

Requirements:
- ~1000 events/second initially, growing to 50K
- Events must be processed in order per customer
- At-least-once delivery is fine
- Team has no Kafka experience but some RabbitMQ exposure
- Running on AWS, open to managed services

Compare: Kafka (self-hosted), Amazon MSK, RabbitMQ, 
Amazon SQS + SNS, and Amazon Kinesis.

Focus on: operational burden, partition ordering guarantees, 
cost at our scale, and learning curve.

The AI won’t just list features. It’ll tell you that Kafka’s partition ordering maps perfectly to your per-customer ordering requirement, but that your team’s lack of Kafka experience plus self-hosting operational burden might make MSK or even SQS with FIFO queues a more practical choice at your current scale.

4. Migration planning

Migrating from monolith to microservices? Changing databases? Moving to a new cloud provider? These are high-stakes architectural changes where AI excels as a planning partner.

We need to migrate our monolithic Django app to a service-
oriented architecture. The monolith has:
- 120 Django models
- 40 API endpoints
- 15 background Celery tasks
- Tight coupling between the billing and user modules

Propose a phased migration strategy. What do we extract 
first? What are the riskiest boundaries? How do we handle 
the shared database during migration?

The AI will propose a strangler fig pattern, identify which modules have the cleanest boundaries for extraction, and warn you about the foreign key relationships that will make the billing extraction painful.

The anti-patterns

AI-assisted system design goes wrong in predictable ways:

Asking for “the best” architecture. There’s no best. There are trade-offs. Frame your questions around constraints and trade-offs, not absolute rankings.

Accepting the first suggestion. AI tends to suggest popular, well-documented architectures. That’s often right, but sometimes your constraints call for something unconventional. Push back. Ask “what if we did X instead?” and make the AI argue against its own suggestion.

Ignoring your team’s capabilities. The AI will happily design a Kubernetes + Kafka + gRPC + event-sourcing architecture for your three-person team. Architectural sophistication that exceeds your team’s ability to operate it is a liability, not an asset. Always include team size and experience in your context.

Using AI diagrams as documentation. AI-generated architecture diagrams are useful for thinking. They’re terrible as actual documentation because they lack the nuance of why decisions were made. Use AI for exploration, then write your Architecture Decision Records yourself.

The design session format

Here’s a format that works well for AI-assisted design sessions:

Round 1 — Context dump (5 min) Give the AI everything: current architecture, team size, growth projections, constraints, compliance requirements, budget. More context produces dramatically better analysis.

Round 2 — Exploration (15 min) Ask broad questions. “What are the architectural options for X?” Let the AI map the solution space before you narrow down.

Round 3 — Deep dive (20 min) Pick 2-3 options and go deep. Ask about failure modes, scaling limits, operational burden, migration paths. This is where AI earns its keep — it explores corners you’d skip.

Round 4 — Stress test (10 min) Play devil’s advocate. “What happens when we hit 100x our current load? What if we lose a team member? What if this compliance requirement changes?” Force the AI to find weaknesses in the approach you’re leaning toward.

Round 5 — Document (10 min) Have the AI draft an Architecture Decision Record: the options considered, the trade-offs analyzed, and the decision made with rationale. Edit it yourself — but let the AI do the first draft.

The coductor’s perspective

System design is where the coductor metaphor becomes most literal. You’re not writing code. You’re not even directing code-writing. You’re shaping the structure that all future code will live inside.

AI doesn’t replace your judgment here. It amplifies it. It lets you consider more options, analyze more trade-offs, and stress-test more failure modes than you could alone. The decision is still yours. But the decision-making process just got dramatically more thorough.

The best architects aren’t the ones who know every technology. They’re the ones who ask the best questions. AI is the tool that lets you ask more of them.

Design better systems

Coductor members share real architecture decisions, design session recordings, and the AI prompts that led to better systems. Join the conversation.

Join the Community

Architecture System Design AI Development Trade-offs Best Practices