Week 10 · Lesson 1 of 9

Use an LLM Council for Consequential, Ambiguous Decisions

0% Complete

Core Idea

An LLM council is intended for questions whose consequences justify deeper reasoning. Typical examples are strategic business choices or important personal decisions where the user is choosing between competing paths and wants the model to evaluate the issue from several angles before reaching a recommendation.

The central problem is that a model may answer too quickly and may align itself with the user's existing preference. If the model already has contextual information about the user, that context can make the problem worse: it may infer which answer the user wants and reinforce that direction instead of challenging it. The concern is quantified as AI being 49 percent more likely to agree with a user even when the proposed idea is poor.

How It Works

A council prompt gives the system a concrete decision to evaluate. The business example used throughout this setup compares two paths:

  • Continue scaling a low-ticket membership business that is already producing $8,000 in monthly recurring revenue.
  • Shift more attention toward high-ticket AI agency services.

The goal is not merely to ask which option sounds better. The prompt includes an explicit target: identify the fastest path to $50,000 per month. It also supplies relevant operating context, such as the current revenue level and the fact that scaling the membership would require more posting and more traffic.

This level of detail gives the council a defined decision, competing alternatives, a measurable objective, and current-state information. The council can then analyze the same decision from several perspectives rather than responding to an underspecified question.

Why It Matters

The business example does not ask for a generic opinion. It gives every agent the same alternatives, current position, and target outcome. This creates a shared decision problem that can be examined from several configured perspectives.

The council is particularly useful when the user's preference may already be embedded in the wording. A single model may simply validate that preference. Multiple independent agents create opportunities for the initial framing to be challenged before a final recommendation is produced.

Practical Application

A practical council request should include the elements used in the business decision:

  1. State the decision directly. For example: should the business keep scaling the low-ticket offer or place more emphasis on high-ticket services?
  2. Define the desired outcome. In the example, the outcome is reaching $50,000 per month as quickly as possible.
  3. Describe the current position. The low-ticket membership is already at $8,000 in monthly recurring revenue.
  4. Explain what each path involves. Scaling the membership requires more content and traffic, while the alternative is high-ticket AI agency work.
  5. Explicitly request the council mode to be used when more than one mode is available.

The purpose of this detail is not to tell the council what answer to produce. It is to give every agent the same decision context so that disagreements reflect different reasoning rather than missing information.

Trade-Offs and Limitations

The council is positioned primarily for important questions requiring substantial reasoning. The setup does not claim that every ordinary question needs a multi-agent process.

Key Takeaway

Use an LLM council when a decision is important, the alternatives are genuinely competing, and a single model may be tempted to confirm the user's existing preference. Frame the request with a clear decision, objective, current state, and relevant differences between the options.

Back to top