11. Provider & Model System

1. Goal

PI-Desktop must support all major market model vendors and models that users commonly need, without hardcoding a tiny allowlist as product ceiling.

Strategy:

Universal provider coverage via pi-ai + OpenAI-compatible escape hatch + refreshable model catalogs.

We do not re-implement every vendor SDK ourselves.
We standardize on pi’s multi-provider layer and add product-level configuration, catalog, and UX.

2. Coverage principle

Must support

1. First-party major vendors
2. Popular aggregators / gateways
3. Any OpenAI-compatible endpoint
4. User-defined custom providers
5. Continuous model catalog refresh

Product promise

• Users can connect practically any mainstream vendor/model available through:
  • native pi provider integrations
  • OpenAI-compatible APIs
  • custom provider definitions

Explicit non-promise

• Guaranteeing every obscure vendor’s proprietary non-standard protocol without an adapter
• Shipping offline full world-model matrix forever without catalog updates

3. Architecture

Settings / UI
  → ProviderConfigStore (Rust host DB)
  → AgentRuntime (Node/pi)
      ├─ built-in vendor providers (via pi-ai)
      ├─ openai-compatible provider
      └─ custom provider definitions
  → ModelCatalogService
      ├─ bundled catalog snapshot
      ├─ runtime discovery (where supported)
      └─ refresh from pi model data / remote catalog source

4. Provider types

type	description	examples
native	first-class vendor integration via pi-ai	openai, anthropic, google, bedrock, mistral, etc.
openai_compatible	any OpenAI Chat Completions/Responses compatible gateway	OpenRouter, Together, Groq, Fireworks, DeepSeek, local gateways, corporate proxies
custom	user-defined provider based on known protocol profile	private deployments, regional gateways

Protocol profiles (MVP):

1. openai
2. anthropic
3. google
4. openai_compatible
5. bedrock (if enabled by runtime support)
6. custom_http (advanced/experimental later)

5. Built-in vendor matrix (ship intent)

Exact availability depends on pi-ai support at pin version; product must expose all supported ones and keep OpenAI-compatible path open for the rest.

Tier A — always exposed in UI

• OpenAI
• Anthropic
• Google Gemini
• OpenAI-Compatible (generic)

Tier B — expose when runtime supports / enable by default if present in pi-ai

• AWS Bedrock
• Azure OpenAI / OpenAI on Azure
• Mistral
• xAI
• DeepSeek
• Groq
• Together
• Fireworks
• Cohere
• Perplexity
• OpenRouter
• Moonshot / Kimi
• Zhipu / GLM
• MiniMax
• Baichuan
• Qwen / DashScope
• 01.AI / Yi
• SiliconFlow
• NVIDIA NIM
• Ollama (local)
• LM Studio (local OpenAI-compatible)
• vLLM / TGI / LocalAI / LiteLLM gateways (via OpenAI-compatible)

Tier C — user custom

Any vendor not listed but reachable by:

• OpenAI-compatible base URL
• custom headers
• custom auth scheme

6. Model support policy

6.1 No hard model allowlist ceiling

PI-Desktop must not permanently restrict users to a short fixed model list.

6.2 Catalog responsibilities

1. pi-ai's bundled catalog is the sole runtime metadata source for known models.
2. Runtime-native discovery and the durable cache provide selection and offline availability, but never rewrite known-model runtime semantics.
3. User-defined model ids remain selectable; ids absent from pi use the explicit generic fallback.
4. pi-ai upgrades refresh the authoritative model metadata snapshot. The current pin is @earendil-works/pi-ai / pi-agent-core ^0.82.1+. That snapshot includes Claude Opus 5 (claude-opus-5 and provider-native aliases such as Bedrock inference profiles and OpenRouter anthropic/claude-opus-5) with 1M context, adaptive thinking, and the published thinking-level map. Free-form gateway ids that match those catalog entries resolve through the same D136 path; ids still absent from the pin remain on the generic non-reasoning fallback.

6.3 Model families to cover

Catalog and custom model entry must support common capability classes:

• text chat / coding models
• reasoning / thinking models
• long-context models
• vision / multimodal input models
• tool-calling capable models
• JSON/structured output capable models (where provider supports)

7. Configuration schema

type ProviderAuthKind =
  | "api_key"
  | "api_key_and_base_url"
  | "bearer"
  | "azure_api_key"
  | "aws_sdk_default"
  | "custom_headers"
  | "none" // local no-auth

type ProviderConfig = {
  id: string                    // uuid/ulid
  name: string                  // display name
  vendorKey: string             // openai/anthropic/google/openrouter/custom/...
  type: "native" | "openai_compatible" | "custom"
  protocol: "openai" | "anthropic" | "google" | "openai_compatible" | "bedrock" | "custom_http"
  enabled: boolean
  baseUrl?: string
  authKind: ProviderAuthKind
  secretRef?: string            // pointer into secret store
  headers?: Record<string, string> // non-secret headers only
  apiStyle?: "chat_completions" | "responses" | "auto"
  compatibility?: {
    supportsTools?: boolean
    supportsVision?: boolean
    supportsStreaming?: boolean
    supportsReasoning?: boolean
    supportedThinkingLevels?: ThinkingLevel[]
  }
  defaultModelId?: string
  models?: UserModelConfig[]    // optional user-defined models
  createdAt: string
  updatedAt: string
}

type UserModelConfig = {
  id: string                    // provider-local model id/slug
  displayName: string
  providerId: string
  contextWindow?: number
  maxOutputTokens?: number
  capabilities?: Array<"text" | "tools" | "vision" | "reasoning" | "json">
  pricingHint?: string
  hidden?: boolean
}

type SelectedModelRef = {
  providerId: string
  modelId: string
}

type ThinkingLevel =
  | "off"
  | "minimal"
  | "low"
  | "medium"
  | "high"
  | "xhigh"
  | "max"

The compatibility fields above are retained as a persisted-schema compatibility surface for older clients. PI-Desktop no longer reads them as runtime model overrides. Reasoning support and supported thinking levels come from the resolved pi-ai model record; unknown free-form ids expose no inferred reasoning capability.

8. Secrets

• API keys stored via secure storage (SECRET_* APIs)
• Provider config stores only secretRef / hasSecret boolean
• Renderer never receives raw key in list APIs
• Optional key validation call: providers.testConnection

9. Model catalog service

interface ModelCatalogService {
  listProviders(): Promise<ProviderDescriptor[]>
  listModels(filter?: ModelQuery): Promise<ModelDescriptor[]>
  refreshCatalog(options?: { providerId?: string }): Promise<RefreshResult>
  resolveModel(ref: SelectedModelRef): Promise<ResolvedModel>
  upsertUserModel(model: UserModelConfig): Promise<void>
}

ModelDescriptor

type ModelDescriptor = {
  providerId: string
  vendorKey: string
  modelId: string
  displayName: string
  source: "bundled" | "discovered" | "user"
  capabilities: Array<"text" | "tools" | "vision" | "reasoning" | "json">
  contextWindow?: number
  maxOutputTokens?: number
  deprecated?: boolean
  tags?: string[]
  supportedThinkingLevels?: ThinkingLevel[]
}

10. UI requirements

Settings → Agent → Providers

• add built-in vendor quickly
• add OpenAI-compatible endpoint
• add custom provider
• edit base URL/headers
• set/replace/delete key
• enable/disable provider
• test connection
• do not expose reasoning, thinking-level, context-window, output-limit, temperature, or compatibility overrides for a selected model

Model selector

• search all models across enabled providers
• group by provider/vendor
• show capability badges (tools/vision/reasoning)
• allow “refresh models”
• allow custom model id entry

Empty/error states

• no provider configured
• key missing
• model not found
• provider unauthorized
• catalog refresh failed (still allow manual model id)

11. Runtime resolution algorithm

When starting a turn with (providerId, modelId):

1. load provider config from host
2. if missing/disabled → fail (MODEL_NOT_CONFIGURED; reserved detail: PROVIDER_DISABLED)
3. resolve secret via secretRef (never log secret; missing → PROVIDER_SECRET_MISSING)
4. resolve the complete pi-ai model record by exact vendor/id or a compatible gateway alias with a separator-bounded suffix
5. when resolved, copy pi's name, reasoning flag, thinking-level map, input modes, pricing, context window, output limit, headers, and compatibility verbatim; when unresolved, accept the raw model id with the generic text-only, non-reasoning fallback
6. clamp the session thinking level against pi's supported levels and build the runtime provider adapter by replacing only provider/model identity, selected API adapter, auth, and an explicitly configured endpoint URL
7. execute stream with abort handle and separate answer/thinking events
8. translate vendor errors into shared AppError codes (§15)

If the model is not in pi's catalog, still allow it when the user explicitly enters a model id and the provider accepts unknown ids. Cached/discovered capability fields do not promote that fallback into a known runtime model.

12. Compatibility tiers

tier	meaning
full	tools + streaming + vision verified/expected
standard	chat streaming expected
limited	best-effort via compatible gateway
unknown	user custom, no guarantees

UI may show tier hints, but must not hard-block unknown models by default.

13. Refresh & update policy

1. App ships with bundled catalog snapshot
2. User can click Refresh model catalog
3. Refresh may update:
   • discovered models for providers with list APIs
   • bundled catalog via app update channel
4. Refresh failure must not wipe existing catalog

14. Local / offline model support

Supported via OpenAI-compatible local servers:

• Ollama (native if pi supports it, otherwise OpenAI-compatible proxy)
• LM Studio
• vLLM / TGI / LocalAI / LiteLLM proxies
• other local gateways

Requirements:

• custom base URL
• auth may be none
• manual model id entry always available
• catalog refresh may use /v1/models when available; otherwise user-defined models

15. Failure taxonomy (provider domain)

Canonical codes live in 08-error-codes; reserved detail codes map to a canonical parent until emitted (§3.6 there).

code	status	meaning	user-facing guidance
PROVIDER_UNAUTHORIZED	live	invalid/expired key or denied auth	re-enter secret / check account
PROVIDER_RATE_LIMITED	live	429 / quota	retry later / switch model
PROVIDER_SECRET_MISSING	live	enabled provider without secret	complete setup
MODEL_NOT_CONFIGURED	live	no selected model or provider rejects selected model with 404	select or configure an available model
PROVIDER_ERROR	live	other upstream provider failure	retry / inspect details
NETWORK_ERROR	live	provider endpoint cannot be reached	check network and base URL
STREAM_FAILED	live	stream dropped mid-turn	retry turn
PROVIDER_BASE_URL_INVALID	reserved → PROVIDER_ERROR	malformed or unreachable base URL	fix endpoint
PROVIDER_PROTOCOL_MISMATCH	reserved → PROVIDER_ERROR	wrong protocol for endpoint	switch protocol profile
PROVIDER_MODEL_NOT_FOUND	reserved → MODEL_NOT_CONFIGURED	model id unknown for provider	refresh catalog or custom id
PROVIDER_TIMEOUT	reserved → TIMEOUT	network or server timeout	retry / check network
PROVIDER_UNSUPPORTED_CAPABILITY	reserved → PROVIDER_ERROR	tools/vision/reasoning unsupported	switch model or disable feature
PROVIDER_DISABLED	reserved → MODEL_NOT_CONFIGURED	provider exists but disabled	enable provider

16. OpenAI-compatible first-class path

Any vendor can be onboarded without a native SDK if it exposes OpenAI-compatible APIs.

Required fields:

• baseUrl
• auth (api_key / bearer / none / custom headers)
• model id (catalog or free-form)

Optional:

• apiStyle (chat_completions | responses | auto)
• compatibility flags
• custom headers (non-secret)

This is the universal escape hatch guaranteeing market coverage beyond native integrations.

17. Multi-provider product rules

1. Multiple providers of same vendorKey are allowed (e.g. two OpenRouter accounts).
2. Provider name is user-editable and unique per workspace/user profile.
3. Default app model is a (providerId, modelId) pair, not modelId alone.
4. Session stores its own (providerId, modelId) binding.
5. Deleting a provider blocks new turns that reference it; historical sessions keep the ids for audit/display.
6. Export settings never includes raw secrets.
7. Import settings can recreate provider shells and prompt for secrets.
8. Reasoning capability is model-specific unless the provider has an explicit compatibility override; provider defaults must not override a session's selected model during turn resolution.

18. Validation rules

• name required
• vendorKey required
• protocol required
• baseUrl required for openai_compatible/custom when endpoint not implicit
• secret required when authKind needs key
• headers must not contain raw api keys (use secret store)
• model id non-empty

19. Acceptance criteria

• [ ] Add OpenAI / Anthropic / Google / OpenAI-Compatible providers from UI
• [ ] Add arbitrary OpenAI-compatible custom provider with base URL + key
• [ ] Select models via catalog search across providers
• [ ] Free-form model id accepted when catalog misses it
• [ ] Catalog refresh populates models for at least one native and one compatible provider, without destroying existing providers
• [ ] Connection test returns structured success/failure without secret leakage
• [ ] Session can switch model between turns
• [ ] Reasoning-capable models expose only supported thinking levels and the selected level reaches pi; unsupported providers resolve to off
• [ ] Missing key/model blocks run with stable, actionable error codes
• [ ] At least one local provider path (Ollama or LM Studio style) documented and testable
• [ ] No product hard-limit like “only 3 vendors / 10 models”

20. Non-goals (MVP)

• Building our own full provider SDK ecosystem
• Guaranteeing identical tool/vision quality across all vendors
• Marketplace of providers (not needed; config is local)
• Full multi-modal attachment studio beyond model capability flags
• Automatic paid-plan discovery for every vendor portal
• Proprietary non-HTTP SDKs without pi-ai support
• Cloud-synced provider profiles