Skip to main content

64. API Gateway for AI Services

Status: Accepted Date: 2025-07-06

Context​

Our system may include multiple AI and machine learning services over time: the local mercury-ai module based on Ollama, potentially a specialized mercury-ta service for technical analysis, or other future analysis components. Exposing each of these services with its own public API would create a fragmented and complex API surface for clients to interact with. Clients would need to know the specific address and contract for each individual AI service.

Decision​

We will use the API Gateway Pattern, with the Morpheus module acting as the single, unified gateway for all AI/ML services. All external and internal requests for AI-powered analysis, predictions, or signals must go through the Morpheus API.

Morpheus's responsibilities include:

  1. Providing a stable, unified set of REST API endpoints for all AI functionality.
  2. Authenticating and authorizing requests.
  3. Routing incoming requests to the appropriate backend AI service (e.g., mercury-ai, mercury-ta).
  4. Potentially handling cross-cutting concerns like rate limiting, caching, and request/response transformations.

Clients will have no direct knowledge of or access to the underlying AI services; they will only communicate with Morpheus.

Consequences​

Positive:

  • Simplified Client Integration: Clients have a single, stable API endpoint to interact with for all AI needs. They don't need to know about the internal architecture of our AI services.
  • Decoupling and Flexibility: We can add, remove, or refactor the internal AI services behind the gateway without impacting external clients, as long as the Morpheus API contract is maintained.
  • Centralized Cross-Cutting Concerns: Provides a single place to implement and enforce policies like authentication, rate limiting, and logging for all AI services.
  • Improved Security: By hiding the internal services, we reduce the overall attack surface of the system.

Negative:

  • Single Point of Failure: If the Morpheus gateway goes down, all AI functionality becomes unavailable.
  • Potential for Increased Latency: Every request to an AI service now involves an extra network hop through the gateway.
  • Development Bottleneck: The Morpheus module could become a bottleneck for development if all teams wanting to expose a new AI feature need to make changes to it.

Mitigation:

  • High Availability: The Morpheus gateway will be a stateless service and will be deployed as multiple redundant instances to ensure high availability.
  • Low-Latency Network: The gateway will be deployed in the same low-latency network as the backend AI services it calls, minimizing the additional network hop time.
  • Automated Routing/Discovery: For more advanced scenarios, Morpheus could use a service discovery mechanism to automatically route requests to newly registered AI services, reducing the need for manual code changes in the gateway itself.
  • Clear Ownership: The Morpheus module will have clear ownership and a well-defined process for how other teams can integrate their services behind it.