Skip to main content

Eliminating Type Safety Bypasses: The Global Optional Nested Structure Problem

Β· 4 min read
Max Kaido
Max Kaido
Architect

The Problem: Repository-Wide Type Safety Nightmares​

This is a monorepo-wide architectural problem, not specific to Mercury. Across multiple domains, we've evolved complex optional nested structures that force developers to bypass TypeScript's type safety. The Mercury tournament configurations are just one example of this systemic issue.

The pattern appears everywhere:

  • Configuration objects with deeply nested optional fields
  • Feature flags with optional nested settings
  • API responses with optional nested data structures
  • Plugin systems with optional nested configurations

Here's the Mercury example that triggered this investigation:

// The problematic structure
experimental?: {
  experimentId?: string;
  comparisonMethod?: string;  // Should be enum
  normalizePrice?: boolean;
  // ... other optional fields
}

This created multiple cascading problems:

1. Optional Nested Structures Encourage Type Casts​

When everything is optional, TypeScript returns string | undefined for nested access. Developers were forced to use type casts like config.experimental?.comparisonMethod as ComparisonMethod to silence compiler errors.

2. Silent Production Bugs​

These type casts bypass validation entirely. The system appears to work but:

  • Settings don't apply due to bypassed validation
  • Fallbacks trigger instead of real logic
  • Business logic operates with wrong parameters
  • In Mercury: Financial losses discovered weeks later
  • In other domains: Data corruption, failed integrations, incorrect behavior

3. Evolving Feature Complexity​

Different features/experiments add different fields to the same optional objects, creating impossible-to-type unions when combining legacy and current configurations across all domains.

4. TypeScript Becomes an Obstacle​

Instead of helping catch bugs, TypeScript became something to work around with as any and other dangerous casts throughout the entire monorepo.

The Solution: Architectural Flattening​

Following the principle of "moving toward the problem rather than away from it", we implemented a radical architectural fix. This approach applies to any domain with similar optional nested structure problems.

Mercury Example Implementation​

1. Eliminate the experimental Field Entirely​

// Before: Nested optional structure
experimental?: {
  comparisonMethod?: ComparisonMethod;
  normalizePrice?: boolean;
  // ...
}

// After: Flattened required fields
topK: number;
topL: number;
comparisonMethod: ComparisonMethod;
normalizePrice: boolean;

2. Make All Fields Mandatory​

No optional chaining, no undefined handling. Every tournament config must explicitly declare all parameters.

3. Remove Runtime Parameters from Base Configs​

Fields like experimentId and variant are runtime parameters, not part of base tournament configuration. They're added separately when tournaments are instantiated.

4. Strategy-Specific Comparison Methods​

Each tournament type gets an appropriate comparison method:

  • STRUCTURED: For momentum, order-blocks, sweeps, divergence (schema-based)
  • HEURISTIC: For volatility breakouts (pattern-based)
  • TWO_STEP: For smart money patterns (detailed analysis)

Implementation Results​

Before (Dangerous)​

// Encouraged type casts
const method = config.experimental?.comparisonMethod as ComparisonMethod;
const normalize = config.experimental?.normalizePrice ?? false;

After (Type Safe)​

// Direct access, no casts needed
const method = config.comparisonMethod;
const normalize = config.normalizePrice;

Files Modified (8 Tournament Configs)​

  • momentum-strength-down/v2.ts
  • momentum-strength-up/v2.ts
  • volatility-breakout-up/v2.ts
  • volatility-breakout-down/v2.ts
  • smart-money-up/v2.ts
  • smart-money-down/v2.ts
  • order-block-buy/v1.ts
  • sweep-rejection-down/v1.ts
  • divergence-reversal-up/v1.ts

Universal Principles for All Domains​

1. No Scripts for Mass Changes​

Critical: Each configuration file has business impact. Never use scripts for mass changes. Each file must be edited manually with full understanding of its domain context. This applies to all domains, not just Mercury.

2. Type Casts Are Production Bug Factories​

Any occurrence of as any, as SomeType, or similar casts should be treated as critical bugs. They create silent failures where systems appear to work but generate incorrect behavior across all business domains.

3. Optional Nested Structures Are Universal Anti-Patterns​

Complex optional nested objects encourage type safety bypasses everywhere in the monorepo. Flatten complexity and require explicitness rather than accommodating ambiguity with loose typing.

4. Architecture Over Compiler Flags​

Proper type safety requires architectural discipline, not just strict: true. The solution is structural: eliminate the conditions that force developers to bypass type safety in any domain.

Remaining Work (Repository-Wide)​

This refactoring is incomplete and needs to be applied across all domains. Future agents continuing this work must:

Mercury Domain (In Progress)​

  1. Update Service Files: All services accessing config.experimental.* need updates to use direct field access
  2. Fix Type Imports: Several files have import issues with @kaido/types
  3. Update Tournament Instantiation: Runtime parameters (experimentId, variant) need to be added during tournament creation, not in base configs
  4. Archive Completed Experiments: Move completed experiments to conducted/ directory and exclude from TypeScript checking

Other Domains (Not Started)​

  1. Identify Similar Patterns: Search for optional nested structures in configuration objects
  2. Audit Type Casts: Find all as any, as SomeType bypasses across the monorepo
  3. Apply Flattening Pattern: Use the same architectural approach in each domain
  4. Domain-Specific Context: Each domain has different business impact - understand it before making changes

Context for Future Agents​

This is not about cleaner codeβ€”it's about preventing production failures from silent type safety bypasses.

  • Mercury: Each config directly impacts trading decisions and financial outcomes
  • Other domains: Each has its own critical business impact that type casts can silently break

When you see type casts in any domain, you're looking at potential production bugs. Fix the architecture, don't accommodate the complexity.

This architectural change follows the Mercury domain principle: make wrong states unrepresentable through type system design, not runtime validation.

Battle-Tested Protocol Addendum​

Critical Lessons from Implementation​

  1. Read ENTIRE files, not just error lines - Type casts hide in unexpected places. Standard grep searches miss them.

  2. Human Override Pattern works: When agent gets stuck β†’ Human points to specific file β†’ Agent reads completely β†’ Fixes root cause, not symptoms.

  3. Concrete search commands:

    grep -r "\.experimental\." src/  # Find nested access patterns
    grep -r " as " src/ | grep -v test  # Find type casts (they hide)

  4. Each file needs individual analysis - No batch fixes. Each config has business impact.

  5. Update imports matter - Adding ComparisonMethod to imports often forgotten but breaks compilation.

  6. strict: true doesn't help - Agent initially tried enabling strict TypeScript mode, creating 1429 noise errors (unused params, null checks). Type casts still compile! The problem is architectural, not compiler flags.

This debugging session took over an hour because type casts are genuinely hard to find systematically.