Skip to Content
OverviewPrediction engine

Prediction Engine

Deep-dive into the balance prediction algorithm — inputs, scenario generation, volatility modeling, and API endpoints.

The Prediction Engine is the core component responsible for generating balance projections based on historical data, scheduled transactions, and user-defined simulations.

Overview

The prediction engine (TransactionService.predictBalance) generates time-series projections of account balances by combining multiple data sources and applying configurable algorithms. It produces realistic forecasts that account for uncertainty and variability in financial behavior.

Prediction Pipeline

The prediction process follows a structured pipeline:

Step-by-Step Process

1. Load Historical Transactions

The engine loads all committed transactions up to the prediction start date to calculate the starting balance:

val transactionsUpToStart = Transaction.find {
    (Transactions.user eq userId) and
    (Transactions.status eq Status.ACTIVE) and
    (Transactions.date.date() lessEq fromDate)
}.toList()

Key Points:

  • Only ACTIVE transactions are included
  • Transactions are filtered by user and optionally by account
  • Starting balance = initial account balance + sum of all transactions up to start date

2. Generate Date Points

Based on the configured granularity, the engine generates a list of dates:

  • DAILY: One point per day
  • WEEKLY: One point per week
  • MONTHLY: One point per month
var currentDate = fromDate
while (currentDate <= toDate) {
    datePoints.add(currentDate)
    currentDate = when (granularity) {
        "WEEKLY" -> currentDate.plus(DatePeriod(days = 7))
        "MONTHLY" -> currentDate.plus(DatePeriod(months = 1))
        else -> currentDate.plus(DatePeriod(days = 1))
    }
}

3. Load Committed Transactions

All transactions within the prediction date range are loaded and grouped by date:

val committedTransactions = Transaction.find {
    (Transactions.date.date() greaterEq fromDate) and
    (Transactions.date.date() lessEq toDate)
}.groupBy { it.date.date }

Transaction Types:

  • INCOME: Adds to balance (+amount)
  • EXPENSE: Subtracts from balance (-amount)
  • TRANSFER: No net effect on total balance (0)

4. Generate Scheduled Transactions

If includeScheduledTransactions is enabled, the engine projects scheduled transactions forward:

ScheduledTransaction.find { ... }.flatMap { scheduled ->
    generateOccurrencesBetween(
        startDate = scheduled.startDate.date,
        frequency = scheduled.frequency,
        interval = scheduled.interval,
        from = fromDate,
        to = toDate
    ).map { date -> ... }
}

Frequency Patterns:

  • DAILY: Every day
  • WEEKLY: Every week (with optional interval)
  • MONTHLY: Every month
  • YEARLY: Every year

The generateOccurrencesBetween function calculates all occurrences of a scheduled transaction within the date range.

5. Apply Periodic Spends

User-defined periodic spending patterns are applied:

filters.periodicSpends.flatMap { periodicSpend ->
    generateOccurrencesBetween(
        startDate = fromDate,
        frequency = periodicSpend.frequency,
        interval = 1,
        from = fromDate,
        to = toDate
    ).map { date -> -periodicSpend.amount }
}

Periodic spends are expenses that occur at regular intervals (e.g., “spend $50 every week on groceries”).

6. Replicate Past Month Behavior

If includePastMonthBehavior is enabled, transactions from the previous month are replicated forward:

val pastMonthStart = fromDate.minus(DatePeriod(months = 1))
val pastMonthEnd = fromDate.minus(1, DateTimeUnit.DAY)
 
val pastMonthTransactions = Transaction.find {
    (Transactions.date.date() greaterEq pastMonthStart) and
    (Transactions.date.date() lessEq pastMonthEnd)
}.toList()
 
// Shift dates forward by one month
pastMonthTransactions.map { pastTx ->
    val newDate = pastTx.date.date.plus(DatePeriod(months = 1))
    // Only include if within prediction range
    if (newDate >= fromDate && newDate <= toDate) {
        // Replicate transaction
    }
}

This feature helps model recurring behavior that isn’t explicitly scheduled (e.g., irregular but frequent expenses).

7. Combine All Transactions

All transaction sources are combined into a single map by date:

val allTransactions: Map<LocalDate, Double> = 
    (committedTransactions.keys + 
     scheduledTransactions.keys + 
     periodicSpends.keys + 
     pastMonthSimulatedTransactions.keys)
    .associateWith { date ->
        (committedTransactions[date] ?: 0.0) +
        (scheduledTransactions[date] ?: 0.0) +
        (periodicSpends[date] ?: 0.0) +
        (pastMonthSimulatedTransactions[date] ?: 0.0)
    }

8. Calculate Category Volatility

The engine calculates volatility scores for each category to inform scenario generation:

val categoryVolatility = calculateCategoryVolatility(userId, accountIds, toDate)

Volatility is measured as the coefficient of variation (standard deviation / mean) of historical spending in each category. Higher volatility indicates more unpredictable spending patterns.

9. Generate Scenarios

For each date point, the engine generates three scenarios:

  • Base: Standard projection
  • Optimistic: Best-case outcomes
  • Pessimistic: Worst-case outcomes

Scenarios are generated by applying noise variations based on category volatility:

val noiseMultiplier = when (scenarioType) {
    ScenarioType.OPTIMISTIC -> 1.0 - (noisePercentage / 100.0)
    ScenarioType.PESSIMISTIC -> 1.0 + (noisePercentage / 100.0)
    else -> 1.0
}

10. Apply Noise

If noisePercentage is configured, random variation is applied to transactions:

if (noisePercentage != null) {
    val noiseRange = noisePercentage / 100.0
    val randomFactor = 1.0 + (random.nextDouble() * 2.0 - 1.0) * noiseRange
    adjustedAmount = baseAmount * randomFactor
}

Noise simulates real-world uncertainty where actual spending may vary from expected amounts.

11. Calculate Balance Points

For each date point, the balance is calculated:

datePoints.forEach { date ->
    val transactionAmount = allTransactions[date] ?: 0.0
    baseBalance += transactionAmount
    
    // Apply noise if configured
    val balanceWithNoise = if (noisePercentage != null) {
        applyNoise(baseBalance, noisePercentage, random)
    } else {
        baseBalance
    }
    
    // Generate scenarios
    val scenarios = generateScenarios(
        baseBalance, 
        categoryVolatility, 
        noisePercentage
    )
    
    balancePoints.add(BalancePredictionPoint(
        date = date.toString(),
        balance = balanceWithNoise,
        transactions = getTransactionDetails(date),
        scenarios = scenarios
    ))
}

12. Analyze Safety and Goals

After generating all balance points, the engine performs additional analyses:

  • Safe-to-Spend Calculation: Determines how much can be safely spent today
  • Danger Event Detection: Identifies dates where balances fall below thresholds
  • Goal Analysis: Evaluates progress toward savings goals
  • Category Forecasting: Projects spending by category with volatility

Key Algorithms and Concepts

Pattern Detection

The engine doesn’t explicitly detect patterns, but uses:

  • Scheduled Transactions: Explicit recurring patterns defined by users
  • Past Month Replication: Implicit pattern detection by replicating recent behavior
  • Category Volatility: Statistical analysis of historical category spending

Future enhancements could include:

  • Automatic detection of recurring transactions
  • Machine learning-based pattern recognition
  • Seasonal adjustment factors

Noise Application

Noise is applied using uniform random distribution:

val noiseRange = noisePercentage / 100.0
val randomFactor = 1.0 + (random.nextDouble() * 2.0 - 1.0) * noiseRange

This creates a ±noisePercentage% variation around the base amount. For example, with 5% noise, a $100 expense could range from $95 to $105.

Alternative Approaches:

  • Normal distribution (more realistic but requires mean/std dev)
  • Category-specific noise (based on volatility)
  • Time-dependent noise (higher uncertainty further in future)

Edge Case Handling

No History

If a user has no transaction history:

  • Starting balance = initial account balance
  • Only scheduled transactions and periodic spends are included
  • Past month replication is skipped

Missing Data

  • Missing scheduled transactions: Skipped (no error)
  • Invalid dates: Validation at API layer prevents invalid ranges
  • Negative balances: Allowed (overdraft scenarios)

Long-Range Predictions

For predictions spanning years:

  • Pagination limits response size
  • Date points generated efficiently (not all stored in memory)
  • Database queries optimized with proper indexes

Multiple Accounts

When multiple accounts are included:

  • Starting balances summed across accounts
  • Transactions aggregated by date
  • Individual account breakdowns available in detailed response

Scenario lab

Saved scenarios, inline adjustments, baseline comparison, budget impact, and affordability are documented in balance-prediction-scenario-lab.md. They reuse the same engine pipeline; ScenarioAdjustmentMerger resolves adjustments before transaction generation.

API Endpoints

Calendar dates in request/response bodies use dd/MM/yyyy (not ISO yyyy-MM-dd).

Generate Balance Prediction

POST /finance/transactions/balance-prediction

Request Body (excerpt):

{
  "fromDate": "01/01/2024",
  "toDate": "31/12/2024",
  "accountIds": ["uuid1", "uuid2"],
  "includeScheduledTransactions": true,
  "includePastMonthBehavior": false,
  "periodicSpends": [
    {
      "amount": 50.0,
      "frequency": "WEEKLY",
      "description": "Groceries"
    }
  ],
  "noisePercentage": 5.0,
  "granularity": "DAILY",
  "safetyFloor": 300.0,
  "overdraftThreshold": 0.0,
  "goalId": "uuid",
  "scenarioIds": [],
  "inlineAdjustments": [],
  "compareToBaseline": false,
  "page": 0,
  "pageSize": 100
}

Affordability check

POST /finance/transactions/balance-prediction/affordability

Body: AffordabilityRequest (amount, date, optional category, optional scenarioIds, safetyFloor, …). See scenario lab doc for verdict precedence.

Balance scenarios CRUD

GET|POST /finance/balance-scenarios
PUT|DELETE /finance/balance-scenarios/{id}

Response (prediction excerpt):

{
  "points": [
    {
      "date": "01/01/2024",
      "balance": 1000.0,
      "transactions": [...],
      "scenarios": [
        {"type": "BASE", "balance": 1000.0},
        {"type": "OPTIMISTIC", "balance": 1050.0},
        {"type": "PESSIMISTIC", "balance": 950.0}
      ]
    }
  ],
  "startingBalance": 1000.0,
  "endingBalance": 1200.0,
  "minBalance": 800.0,
  "maxBalance": 1500.0,
  "safeToSpendToday": 700.0,
  "dangerEvents": [...],
  "goalAnalysis": {...},
  "categoryForecasts": [...],
  "budgetImpact": [...],
  "baselinePoints": null,
  "compareToBaseline": false,
  "totalPoints": 365,
  "currentPage": 0,
  "totalPages": 4,
  "pageSize": 100
}

Performance Considerations

Optimization Strategies

  1. Database Indexing: Indexes on user_id, date, account_id, status
  2. Query Optimization: Efficient joins and filters
  3. Pagination: Limits response size for long-range predictions
  4. Caching: Frequently accessed data cached on client
  5. Lazy Evaluation: Date points generated on-demand where possible

Scalability

  • Horizontal Scaling: Stateless server design allows multiple instances
  • Database Sharding: User-based sharding for large user bases
  • Caching Layer: Redis for frequently accessed predictions
  • Async Processing: Long predictions could be processed asynchronously

Future Enhancements

  • Machine Learning: ML-based pattern detection and prediction
  • Seasonal Adjustments: Account for seasonal spending patterns
  • Multi-Currency: Support for currency conversion in predictions
  • Collaborative Filtering: Learn from similar users’ patterns
  • Real-Time Updates: WebSocket updates for live prediction adjustments
Domain & modelsUI & UX