Forecasting with Boundaries: A Look at State-of-the-Art Time Series Models With Ceilings & Floors

These constraints, often referred to as carrying capacity, are crucial in various fields, from predicting product adoption and population growth to forecasting market saturation, attendance with strict limits or even revenue growth (you don't make less than zero revenue in a day). While traditional statistical models have long incorporated these features, the landscape of state-of-the-art forecasting now includes advanced machine learning and deep learning techniques adapted for this purpose.

Established Methods: Prophet and Generalized Additive Models

For time series forecasting with logistic growth, Facebook's Prophet model stands out as a widely used and effective tool. Prophet is designed to handle the complexities of business time series and has built-in functionalities to model saturating growth. This is achieved by specifying a cap for the carrying capacity and, if needed, a floor for the minimum value. The model then uses a logistic growth trend component, which naturally incorporates these saturation points into the forecast. This makes it a straightforward and powerful option for many common forecasting scenarios involving logistic growth.

Similarly, Generalized Additive Models (GAMs) offer a flexible framework for modeling non-linear relationships in time series data. GAMs can be configured to use a logistic growth function as the trend component, allowing them to capture the characteristic S-shaped curve of saturating processes. This provides a statistically robust and interpretable approach to forecasting with carrying capacity.

The Rise of Deep Learning and Hybrid Approaches

While Prophet and GAMs are strong contenders, the field is increasingly looking towards deep learning for more complex forecasting problems. However, unlike Prophet, popular deep learning architectures like Long Short-Term Memory (LSTM) networks, Recurrent Neural Networks (RNNs), and Transformers do not inherently account for logistic growth. Instead, several techniques can be employed to enforce these constraints on their output.

Techniques for Bounded Deep Learning Forecasts:

1. Data Normalization: A common practice is to normalize the time series data to a specific range, typically between 0 and 1, before training the model. This is often done using a Min-Max scaler. After the model produces its forecast on the scaled data, the output is then inverse-transformed back to the original scale. While this doesn't strictly guarantee that the forecast will adhere to the ceiling and floor, it helps in constraining the output.
2. Activation Functions: The activation function of the output layer in a neural network can be used to enforce hard limits. A sigmoid activation function, for instance, naturally outputs values between 0 and 1, which can then be scaled to the desired range of the forecast. For cases where the output needs to be strictly positive, a Rectified Linear Unit (ReLU) can be used.
3. Custom Loss Functions: A more direct way to enforce these boundaries is by designing a custom loss function. This function can heavily penalize the model for making predictions that fall outside the predefined ceiling and floor. This guides the model during the training process to respect the specified constraints.

The current state-of-the-art is also seeing a trend towards hybrid models. These models combine the strengths of different approaches. For example, a common hybrid approach is to first model the main trend and seasonality using a statistical model like ARIMA or Prophet, and then use a deep learning model, such as an LSTM, to learn and forecast the remaining residual component of the time series. This allows for the explicit modeling of logistic growth by the statistical component, while the deep learning component captures more complex, non-linear patterns in the data.

The Future of Forecasting with Constraints

While there isn't a single, universally acclaimed "state-of-the-art" deep learning model specifically designed for logistic growth time series, the techniques to adapt powerful architectures like Transformers for this task are evolving. The development of foundation models for time series and the use of Neural Architecture Search (NAS) could lead to the automatic discovery of optimal model architectures for specific tasks, including forecasting with carrying capacity. These advancements promise to further enhance the ability to generate accurate and reliable forecasts for time series data with natural limits to its growth.

At Trackura, we take the hybrid approach.

Of the options provided, the RNNModel (including LSTM and GRU), especially in its probabilistic version equivalent to DeepAR, is the best choice to use in combination with Prophet for achieving a state-of-the-art forecasting model that respects logistic growth.

Here's a detailed breakdown of why this approach is superior and how it works in practice.

The Hybrid Forecasting Strategy: Prophet + RNN

The most effective way to combine Prophet with a recurrent neural network (RNN) is to use a residual fitting approach. This strategy leverages the strengths of both models:

1. Prophet for the Baseline: Prophet is exceptionally good at modeling the primary, interpretable components of a time series. You use it to capture the main trend (including the logistic growth with its cap and floor), seasonality (yearly, weekly), and holiday effects.
2. RNN for the Residuals: After Prophet makes its forecast, you calculate the residuals (i.e., actual values - prophet's forecast). These residuals contain the complex, non-linear patterns and temporal dependencies that Prophet's model couldn't capture. An RNN, such as an LSTM or GRU, is perfectly suited to learn these remaining patterns from the residual series.
3. Final Forecast: The final, more accurate forecast is the sum of Prophet's forecast and the RNN's forecast of the residuals.

This hybrid model combines Prophet's ability to enforce structural constraints (like carrying capacity) with the RNN's power to model complex, dynamic behavior.

Why RNNModel (Probabilistic/DeepAR-style) is the Best Choice

While both RNNModel and BlockRNNModel can be used to model residuals, the probabilistic RNNModel offers a significant advantage that pushes it into the "state-of-the-art" category: quantifying uncertainty.

1. Probabilistic Forecasting:

• A standard RNN provides a single point forecast. However, the RNNModel described as a "probabilistic version equivalent to DeepAR" does not just predict a single value; it predicts the parameters of a probability distribution (e.g., a mean and a standard deviation) for each future time step.
• Key Advantage: This provides a rich, quantitative measure of uncertainty. Instead of just a single prediction line, you get a full predictive distribution, allowing you to generate confidence intervals. For business decisions, knowing that future demand is likely to be "1000 units, with a 90% chance of being between 800 and 1200" is far more valuable than a simple "1000 units" forecast.

2. Handling Multiple Time Series:

The DeepAR algorithm was specifically designed to be trained on many related time series simultaneously. If you are forecasting sales for multiple products or locations, a DeepAR-style model can learn global patterns from all series, often leading to more robust and accurate forecasts, especially for series with limited history.

3. Autoregressive Nature:

The standard RNNModel is fully recurrent, meaning the prediction for the current time step is used as an input for predicting the next time step. This autoregressive behavior is very effective for modeling the evolving, often noisy, nature of residual data.

BlockRNNModel: A Strong but Second-Best Alternative

The BlockRNNModel is also a powerful choice and can be very effective.

• How it Works: It uses an RNN to encode a block of past data and then uses a decoder to output a whole block of future predictions at once.
• Advantages: This block-based approach can be computationally faster for generating long-range forecasts compared to the step-by-step method of a standard RNNModel.
• Disadvantage (in this context): Unless specifically designed to be probabilistic, it will likely produce point forecasts, losing the crucial uncertainty information that the DeepAR-style model provides.

Comparison Table

Conclusion

To achieve a true state-of-the-art model that combines the logistic growth capabilities of Prophet with the pattern-recognition power of deep learning, the RNNModel in its probabilistic (DeepAR-equivalent) version is the superior choice.

By using this model to learn the patterns in Prophet's residuals, you not only improve the accuracy of the point forecast but also gain invaluable insights into the forecast's uncertainty, which is a hallmark of a modern, sophisticated forecasting system.