%\VignetteIndexEntry{Functional Time Series Forecasting: the ftsa Package for R}
%\VignetteDepends{ftsa, rainbow, forecast}
%\VignetteKeywords{Functional time series, Functional time series modeling, Functional time series forecasting}
%\VignettePackage{ftsa}

\documentclass[nojss]{jss}

\usepackage{amsmath,amsfonts,bbm,enumitem,microtype,alltt,verbatim,subfig,bm,animate,tikz}
\usepackage[utf8]{inputenc} 
\usepackage[figuresright]{rotating}

\newcommand{\field}[1]{\mathbb{#1}}
\newcommand{\R}{$\field{R}$}
\usetikzlibrary[decorations.shapes]
\usetikzlibrary[petri]
  
  %% Change the default page sizes.
  
  \setlength{\topmargin}{-0.25in}
  \setlength{\textheight}{8.5in}
  \setlength{\oddsidemargin}{.0in}
  \setlength{\evensidemargin}{.0in}
  \setlength{\textwidth}{6.5in}
  \setlength{\footskip}{.5in}
  
  \newenvironment{smallexample}{\begin{alltt}\small}{\end{alltt}}
  \newenvironment{smallverbatim}{\small\verbatim}{\endverbatim}
  \graphicspath{{plots/}}
  
  %% need no \usepackage{Sweave.sty}
  
  \author{Han Lin Shang\\Australian National University}
  
  \title{Modeling and forecasting for Functional Time Series}
  
  \Plainauthor{Han Lin Shang}
  
  \Plaintitle{Modeling and forecasting for Functional Time Series}
  
  \Abstract{
  Recent advances in computer recording and storing technology have tremendously increased the presence of functional data, whose graphical representation can be infinite-dimensional curve, image, or shape. When the same functional object is observed over a period of time, such data are known as functional time series. This article makes first attempt to describe several techniques (centered around functional principal component analysis) for modeling and forecasting functional time series from computational and practical aspects, using a readily-available \textsf{R} addon package. These methods are demonstrated using age-specific Australian fertility rate data from 1921 to 2006, and monthly sea surface temperature data from January 1950 to December 2011.
}
  
  \Keywords{dynamic updating; functional time series visualization; functional principal component analysis; functional partial least squares regression; functional time series forecasting; stationary test for functional time series}
  
  \Plainkeywords{functional data analysis, functional time series}
  
  \Address{Han Lin Shang\\
           Research School of Finance, Actuarial Studies and Statistics \\
           Australian National University\\
           Canberra, ACT 2601, Australia\\
           E-mail: \email{hanlin.shang@anu.edu.au}\\
           URL: \url{https://researchers.anu.edu.au/researchers/shang-h} \\
  }
  
  \begin{document}
\SweaveOpts{concordance=FALSE}
  
  %% Note that you should use the \pkg{}, \proglang{} and \code{} commands.
  
  %% Note - fragile using \label{} in \section{} - must be outside
  
  %% For graphics

  %% <<fig=TRUE,eval=TRUE, height=, width=>>=
    
  \section*{Introduction}
  
The aim of this article is to describe the \textsf{R} functions that are readily-available in the \pkg{ftsa} package \citep{HS12}, for modeling and forecasting functional time series. This article was motivated by recent advances in computer recording and storing technology that have enabled researchers to collect and store (ultra) high-dimensional data. When the high-dimensional data are repeatedly measured on the same object over a period of time, a time series of continuous functions is observed within a common bounded interval \citep{SH08}.

Analyzing functional time series has received increasing attention in the functional data analysis literature \citep[see for example,][]{AGH+09, HK10, HHK10, HK12, KMZ15}. \cite{HS10} presented a rainbow plot for visualizing functional time series, where the distant past data are shown in red and most recent data are shown in purple. \cite{AOV99} proposed functional principal component regression (FPCR) to model and forecast functional time series.

Before reviewing the FPCR, we first define the problem more precisely. Let $y_t(x)$ denote a function, such as age-specific fertility rates for the continuous age variable $x$ in year $t$, or monthly sea surface temperatures for the continuous time variable $x$ in year $t$. In the latter example, functional time series techniques allow us to capture the underlying dynamic of the multiple seasonality in the data \citep[see][for example]{SH11, Shang12a}. We assume that there is an underlying smooth function $f_t(x)$ that observes with error at discretized grid points of $x$. In practice, we observe $\{x_i, y_t(x_i)\}$ for $t=1,2,\dots,n$ and $i=1,2,\dots,p$, from which we extract a smooth function $f_t(x)$, given by
\begin{equation}
  y_t(x_i) = f_t(x_i) + \sigma_t(x_i)\varepsilon_{t,i},\label{eq:51}
\end{equation}
where $\varepsilon_{t,i}$ is an independent and identically distributed (iid) standard normal random variable, $\sigma_t(x_i)$ allows the amount of noise to vary with $x_i$, and $\{x_1,x_2,\dots,x_p\}$ is a set of discrete data points. For each $t$, $\{x_1,x_2,\dots,x_p\}$ can be un-equally spaced. Given a set of functional data $\bm{f}(x)=[f_1(x),f_2(x),\dots,f_n(x)]^{\top}$, we are interested in finding underlying patterns using the FPCR, from which we obtain forecasts of $y_{n+h}(x)$, where $h$ denotes the forecast horizon.

This article proceeds as follows. Techniques for modeling and forecasting functional time series are reviewed and implemented using the \pkg{ftsa} package. Conclusions are given in the end.


\section*{Functional time series modeling and forecasting techniques}\label{sec:3}

\subsection*{Functional principal component regression}

The theoretical, methodological and practical aspects of functional principal component analysis (FPCA) have been extensively studied in the functional data analysis literature, since it allows finite dimensional analysis of a problem that is intrinsically infinite-dimensional \citep{HH06}. Numerous examples of using FPCA as an estimation tool in regression problem can be found in different fields of applications, such as breast cancer mortality rate modeling and forecasting \citep{EHG07}, call volume forecasting \citep{SH08}, climate forecasting \citep{SH11}, demographical modeling and forecasting \citep{HS09}, and electricity demand forecasting \citep{APD+08}.

At a population level, a stochastic process denoted by $f$ can be decomposed into the mean function and the sum of the multiplications of orthogonal functional principal components and uncorrelated principal component scores. It can be expressed as
\begin{equation*}
f = \mu + \sum^{\infty}_{k=1}\beta_{k}\phi_k,
\end{equation*}
where $\mu$ is the unobservable population mean function, $\beta_{k}$ is the $k^{\text{th}}$ principal component scores, and $\phi_k$ is the $k^{\text{th}}$ population functional principal component.

In practice, we can only observe $n$ realizations of $f$ evaluated on a compact interval $x\in[0,\tau]$, denoted by $f_t(x)$ for $t=1,2,\dots,n$. At a sample level, the functional principal component decomposition can be written as
\begin{equation}
f_t(x) = \bar{f}(x) + \sum^K_{k=1}\widehat{\beta}_{t,k}\widehat{\phi}_k(x) + \widehat{\varepsilon}_t(x),\label{eq:61}
\end{equation}
where $\bar{f}(x)=\frac{1}{n}\sum^n_{t=1}f_t(x)$ is the estimated mean function, $\widehat{\phi}_k(x)$ is the $k^{\text{th}}$ estimated orthonormal eigenfunction of the empirical covariance operator
\begin{equation*}
  \widehat{\Gamma}(x)=\frac{1}{n}\sum^n_{t=1}[f_t(x)-\bar{f}(x)][f_t(x)-\bar{f}(x)],
\end{equation*}
the coefficient $\widehat{\beta}_{t,k}$ is the $k^{\text{th}}$ principal component score for year $t$, it is given by the projection of $f_t(x)-\bar{f}(x)$ in the direction of $k^{\text{th}}$ eigenfunction $\widehat{\phi}_k(x)$, that is, $\widehat{\beta}_{t,k}$=$<f_t(x)-\bar{f}(x), \widehat{\phi}_k(x)>$=$\int_x [f_t(x)-\bar{f}(x)]\widehat{\phi}_k(x)dx$, $\widehat{\varepsilon}_t(x)$ is the residual, and $K$ is the optimal number of components, which can be chosen by cross validation. \cite{HB08} studied the impact on forecast accuracy with a smaller or larger than the optimal value of $K$.

The functional principal component decomposition is first demonstrated using the age-specific Australian fertility rate data between ages 15 and 49 observed from 1921 to 2006. This data set was obtained from the Australian Bureau of Statistics (Cat No, 3105.0.65.001, Table 38). A functional graphical display is given in \cite{Shang11}.

Figure~\ref{fig:3} presents the first two functional principal components and their associated principal component scores. The bottom panel of Figure~\ref{fig:3} also plots the forecasted principal component scores, and their 80\% prediction intervals (in yellow color), using an exponential smoothing state-space model \citep{HKO+08}. As pointed out by a referee, the forecasts of principal component scores appear to quickly level off, and the prediction intervals widen very quickly. This reflects the difficulty of our model in forecasting medium or long term horizon, as a result of the increase in variability.

\setkeys{Gin}{width=0.6\textwidth}

\begin{figure}[!ht]
\centering
<<fig=TRUE,echo=FALSE,keep.source=TRUE>>=
library(ftsa)
plot(forecast(ftsm(Australiasmoothfertility,
    order=2), h=20), "components")
@
\caption{The first two functional principal components and their associated principal component scores for the Australian fertility rate data from 1921 to 2006.}\label{fig:3}
\end{figure}

Figure~\ref{fig:3} was produced by the following code.
\begin{smallexample}
\begin{smallverbatim}
# load the package used throughout this article
library("ftsa")
# Fit and plot functional principal components, order specifies the number of principal components
# h specifies the forecast horizon
plot(forecast(ftsm(Australiasmoothfertility, order=2), h = 20), "components")
\end{smallverbatim}
\end{smallexample}

By conditioning on the observed data $\bm{f}(x)=[f_1(x),f_2(x),\dots,f_n(x)]^{\top}$ and the fixed functional principal components $\mathcal{\bm{B}}=[\widehat{\phi}_1(x),\widehat{\phi}_2(x),\dots,\widehat{\phi}_K(x)]^{\top}$, the $h$-step-ahead forecasts of $y_{n+h}(x)$ can be obtained as
\begin{small}
\begin{align*}
\widehat{y}_{n+h|n}(x)=\text{E}[y_{n+h}(x)|\bm{f}(x),\mathcal{\bm{B}}]=\bar{f}(x)+\sum^K_{k=1}\widehat{\beta}_{n+h|n,k}\widehat{\phi}_k(x),
\end{align*}
\end{small}
where $\widehat{\beta}_{n+h|n,k}$ denotes the $h$-step-ahead forecasts of $\beta_{n+h,k}$ using a univariate time series.

Figure~\ref{fig:4} shows the forecasts of Australian fertility rate data from 2007 to 2026 highlighted in rainbow color, while the data used for estimation are grayed out. It exhibits a continuing shift to older ages of peak fertility rates, caused by a recent tendency to postpone child-bearing while pursuing careers.


\begin{figure}[!ht]
  \begin{center}
<<fig=TRUE,echo=FALSE,keep.source=TRUE,include=FALSE>>=
# Plot the historical data in gray
plot(Australiasmoothfertility, col = gray(0.8), xlab = "Age",
    ylab = "Count of live birth (per 1,000 females)",
    main = "Forecasted fertility rates (2007-2026)")
# Plot the forecasts in rainbow color for Figure 4(a)
plot(forecast(ftsm(Australiasmoothfertility, order=2), h = 20), add = TRUE)
legend("topright", c("2007", "2026"), col = c("red", "blue"), lty = 1)
@
\includegraphics[width=0.6\textwidth]{ftsa-002}
\end{center}
\caption{Multiple-step-ahead forecasts. Based on the historical data from 1921 to 2006, we obtain 20-step-ahead forecasts for 2007 to 2026.}\label{fig:4}
\end{figure}

Figure~\ref{fig:4} was produced by the following code.
\begin{smallexample}
\begin{smallverbatim}
# Plot the historical data in gray
plot(Australiasmoothfertility, col = gray(0.8), xlab = "Age", ylab = 
"Count of live birth (per 1,000 females)", main = "Forecasted fertility rates (2007-2026)")
# Plot the forecasts in rainbow color for Figure 4(a)
plot(forecast(ftsm(Australiasmoothfertility, order=2), h=20), add=TRUE)
legend("topright", c("2007", "2026"), col = c("red", "blue"), lty = 1)
\end{smallverbatim}
\end{smallexample}

To construct a prediction interval, we calculate the forecast variance that follows from~\eqref{eq:51} and~\eqref{eq:61}. Because of orthogonality, the forecast variance can be approximated by the sum of component variances
\begin{align*}
  \xi_{n+h}(x)&=\text{Var}[y_{n+h}(x)|\bm{f}(x),\mathcal{\bm{B}}]\\
  &=\hat{\sigma}_{\mu}^2(x)+\sum^K_{k=1}u_{n+h,k}\hat{\phi}_k^2(x) + v(x) + \sigma^2_{n+h}(x),
\end{align*}
where $u_{n+h,k}=\text{Var}(\beta_{n+h,k}|\beta_{1,k},\beta_{2,k},\dots,\beta_{n,k})$ can be obtained from the time series model, and the model error variance $v(x)$ is estimated by averaging $\{\hat{\varepsilon}_1^2(x),\hat{\varepsilon}_2^2(x),\dots,\hat{\varepsilon}_n^2(x)\}$ for each $x$, and $\hat{\sigma}_{\mu}^2(x)$ and $\sigma^2_{n+h}(x)$ can be obtained from the smoothing method used.

Based on the normality assumption, the $100(1-\alpha)\%$ prediction interval for $y_{n+h}(x)$ is constructed as $\hat{y}_{n+h|n}(x)\pm z_{\alpha}\sqrt{\xi_{n+h}(x)}$, where $z_{\alpha}$ is the $(1-\alpha/2)$ standard normal quantile.

\begin{figure}[!htbp]
\centering
<<fig=TRUE,echo=FALSE,keep.source=TRUE>>=
# Plot the point forecast
aus = forecast(ftsm(Australiasmoothfertility, order=2), h=1)
plot(aus, ylim=c(0,140))
# Plot the lower and upper bounds
lines(aus$lower, col=2)
lines(aus$upper, col=2)
# Add a legend to the plot
legend("topright", c("Point forecasts", "Interval forecasts"),col=c(1,2), lty=1, cex=0.9)
@
\caption{Forecasts of fertility rates in 2007, along with the 80\% prediction interval.}\label{fig:5}
\end{figure}

Figure~\ref{fig:5} displays the forecasts of fertility rates in 2007, along with the 80\% prediction interval. It was created by the following code.

\begin{smallexample}
  \begin{smallverbatim}
# Plot the point forecast
aus = forecast(ftsm(Australiasmoothfertility, order=2), h=1)
plot(aus, ylim=c(0,140))
# Plot the lower and upper bounds
lines(aus$lower, col=2)
lines(aus$upper, col=2)
# Add a legend to the plot
legend("topright", c("Point forecasts", "Interval forecasts"),col=c(1,2), lty=1, cex=0.9)
  \end{smallverbatim}
\end{smallexample}

\subsection*{Updating point and interval forecasts}

A special case of functional time series is when the continuous variable is also a time variable, such as the monthly sea surface temperature from 1950 to 2011, obtained from National Oceanic and Atmospheric Administration (\url{http://www.cpc.noaa.gov/ data/indices/sstoi.indices}). A similar type of functional graphical display is given in \citet[][Figure 2]{Shang11}. Such data originate from a univariate seasonal time series. Let $\{Z_w, w\in[1,N]\}$ be a seasonal time series which has been observed at $N$ equispaced times. We divide the observed time series into $n$ trajectories, and then consider each trajectory of length $p$ as a curve rather than $p$ distinct data points. The functional time series is given by
\begin{equation*}
  y_t(x)=\{Z_w, w\in(p(t-1), pt]\},\quad t=1,2,\dots,n.
\end{equation*}
The problem of interest is to forecast $y_{n+h}(x)$, where $h$ denotes forecast horizon. In the sea surface temperature data, we consider $\{Z_w\}$ be monthly sea surface temperatures from 1950 to 2011, so that $p=12$ and $N=62\times 12=744$, and we are interested in forecasting sea surface temperatures in 2012 and beyond.

When $N=np$, all trajectories are complete, and forecasts can be obtained by the FPCR. However, when $N\neq np$, we revisited the block moving (BM) and penalized least squares (PLS) to update point and interval forecasts, when the most recent curve is partially observed.

When functional time series are segments of a univariate time series, the most recent trajectory is observed sequentially \citep{HS10}. When we observe the first $m_0$ time period of $y_{n+1}(x_l)$, denoted by $\bm{y}_{n+1}(x_e)=[y_{n+1}(x_1),y_{n+1}(x_2),\dots,y_{n+1}(x_{m_0})]^{\top}$, we are interested in forecasting the data in the remaining time period, denoted by $y_{n+1}(x_l)$ for $m_0<l\leq p$. By using the FPCR, the partially observed data in the most recent curve are not incorporated into the forecasts of $y_{n+1}(x_l)$. Indeed, the point forecasts obtained from the FPCR can be expressed as
\begin{align*}
  \hat{y}_{n+1|n}(x_l)&=\text{E}[y_{n+1}(x_l)|\bm{f}(x_l),\mathcal{\bm{B}}_l]\\
  &=\bar{f}(x_l)+\sum^K_{k=1}\hat{\beta}_{n+1|n,k}\hat{\phi}_k(x_l),
\end{align*}
for $m_0<l\leq p$, where $\bm{f}(x_l)$ denotes the historical data corresponding to the remaining time periods; $\bar{f}(x_l)$ is the mean function corresponding to the remaining time periods; and $\mathcal{\bm{B}}_l=\{\hat{\phi}_1(x_l),\hat{\phi}_2(x_l),\dots,\hat{\phi}_K(x_l)\}$ is a set of the estimated functional principal components corresponding to the remaining time periods.

In order to improve point forecast accuracy, it is desirable to dynamically update the point and interval forecasts for the rest of year $n+1$ by incorporating the partially observed data. In what follows, I shall revisit two methods for updating point and interval forecasts.

\subsubsection*{Block moving (BM)}

The BM method re-defines the start and end points of trajectories. Because time is a continuous variable, we can change the function support from $[1,p]$ to $[m_0+1, p]\bigcup [1,m_0]$. The re-defined functional time series forms a complete block, at the cost of losing some observations in the first year. With the complete data block, the FPCR can then be applied to update the point and interval forecasts.

\tikzset{decorate with/.style={fill=cyan!20,draw=cyan}}
\begin{figure}[!ht]
\begin{center}
\scalebox{.55}{
\begin{tikzpicture}
\draw (0,0) node[anchor=north east]{\LARGE{$x_p$}} rectangle (12,8);
\draw (1.1,4.1) rectangle (13.1,12);
\draw (0, 8) node[anchor=north east]{\LARGE{$x_1$}} -- (0, 0);
\draw (0,4.1) node[anchor=north east]{  \begin{rotate}{90} \hspace {-.5in} \LARGE{Dimensionality} \end{rotate} \hspace{.05in} \LARGE{$x_{m_0}$}} -- (1.1, 4.1);
\draw[dashed] (1.1,0) -- (1.1,4.1);
\draw[dashed] (2.2,12) -- (2.2,8);
\draw[dashed] (14.2, 12) -- (14.2, 8);
\draw (11.82, 8) node[anchor=south west]{\Large{$n+1$}} -- (13, 8);
\draw[dashed] (13, 8) -- (14.2, 8);
\draw[dashed] (13, 12) -- (14.2, 12);
\draw[dashed] (13.1, 4.1) -- (13.1, 0);
\draw[dashed] (12, 0) -- (13.1, 0);
\draw (8.5,0) node[anchor=north east]{\LARGE{Number of curves}} -- (12,0);
\draw(0.4,8) node[anchor=south west]{\Large{1}} -- (1.1,8);
\draw [decorate with=rectangle] (0.0305,4.1365) -- (0.0305,7.98) --  (1.074,7.98) -- (1.074,4.1365);
\end{tikzpicture}}
\end{center}
\caption{\small{Dynamic update via the block moving approach. The colored region shows the data loss in the first year. The forecasts for the remaining months in year $n+1$ can be updated by the forecasts using the TS method applied to the upper block.}}\label{fig:11}
\end{figure}

The redefined data are shown diagrammatically in Figure~\ref{fig:11}, where the bottom box has moved to become the top box. The cyan colored region shows the data loss in the first year. The partially observed last trajectory under the old ``year" completes the last trajectory under the new year.

As an illustration, suppose we observe the monthly sea surface temperature from January to May 2011, we aim to update the point and interval forecasts from June to December. Figure~\ref{fig:9} displays the point and interval forecasts for the remaining months of 2011, by using the BM method.

\begin{figure}[!ht]
  \centering
<<fig=TRUE,echo=FALSE,keep.source=TRUE,include=TRUE>>=
# Name history to represent historical data,
history = ElNino_ERSST_region_1and2
# Name obs to represent partially observed data,
obs = ElNino_ERSST_region_1and2$y[1:5,69]
# Name fore to represent the forecasting period
fore = ElNino_ERSST_region_1and2$y[6:12,69]
int = dynupdate(data=history, newdata=obs, holdoutdata=fore, method="block", interval=TRUE)
bmupdate = dynupdate(data=history, newdata=obs, holdoutdata=fore, method="block", value=TRUE)
plot(6:12, fore, type="l", ylim=c(19,26),  xlab="Month", ylab="Sea surface temperature")
lines(6:12, bmupdate, col=4)
lines(6:12, int$low$y, col=2)
lines(6:12, int$up$y, col=2)
legend("topright", c("True observations", "Point forecasts", "Interval forecasts"), 
       col=c(1,4,2), lty=1, cex=0.8)
@
\caption{Prediction intervals of the sea surface temperatures between June and December 2011. By incorporating the sea surface temperatures between January and May, the prediction intervals can be updated using the BM method}.\label{fig:9}
\end{figure}

Figure~\ref{fig:9} was created by the following code
\begin{smallexample}
  \begin{smallverbatim}
# Name history to represent historical data,
history = ElNino_ERSST_region_1and2
# Name obs to represent partially observed data,
obs = ElNino_ERSST_region_1and2$y[1:5,69]
# Name fore to represent the forecasting period
fore = ElNino_ERSST_region_1and2$y[6:12,69]
int = dynupdate(data=history, newdata=obs, holdoutdata=fore, method="block", interval=TRUE)
bmupdate = dynupdate(data=history, newdata=obs, holdoutdata=fore, method="block", value=TRUE)
plot(6:12, fore, type="l", ylim=c(19,26), xlab="Month", ylab="Sea surface temperature")
lines(6:12, bmupdate, col=4)
lines(6:12, int$low$y, col=2)
lines(6:12, int$up$y, col=2)
legend("topright", c("True observations", "Point forecasts", "Interval forecasts"), 
      col=c(1,4,2), lty=1, cex=0.8)
  \end{smallverbatim}
\end{smallexample}

\subsubsection{Penalized least squares (PLS)}

We can also update the remaining part of the trajectory by using regression-based approaches. Let $\bm{F}_e$ be $m_0\times K$ matrix, whose $(j,k)^{\text{th}}$ entry is $\hat{\phi}_k(x_j)$ for $1\leq j\leq m_0$. Let $\bm{\beta}_{n+1}=[\beta_{n+1,1},\beta_{n+1,2},\dots,\beta_{n+1,K}]^{\top}$, $\bm{\bar{f}}(x_e)=[\bar{f}(x_1),\bar{f}(x_2),\dots,\bar{f}(x_{m_0})]^{\top}$, and $\bm{\epsilon}_{n+1}(x_e)=[\epsilon_{n+1}(x_1),\epsilon_{n+1}(x_2),\dots,\epsilon_{n+1}(x_{m_0})]^{\top}$. As the mean-adjusted $\hat{\bm{y}}_{n+1}^*(x_e)=\bm{y}_{n+1}(x_e)-\bm{\bar{f}}(x_e)$ become available, a regression can be expressed as
\begin{equation*}
  \hat{\bm{y}}_{n+1}^*(x_e) = \bm{F}_e\bm{\beta}_{n+1}+\bm{\epsilon}_{n+1}(x_e).
\end{equation*}
The $\bm{\beta}_{n+1}$ can be estimated by ordinary least squares, assuming $(\bm{F}_e^{\top}\bm{F}_e)$ is invertible,
\begin{equation*}
  \hat{\bm{\beta}}_{n+1}^{\text{OLS}}=(\bm{F}_e^{\top}\bm{F}_e)^{-1}\bm{F}_e^{\top}\hat{\bm{y}}_{n+1}^*(x_e).
\end{equation*}
However, if $(\bm{F}_e^{\top}\bm{F}_e)$ is not invertible, then a regularized approach can be implemented, such as the ridge regression (RR) and penalized least squares (PLS). The regression coefficients of the RR and PLS are
\begin{align}
  \hat{\bm{\beta}}_{n+1}^{\text{RR}}&=(\bm{F}_e^{\top}\bm{F}_e+\lambda\bm{I}_K)^{-1}\bm{F}_e^{\top}\hat{\bm{y}}_{n+1}(x_e),\notag\\
  \hat{\bm{\beta}}_{n+1}^{\text{PLS}}&=(\bm{F}_e^{\top}\bm{F}_e+\lambda\bm{I}_K)^{-1}(\bm{F}_e^{\top}\hat{\bm{y}}_{n+1}(x_e)+\lambda\hat{\beta}_{n+1|n}),\label{eq:21}
\end{align}
where $\hat{\bm{\beta}}_{n+1}^{\text{RR}} \rightarrow \bm{0}$ as $\lambda\rightarrow \infty$, and $\hat{\bm{\beta}}_{n+1}^{\text{RR}} \rightarrow \hat{\bm{\beta}}_{n+1}^{\text{OLS}}$ as $\lambda\rightarrow 0$. In contrast, the $\hat{\bm{\beta}}_{n+1}^{\text{PLS}}\rightarrow \hat{\bm{\beta}}_{n+1|n}$ as $\lambda\rightarrow \infty$, and $\hat{\bm{\beta}}_{n+1}^{\text{PLS}}\rightarrow \hat{\bm{\beta}}_{n+1}^{\text{OLS}}$ as $\lambda\rightarrow 0$.

The point forecasts of $y_{n+1}(x_l)$ obtained by the RR and PLS are given by
\begin{align*}
  \hat{y}_{n+1}^{\text{RR}}(x_l)&=\text{E}[y_{n+1}(x_l)|\bm{f}(x_l),\mathcal{\bm{B}}_l]=\bar{f}(x_l)+\sum^K_{k=1}\hat{\bm{\beta}}_{n+1,k}^{\text{RR}}\hat{\phi}_k(x_l),\\
  \hat{y}_{n+1}^{\text{PLS}}(x_l)&=\text{E}[y_{n+1}(x_l)|\bm{f}(x_l),\mathcal{\bm{B}}_l]=\bar{f}(x_l)+\sum^K_{k=1}\hat{\bm{\beta}}_{n+1,k}^{\text{PLS}}\hat{\phi}_k(x_l),
\end{align*}

Among these regression-based approaches, the PLS method can also update the interval forecasts. Let the one-step-ahead forecast errors of the principal component scores be given by
\begin{equation*}
  \hat{\xi}_{j,k}=\hat{\beta}_{n-j+1,k}-\hat{\beta}_{n-j+1|n-j,k},\quad \text{for}\quad j=1,2,\dots,n-K.
\end{equation*}
$\{\hat{\xi}_{1,k},\hat{\xi}_{2,k},\dots,\hat{\xi}_{n-K,k}\}$ can then be sampled with replacement to give a bootstrap sample of $\beta_{n+1|n,k}$:
\begin{equation*}
  \hat{\beta}^b_{n+1|n,k}=\hat{\beta}_{n+1|n,k}+\hat{\xi}_{*,k}^b,\quad \text{for}\quad b=1,2,\dots,B,
\end{equation*}
where $\hat{\xi}_{*,k}^b$ denotes the bootstrap samples, and $B$ is the number of bootstrap replications. Based on~\eqref{eq:21}, the bootstrapped $\hat{\bm{\beta}}_{n+1|n}^b$ leads to the bootstrapped $\hat{\bm{\beta}}_{n+1}^{b,\text{PLS}}$, we obtain $B$ replications of
\begin{equation*}
  \hat{y}^{b,\text{PLS}}_{n+1}(x_l)=\bar{f}(x_l)+\sum^K_{k=1}\hat{\beta}_{n+1,k}^{b,\text{PLS}}\hat{\phi}_k(x_l)+\hat{\epsilon}_{n+1}^b(x_l),
\end{equation*}
where $\hat{\epsilon}_{n+1}^b(x_l)$ is obtained by sampling with replacement from $\{\hat{\epsilon}_1(x_l),\hat{\epsilon}_2(x_l),\dots,\hat{\epsilon}_n(x_l)\}$. Hence, the $100(1-\alpha)\%$ prediction intervals for the updated forecasts are defined as $\alpha/2$ and $(1-\alpha/2)$ 
quantiles of $\hat{y}_{n+1}^{b,\text{PLS}}(x_l)$.

For this sea surface temperature data set, the point and interval forecast accuracy obtained by the FPCR, BM and PLS methods have already been studied by \cite{SH11}.

\section*{Conclusion}

This article described several techniques in the \pkg{ftsa} package, for modeling and forecasting functional time series. These methods centered on the FPCR, which is a common dimension reduction technique in the functional data analysis literature. FPCR reduces intrinsically infinite number of variables to several orthogonal regressors, which captures the main mode of variation in data. As illustrated by the Australian fertility rates, FPCR is able to model and forecast annual Australian fertility rates. When the continuous variable in a functional time series is also a time variable, a new observation arrives sequentially. As shown in the monthly sea surface temperature, the BM and PLS methods can update the point and interval forecasts based on the FPCR. 

To sum up, the methods reviewed in this article focus on extracting patterns from a set of functional time series, and should be considered when the interest lies in modeling and forecasting the future realizations of a stochastic process.

\bibliography{master}
    
  \end{document}