03. Time Series Analysis (Jhon)

Haiyue8/9/23About 8 min

Methods

How to remove seasonality

There are two forms of ridding the original of seasonality, by either dividing by appropriate seasonal indices or by subtracting them.

Method

Form the mean for each season
$\bar{x_{i}} = \frac{1}{4} \sum_{j = 1}^{4} x_{i} + 4 j$
$\hat{x_{i}} = x_{i} - \bar{x_{i}}$ for additive.
$\hat{x_{i}} = \frac{x_{i}}{\bar{x_{i}}}$ for multiplicative.

Fourier Series Model for Seasonality

Using Fourier Series Model for modeling Seasonality.

Any periodic function satisfies the relation: $f (t) = f (t + T)$ where $T$ is the period of the function.
The simplest and most common periodic functions are the trigonometric functions. The functions $s i n (n t)$ and $c o s (n t), n \in 0, 1, 2, 3. . .$ being harmonic are by definition periodic with periods $\frac{2 π}{n}$ . Because trigonometric functions are relatively easy to work with and because they possess the important property of orthogonality they are useful to represent the seasonality of data of this type.

orthonormal set.

$\frac{1}{\sqrt{2 π}}, \frac{s i n (t)}{\sqrt{π}}, \frac{c o s (t)}{\sqrt{π}}, \frac{s i n (2 t)}{\sqrt{π}}, \frac{c o s (2 t)}{\sqrt{π}}, \frac{s i n (3 t)}{\sqrt{π}}, \frac{c o s (3 t)}{\sqrt{π}} . . .$

Because this set of functions is complete in the interval $0 \leq t \leq 2 π$ every function $f (t)$ which is continuous in that interval can be represented by the Fourier Series

$f (t) = \frac{1}{2} a_{0} + \sum_{r = 1}^{\infin} (a_{r} c o s (r t) + b_{r} s i n (r t))$

where the constants $a_{r}$ and $b_{r}$ are known as the Fourier coefficients.

Now, if we multiply both sides by $c o s (s t)$ and integrate we get

\begin{aligned} \int_{0}^{2 π} f (t) c o s (s t \cdot d t) & = \frac{1}{2} a_{0} \int_{0}^{2 π} c o s (s t) + \sum_{r = 1}^{\infin} a_{r} \int_{0}^{2 π} c o s (r t) c o s (s t \cdot d t) \\ = \sum_{r = 1}^{\infin} b_{r} \int_{0}^{2 π} s i n (r t) c o s (s t \cdot d t) \end{aligned}

For $s = 0$ , this yields $a_{o} = \frac{1}{π} \int_{0}^{2 π} f (t) d t$ so $a_{0}$ may be referred to as the average value of $f (t)$ . if $s \neq 0$ then it reduces to $a_{r} = \frac{1}{π} \int_{0}^{2 π} f (t) c o s (r d \cdot d t)$ and $b_{r} = \frac{1}{π} \int_{0}^{2 π} f (t) s i n (r t \cdot d t)$ . When $f (t)$ is an even series, that is, $f (t)$ is reflected at the origin, the $b$ coefficients vanish and the series is known as a Fourier cosine series. Similarly when the series is an odd series, $f (t) = - f (- t)$ the a coefficients vanish and the series is known as a Fourier sine series.

Discrete Fourier Transform (DFT)

What we need for our purposes is the discrete version of the series representation.
We also need to adjust the period, where in the case of the deaths data, the fundamental period is 12 months.
We will also consider how to find how many terms we use in the expansion.

The discrete Fourier Transform of a sequence of $N$ real or complex numbers, $x_{0}, x_{1}, . . ., x_{_{N - 1}}$ is the sequence of complex numbers $c_{0}, c_{1}, . . ., c_{_{N - 1}}$ defined by

$c_{j} = \frac{1}{N} \sum_{n = 0}^{N - 1} x_{n} e^{- \frac{2 π j n}{N}}, j = 0, 1, . . ., N - 1$

Inverse Discrete Fourier Transform (IDFT)

The original $x_{j}$ can be recovered using the Inverse DFT or IDFT, defined by

$x_{j} = \frac{1}{N} \sum_{n = 0}^{N - 1} c_{n} e^{\frac{2 π j n}{N}}, j = 0, 1, . . ., N - 1$

Assume $x_{0}, x_{1}, . . ., x_{_{N - 1}}$ are real numbers and thus $c_{_{N - j}} = c_{j}^{*}$
$x_{j} = a_{0} + \sum_{n = 1}^{\frac{N}{2} - 1} (a_{n} c o s (\frac{2 π j n}{N}) + b_{n} s i n (\frac{2 π j n}{N})) + a_{_{N / 2}} (- 1)^{j}$ for $N$ even

$x_{j} = a_{0} + \sum_{n = 1}^{\frac{N - 1}{2}} (a_{n} c o s (\frac{2 π j n}{N}) + b_{n} s i n (\frac{2 π j n}{N}))$ for $N$ odd

Power Spectrum

Note that is we used the formulae above we would capture all the values.
But, the Principle of Parsimony leads us to construct the model that fits the data best with the least number of coefficients.
Note that this will also the one that is most identifiable in a physical sense - it has meaning.
We do this by identifying the most important frequencies to include via using the so-called Power Spectrum.

I will use a particular form that has useful corresponding meaning: $I_{j} = \frac{a_{j}^{2} + b_{j}^{2}}{2}$
In this form, it also gives the variance explained by the inclusion of that frequency in the Fourier series.

Fourier Series Representation

\begin{aligned} d t & = a_{0} \\ + a_{1} c o s (2 π t / 12) + b_{1} s i n (2 π t / 12) \\ + a_{2} c o s (4 π t / 12) + b_{2} s i n (4 π t / 12) \\ + a_{3} c o s (6 π t / 12) + b_{3} s i n (6 π t / 12) \end{aligned}

Methods of Estimating Parameters

Direct calculation.
Optimisation.
Linear Regression.
I will show how to do all of these, but in the next slides I will show a particular form of how to use regression for this.

Normal Equations for Regression

\begin{aligned} \hat{D} (t) & = a_{0} \\ + a_{1} c o s (2 π t / 12) + a_{2} s i n (2 π t / 12) \\ + a_{3} c o s (4 π t / 12) + a_{4} s i n (4 π t / 12) \\ + a_{5} c o s (6 π t / 12) + a_{6} s i n (6 π t / 12) \end{aligned}

Let

$x_{1} = c o s (2 π t / 12)$ ,
$x_{2} = s i n (2 π t / 12)$ ,
$x_{3} = c o s (4 π t / 12)$ ,
$x_{4} = s i n (4 π t / 12)$ ,
$x_{5} = c o s (6 π t / 12)$ ,
$x_{6} = s i n (6 π t / 12)$ .

Therefore we can rewrite the formula above as

\begin{array}{r} \hat{D} (t) = a_{0} + a_{1} x_{1} + a_{2} x_{2} + a_{3} x_{3} + a_{4} x_{4} + a_{5} x_{5} + a_{6} x_{6} \end{array}

Derivation

The problem is to find $a_{0}, a_{1}, . . . a_{6}$ to minimise the sum of squared deviations between the model and the data over the day.

\begin{aligned} E & = \sum_{t = t_{0}}^{t_{n}} (\hat{S} (t) - S (t))^{2} \\ = \sum_{t = t_{0}}^{t_{n}} (a_{0} + \sum_{j = 1}^{6} a_{j} x_{j} (t) - S (t))^{2} \end{aligned}

To calculate $a_{0}, a + 1, . . . a_{6}$ , we take the partial derivatives of $E$ with respect to each and set these to zero to form a system of linear equations which we then solve.

\begin{aligned} \frac{\partial E}{\partial a_{0}} & = 2 \sum_{t = t_{0}}^{t_{n}} (a_{0} + \sum_{j = 0}^{6} a_{j} x_{j} (t) - S (t)) = 0 \\ \frac{\partial E}{\partial a_{1}} & = 2 \sum_{t = t_{0}}^{t_{n}} x_{1} (a_{0} + \sum_{j = 0}^{6} a_{j} x_{j} (t) - S (t)) = 0 \\ . . . . . . . & = . . . . . . . . . . . \\ \frac{\partial E}{\partial a_{6}} & = 2 \sum_{t = t_{0}}^{t_{n}} x_{4} (a_{0} + \sum_{j = 0}^{6} a_{j} x_{j} (t) - S (t)) = 0 \end{aligned}

The set of equations above can be written in the following form
$A ϕ = B$

A (7, 7) = [\begin{matrix} n & \sum_{t = t_{0}}^{t_{n}} x_{1} (t) & \dots & \sum_{t = t_{0}}^{t_{n}} x_{6} (t) \\ \sum_{t = t_{0}}^{t_{n}} x_{1} (t) & \sum_{t = t_{0}}^{t_{n}} x_{1}^{2} (t) & \dots & \sum_{t = t_{0}}^{t_{n}} x_{1} (t) x_{6} t \\ ⋮ & ⋱ & ⋱ & ⋮ \\ \sum_{t = t_{0}}^{t_{n}} x_{6} (t) & \sum_{t = t_{0}}^{t_{n}} x_{1} (t) x_{6} (t) & \dots & \sum_{t = t_{0}}^{t_{n}} x_{6}^{2} (t) \end{matrix}]

ϕ = [\begin{matrix} a_{0} \\ a_{1} \\ ⋮ \\ a_{6} \end{matrix}]

B = [\begin{matrix} \sum_{t = t_{0}}^{t_{n}} S (t) \\ \sum_{t = t_{0}}^{t_{n}} x_{1} (t) S (t) \\ ⋮ \\ \sum_{t = t_{0}}^{t_{n}} x_{6} (t) S (t) \end{matrix}]

$ϕ = A^{- 1} B$

Intraday Solar Radiation

One can deal with the seasonalities inherent in climate variable in a multiplicative or additive modelling framework.
Interestingly, though I will argue that the multiplicative approach is problematic, it is the approach most often used for solar forecasting models in particular. There are two versions of the multiplicative approach with respect to solar radiation, calculating the clearness index, and estimating the clear sky index.
To form the clearness index, one divides the global solar radiation by the extraterrestrial radiation, a quantity determined only via astronomical formulae. On the other hand the clear sky index involves dividing the global radiation by a clear sky model.
Note that the wind resource is not as seasonally dependent as the solar radiation, and both multiplicative and additive versions of dealing with seasonality are used.
Additive de-seasoning is enacted through subtracting a mean function from the solar radiation, that function formed usually through the addition of terms involving a basis of the function space.
I will argue that an appropriate way to perform this operation is through the use of a Fourier set of basis functions.

Seasonality modelling methods

Multiplicative, dividing the data by clearness index.
Multiplicative, dividing the data by clearness sky model.
Additive, using Fourier series or wavelets.

Clearness Index - advantages and disadvantages

It is a calculated, not modelled value as the divisor is the so-called extraterrestrial radiation $H_{t}$ .
$H_{t}$ is the result of a calculation using spherical trigonometry applied to the ‘solar constant’.
From my experience it does not pick up all the seasonal effects.

Clear Sky Index - advantages and disadvantages

It is based on a physical model of a so-called clear sky, and varies throughout the year.
It uses some parameter values that vary at high frequency but are assumed to vary at low frequency.
There are several clear sky models - Ineichen evaluates thirteen.

Seasonality

The first step is to identify and model the seasonality. We have identified several significant cycles using spectral analysis. Fourier series will be used in this step.
。。。。。。。

Objectives of time series analysis

Use model to provide compact description of data
Recognise presence of seasonal components and remove them so as not to confuse them with long-term trends (seasonal adjustment).
Other applications of time series models
- Separation (or filtering) of noise from signals
- Prediction of future values of a series
- Testing hypotheses
- Predicting one series from observations of another
- Controlling future values of a series by adjusting parameters
Time series models are also useful in simulation studies

Some simple time series models

Important part of time series analysis: select, suitable probability model for data

Definition

A time series model for observed data $x_{t}$ is a specification of the joint distributions (or only means and covariances) of a sequence of random variables $X_{t}$ of which $x_{t}$ is a realization.

A complete probabilistic time series model for $X_{1}, X_{2}, . . .$ specifies all the probabilities

$P [X_{1} \leq x_{1}, ..., X_{n} \leq x_{n}], - \infin < x_{1}, . . ., x_{n} < \infin, n = 1, 2, . . .$

Such a specification of all joint distributions is rarely used because it contains too many parameters to be estimated from avaliable data.