# One Sample

## Contents

# One Sample#

This notebook demonstrates the one-sample *t*-test and checking normality with a Q-Q plot, using the ZARR13.DAT file from the *NIST e-Handbook of Statistical Methods*.

## Setup#

Import our Python modules:

```
import pandas as pd
import numpy as np
import scipy.stats as sps
import statsmodels.api as sm
import matplotlib.pyplot as plt
```

Read the ZARR13.DAT file:

```
data = pd.read_table('ZARR13.DAT', skiprows=25, names=['X'])
data
```

## Showing the Distribution#

Letâ€™s look at histogram:

```
plt.hist(data.X, bins=20)
plt.show()
```

And the distribution statistics:

```
data.X.describe()
```

### Q-Q Plot#

The data looks normal-ish, but a histogram isnâ€™t a very reliable way to assess normality. A Q-Q plot against the normal distribution lets us be much more precise:

```
sm.qqplot(data.X, fit=True, line='45')
plt.show()
```

That data looks normal! Itâ€™s a straight line right through the bulk of the data. Itâ€™s common for the first and last few points to deviate from normal just a little bit more than the central mass of data points.

### Drawing Q-Q Ourselves#

Letâ€™s try to draw our own. The way we draw a Q-Q plot is this:

Sort the data values in ascending order. These will be plotted on the

*y*axis.Compute the percentile for each data point - where is it in the range of data points? We can do this with its position or count. We donâ€™t label any point 0 or 1; instead, for point \(i \in [1,n]\), we compute \(v = i / (n + 1)\).

Compute the quantiles in the reference distribution (in our case, normal) for each data point position. These will be plotted on the

*x*axis.

Letâ€™s sort:

```
observed = data['X'].sort_values()
```

Now we need to compute the percentiles for each position. The `arange`

function is useful for this - it can generate \(i = 1 \dots n\), and we can divide to rescale the points:

```
nobs = len(observed)
pred_ps = np.arange(1, nobs + 1) / (nobs + 1)
```

And we need to convert these percentiles into quantile values from the standard normal:

```
norm_dist = sps.norm()
pred_vs = norm_dist.ppf(pred_ps)
```

Finally, we can plot them against each other:

```
plt.scatter(pred_vs, observed)
plt.xlabel('Theoretical Quantiles')
plt.ylabel('Sample Quantiles')
plt.show()
```

We can see the straight line, but two issues remain:

Our sample quantiles are on the original scale, but theoretical are standardized. Letâ€™s standardize the sample quantiles so the values, not just relative shape, are comparable.

We donâ€™t have the refernce line. This will be easier to draw with standardized sample quantiles.

**Standardization**, for normally-distributed data, is transforming it to have a mean of 0 and standard deviation of 1. We do this by subtracting the mean and dividing by the sample standard deviation:

```
std_observed = (observed - observed.mean()) / observed.std()
# reference values; once standardized, the line will be y=x
ref_xs = np.linspace(np.min(pred_vs), np.max(pred_vs), 1000)
plt.plot(ref_xs, ref_xs, color='red')
# add the points
plt.scatter(pred_vs, std_observed)
# and labels
plt.xlabel('Theoretical Quantiles')
plt.ylabel('Sample Quantiles')
plt.show()
```

## T-tests#

Letâ€™s start with a 1-sample T-test for \(H_0: \mu = 9\):

```
sps.ttest_1samp(data.X, 9)
```

If \(\mu\) were nine, it would be extremely unlikely to find a sample with this observed mean.

These values are very tightly distributed. A 1-sample T-test for \(H_0: \mu = 9.25\):

```
sps.ttest_1samp(data.X, 9.25)
```

Our data would also be very unlikely, but not *as* unlikely, under this null hypothesis.

And just to see it accept the null, letâ€™s try \(H_0: \mu = 9.26\):

```
sps.ttest_1samp(data.X, 9.26)
```

Our data is consistent with \(\mu=9.26\).

**Note:** What I just did here â€” try several different null hypotheses in a row â€” is **not a valid statistical procedure**. I am only doing it to demonstrate the results of a *t*-test both when the null hypothesis holds, and when it does not.