# Tricks with Boolean Series¶

This notebook discusses Boolean series, and various useful things you can do with them.

First let’s import our libraries:

import pandas as pd
import numpy as np

rng = np.random.default_rng(20201103)


## Computing Probability¶

If we have a logical series:

xb = pd.Series([True, False, True, True, True, False, False, True, False, True])
xb

0     True
1    False
2     True
3     True
4     True
5    False
6    False
7     True
8    False
9     True
dtype: bool


We can count the number of True values with sum:

xb.sum()

6


We can count the fraction of True values, or the probability of True, with mean():

xb.mean()

0.6


The NumPy equivalents also work:

np.mean(xb)

0.6


## Creating with Logical Operations¶

If you do a logical operation (==, <, >, <=, or >=) on a series, comparing it to either a fixed value or another series, you will get a Boolean series.

This is very useful for creating the outcome vector for a logistic regression, as the boolean series will be treated as 1 (True) and 0 (False), and can be predicted with either StatsModels or SciKit-Learn’s logistic regression or other classifier.

For example, if we draw some random numbers:

xs = pd.Series(rng.random(size=1000))
xs

0      0.163352
1      0.333021
2      0.054169
3      0.662442
4      0.517284
...
995    0.882045
996    0.897472
997    0.161948
998    0.850922
999    0.222618
Length: 1000, dtype: float64

xs.hist()

<matplotlib.axes._subplots.AxesSubplot at 0x22e1f5277f0>

xs.describe()

count    1000.000000
mean        0.491382
std         0.285871
min         0.000805
25%         0.235146
50%         0.489305
75%         0.737922
max         0.997661
dtype: float64


We can get a series that is True when the number is at least 0.8:

xs_ge = xs >= 0.8
xs_ge

0      False
1      False
2      False
3      False
4      False
...
995     True
996     True
997    False
998     True
999    False
Length: 1000, dtype: bool


We can verify that values start at 0.8:

xs[xs_ge].min()

0.8050954039782324


Since the values are drawn uniformly in the range $$[0,1)$$, then approximately 20% of the values should be at least 0.8 ($$P(X \ge 0.8) = 0.2$$). Let’s check:

xs_ge.mean()

0.183


## Logical Operations¶

The bitwise negation operator, ~, negates a boolean series:

~xb

0    False
1     True
2    False
3    False
4    False
5     True
6     True
7    False
8     True
9    False
dtype: bool


The np.logical_not function also does:

np.logical_not(xb)

0    False
1     True
2    False
3    False
4    False
5     True
6     True
7    False
8     True
9    False
dtype: bool


We can combine them with bitwise and (&) or or (|).

Let’s find all the values between 0.8 and 0.9:

xs_in_range = xs_ge & (xs < 0.9)
xs[xs_in_range].describe()

count    87.000000
mean      0.849424
std       0.028517
min       0.805095
25%       0.826880
50%       0.845260
75%       0.878704
max       0.897472
dtype: float64


We can find everything except $$[0.2,0.8)$$:

xs_lohi = xs_ge | (xs < 0.2)
xs[xs_lohi].hist()

<matplotlib.axes._subplots.AxesSubplot at 0x22e1edadeb0>


Look, we cut out the middle!