pandas · Lesson 0.2

Columns, not loops

The big idea of pandas: do maths on whole columns at once instead of looping over rows.

1A column is a Series

Each column of a DataFrame is a Series: a list of values with labels. Maths on a Series happens to every value at once.

import pandas as pd

df = pd.DataFrame({"product": ["tea", "mug", "cake"], "price": [3.5, 9.0, 4.25]})
df["price"] * 1.2

pandas

Practice

Store each row's revenue, price times qty, in result (a Series).

import pandas as pd

df = pd.DataFrame({"price": [3.5, 9.0, 4.25], "qty": [10, 3, 8]})
result = ...

pandas

2Compare without a loop

In plain Python you'd loop and add up. In pandas, the column maths and .sum() do the whole job in one line, and much faster on big data.

import pandas as pd

prices = [3.5, 9.0, 4.25]
qty = [10, 3, 8]

total = 0
for p, q in zip(prices, qty):
    total += p * q
print(total)

df = pd.DataFrame({"price": prices, "qty": qty})
print((df["price"] * df["qty"]).sum())

pandas

Practice

Store the total revenue (the sum of price * qty) in result.

import pandas as pd

df = pd.DataFrame({"price": [12.0, 4.5, 30.0, 2.25], "qty": [2, 10, 1, 8]})
result = ...

pandas

3True and False columns

A comparison gives a column of True and False. True counts as 1, so .sum() counts the matches and .mean() gives the share.

import pandas as pd

scores = pd.Series([55, 81, 92, 47, 76])
passed = scores >= 60
print(passed.sum(), passed.mean())

pandas

Practice

Store the share of orders over 100 in result (a number between 0 and 1).

import pandas as pd

amounts = pd.Series([25, 140, 60, 310, 15, 105, 99, 230])
result = ...

pandas

4Built-in maths for whole columns

Rounding, absolute values and clipping also work on whole columns, with methods like .round(), .abs() and .clip().

import pandas as pd

changes = pd.Series([-3.456, 2.1, -0.5, 7.891])
print(changes.round(1).tolist())
print(changes.abs().tolist())
print(changes.clip(lower=0).tolist())

pandas

Practice

Some stock counts are negative because of data errors. Store a Series in result where every negative value becomes 0.

import pandas as pd

stock = pd.Series([12, -3, 0, 7, -1])
result = ...

pandas