Categorical Encoding¶

Many machine learning models cannot work with categorical data directly and require numerical inputs. sorix provides encoding tools to handle these scenarios.

1. OneHotEncoder¶

The OneHotEncoder creates new binary columns for each unique category in a feature. A 1 represents the presence of a category, while a 0 represents its absence.

Example¶

Let's see an example with categorical features.

In [1]:

# Uncomment the next line and run this cell to install sorix
#!pip install 'sorix @ git+https://github.com/Mitchell-Mirano/sorix.git@main'

# Uncomment the next line and run this cell to install sorix #!pip install 'sorix @ git+https://github.com/Mitchell-Mirano/sorix.git@main'

In [2]:

import numpy as np
import pandas as pd
import sorix
from sorix.preprocessing import OneHotEncoder

# Create sample data with categorical features
data = {
    'color': ['red', 'blue', 'green', 'blue', 'red'],
    'size': ['S', 'M', 'L', 'S', 'M']
}
X = pd.DataFrame(data)
X

encoder = OneHotEncoder()
X_encoded = encoder.fit_transform(X)

print("OneHot Encoded data:\n", X_encoded)
print("\nEncoded features names:\n", encoder.get_features_names())

import numpy as np import pandas as pd import sorix from sorix.preprocessing import OneHotEncoder # Create sample data with categorical features data = { 'color': ['red', 'blue', 'green', 'blue', 'red'], 'size': ['S', 'M', 'L', 'S', 'M'] } X = pd.DataFrame(data) X encoder = OneHotEncoder() X_encoded = encoder.fit_transform(X) print("OneHot Encoded data:\n", X_encoded) print("\nEncoded features names:\n", encoder.get_features_names())

OneHot Encoded data:
 [[0. 0. 1. 0. 0. 1.]
 [1. 0. 0. 0. 1. 0.]
 [0. 1. 0. 1. 0. 0.]
 [1. 0. 0. 0. 0. 1.]
 [0. 0. 1. 0. 1. 0.]]

Encoded features names:
 ['color_blue', 'color_green', 'color_red', 'size_L', 'size_M', 'size_S']

2. Mathematical Formalism of OneHotEncoder¶

Let $X = (x_1, x_2, \ldots, x_L)^\top$ be a sequence of $L$ observations of a nominal categorical feature taking values from a discrete set of $N$ unique categories $\mathcal{C} = \{c_0, c_1, \ldots, c_{N-1}\}$.

Category Indexing and Standard Basis Vectors¶

We define an index mapping function $\sigma : \mathcal{C} \to \{0, 1, \ldots, N-1\}$ that assigns each unique category $c_k$ an integer index $i_k = \sigma(x_k)$.

The One-Hot encoding of an observation $x_k$ with category index $i_k$ is defined as the standard basis vector $\mathbf{e}_{i_k} \in \{0, 1\}^N$, whose $j$-th component is specified by the Kronecker delta:

$$ (\mathbf{e}_{i_k})_j = \delta_{j, i_k} = \begin{cases} 1 & \text{if } j = i_k \\ 0 & \text{if } j \neq i_k \end{cases} \quad \text{for } j \in \{0, 1, \ldots, N-1\} $$

For example, given $N = 4$ categories $\mathcal{C} = \{\text{'blue'}, \text{'green'}, \text{'red'}, \text{'yellow'}\}$ mapped to indices $\{0, 1, 2, 3\}$, the category $\text{'red'}$ ($i_k = 2$) is encoded as:

$$ \mathbf{e}_2 = \begin{bmatrix} 0 & 0 & 1 & 0 \end{bmatrix}^\top \in \{0, 1\}^4 $$

Binary Indicator Matrix Construction¶

Given a dataset of $L$ samples, OneHotEncoder.transform() constructs a binary indicator matrix $\mathbf{E} \in \{0, 1\}^{L \times N}$ by stacking the transposed One-Hot vectors row-wise:

$$ \mathbf{E} = \begin{bmatrix} \mathbf{e}_{i_1}^\top \\ \mathbf{e}_{i_2}^\top \\ \vdots \\ \mathbf{e}_{i_L}^\top \end{bmatrix} = \begin{bmatrix} \delta_{0, i_1} & \delta_{1, i_1} & \dots & \delta_{N-1, i_1} \\ \delta_{0, i_2} & \delta_{1, i_2} & \dots & \delta_{N-1, i_2} \\ \vdots & \vdots & \ddots & \vdots \\ \delta_{0, i_L} & \delta_{1, i_L} & \dots & \delta_{N-1, i_L} \end{bmatrix} \in \{0, 1\}^{L \times N} $$

Each row $k$ contains exactly one non-zero entry ($1$) at position $i_k$, satisfying the partition-of-unity property:

$$ \sum_{j=0}^{N-1} E_{k, j} = 1 \quad \forall k \in \{1, 2, \ldots, L\} $$

---

3. Conceptual Note & API Methods¶

The OneHotEncoder is especially useful for nominal categorical data (where there is no inherent ordinal relationship).

Available methods include:

• fit: Learns the unique categories $\mathcal{C}$ from the training data.
• transform: Transforms categorical inputs into the binary matrix $\mathbf{E} \in \{0, 1\}^{L \times N}$.
• inverse_transform: Reconstructs original categorical labels from binary indicator vectors.
• get_features_names: Returns column names corresponding to each one-hot category feature.

Methods¶

• fit: Learns the unique categories $\mathcal{C}$ and the index mapping $\sigma$ from the data.
• transform: Builds the indicator matrix $\mathbf{E}$.
• inverse_transform: Recovers the category index $i_k$ from an encoded row.
• get_features_names: Returns the names of the generated binary columns.

4. Persistence: Saving and Loading¶

To store the mapping of your labels or weights, sorix recommends using the .sor extension.

A. Direct Saving with sorix.save¶

The most direct way to save a preprocessor is use sorix.save on the whole object.

In [3]:

# Save the fitted encoder
sorix.save(encoder, 'my_encoder.sor')

# Reload the encoder instance
loaded_encoder = sorix.load('my_encoder.sor')

# Verify consistency
X_test = pd.DataFrame({'color': ['blue'], 'size': ['S']})
assert np.allclose(encoder.transform(X_test), loaded_encoder.transform(X_test))
print("OneHotEncoder object successfully saved and reloaded using sorix components!")

# Save the fitted encoder sorix.save(encoder, 'my_encoder.sor') # Reload the encoder instance loaded_encoder = sorix.load('my_encoder.sor') # Verify consistency X_test = pd.DataFrame({'color': ['blue'], 'size': ['S']}) assert np.allclose(encoder.transform(X_test), loaded_encoder.transform(X_test)) print("OneHotEncoder object successfully saved and reloaded using sorix components!")

OneHotEncoder object successfully saved and reloaded using sorix components!

B. Using state_dict and load_state_dict¶

If you only want to save the internal categories without pickling the entire object, you can save the state_dict explicitly.

In [4]:

# 1. Extract the state dictionary
params_dict = encoder.state_dict()

# 2. Save the dictionary with sorix.save (.sor extension)
sorix.save(params_dict, 'encoder_params.sor')

# 3. Load the dictionary with sorix.load
loaded_params = sorix.load('encoder_params.sor')

# 4. Apply state to a fresh instance
new_encoder = OneHotEncoder()
new_encoder.load_state_dict(loaded_params)

# 5. Verify results
assert np.allclose(encoder.transform(X_test), new_encoder.transform(X_test))
print("Encoder state_dict successfully saved and loaded!")

# 1. Extract the state dictionary params_dict = encoder.state_dict() # 2. Save the dictionary with sorix.save (.sor extension) sorix.save(params_dict, 'encoder_params.sor') # 3. Load the dictionary with sorix.load loaded_params = sorix.load('encoder_params.sor') # 4. Apply state to a fresh instance new_encoder = OneHotEncoder() new_encoder.load_state_dict(loaded_params) # 5. Verify results assert np.allclose(encoder.transform(X_test), new_encoder.transform(X_test)) print("Encoder state_dict successfully saved and loaded!")

Encoder state_dict successfully saved and loaded!