Skip to content
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `nbformat >= 4.2.0` dependency to `[visuals]` extra ([#169](https://github.com/MobileTeleSystems/RecTools/pull/169))
- `filter_interactions` method of `Dataset` ([#177](https://github.com/MobileTeleSystems/RecTools/pull/177))
- `on_unsupported_targets` parameter to `recommend` and `recommend_to_items` model methods ([#177](https://github.com/MobileTeleSystems/RecTools/pull/177))
- Implement `fit_partial()` for `ImplicitALSWrapperModel` and `LightFMWrapperModel` ([#179](https://github.com/MobileTeleSystems/RecTools/pull/179))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very excited about the new feature. Thank you for the PR!

Let's discuss what we're trying to achieve with fit_partial.
In implicit ALS has partial_fit_users and partial_fit_items methods that allow to update weights only for specific parts of users/items and also support feeding new users and items to already fitted model.
LightFM model has different logic. It has fit_partial method. From docs: "Unlike fit, repeated calls to this method will cause training to resume from the current model state". This method does not support adding new users/items. And it is not meant to update only parts of embeddings.

What exactly do we want to achieve with fit_partial in rectools? I would say that first step would be to follow LightFM logic. We are missing the ability to resume training, out fit always refits models from the beginning, just like LightFM fit.

As for partial_fit_users and partial_fit_items - these are very model specific ALS methods. I wouldn't call them fit_partial. I would move them to a separate issue and PR.

@chezou @feldlime what do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

My motivation for adding fit_partial is to enable incremental training. For example, weekly batches for full training and daily incremental training to adopt new and existing user transactions.

I don't seriously care about preserving old users/items. Rather, shortening training time would be important.

Actually, LightFM has a known workaround for adding a new user/item. lyst/lightfm#347 (comment)
I included that workaround into the patch. ea33c05

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great feature for LigtFM, thank you!

So there are two major usage cases for fit_partial:

  1. Incremental training and support for adding new items and features. This is executed when dataset is changing during calls.
  2. Epochal training. This is executed when dataset is staying the same during calls. It just resumes training from the previous state like Lightfm.fit_partial

Right now in LightFM you implemented support for both cases but in ALS only for one of them (incremental). Could you please add support for ALS epochal training (with NotImplementedError if features are present)? Otherwise users would be very confused for different behaviour of the same method. And could you please add an optional epochs argument to fit_partial of ALS?

We would also need tests for both models. With fitting model for 10 epochs with fit and 10 times fitting models with fit_partial for 1 epoch. And finals embeds should be equal.

How do you feel about the whole story?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Let me add tests for re-training with the same dataset.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the tests 117c5c7

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blondered I think 117c5c7 shows how it works for the use case 2.

Please let me know if you'd like to add anything else.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Sorry for the delay. It's summer and it was a vacation time :)

- Use nmslib-metabrainz for Python 3.11 and upper ([#180](https://github.com/MobielTeleSystems/RecTools/pull/180))

### Fixed
Expand Down
102 changes: 99 additions & 3 deletions rectools/dataset/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
from scipy import sparse

from rectools import Columns
from rectools.types import ExternalIdsArray, InternalIdsArray

from .features import AbsentIdError, DenseFeatures, Features, SparseFeatures
from .identifiers import IdMap
Expand Down Expand Up @@ -92,6 +93,22 @@ def get_hot_item_features(self) -> tp.Optional[Features]:
return None
return self.item_features.take(range(self.n_hot_items))

def get_hot_users_internal(self) -> InternalIdsArray:
"""Return internal ids of hot users."""
return np.arange(self.n_hot_users)

def get_hot_items_internal(self) -> InternalIdsArray:
"""Return internal ids of hot items."""
return np.arange(self.n_hot_items)

def get_hot_users_external(self) -> ExternalIdsArray:
"""Return external ids of hot users."""
return self.user_id_map.convert_to_external(self.get_hot_users_internal())

def get_hot_items_external(self) -> ExternalIdsArray:
"""Return external ids of hot items."""
return self.item_id_map.convert_to_external(self.get_hot_items_internal())

@classmethod
def construct(
cls,
Expand Down Expand Up @@ -139,9 +156,7 @@ def construct(
Dataset
Container with all input data, converted to `rectools` structures.
"""
for col in (Columns.User, Columns.Item):
if col not in interactions_df:
raise KeyError(f"Column '{col}' must be present in `interactions_df`")
cls._check_columns_present(interactions_df)
user_id_map = IdMap.from_values(interactions_df[Columns.User].values)
item_id_map = IdMap.from_values(interactions_df[Columns.Item].values)
interactions = Interactions.from_raw(interactions_df, user_id_map, item_id_map)
Expand Down Expand Up @@ -195,6 +210,12 @@ def _make_features(
except Exception as e: # pragma: no cover
raise RuntimeError(f"An error has occurred while constructing {feature_type} features: {e!r}")

@staticmethod
def _check_columns_present(interactions_df: pd.DataFrame) -> None:
for col in (Columns.User, Columns.Item):
if col not in interactions_df:
raise KeyError(f"Column '{col}' must be present in `interactions_df`")

def get_user_item_matrix(
self,
include_weights: bool = True,
Expand Down Expand Up @@ -309,3 +330,78 @@ def _handle_features(
item_features=item_features_new,
)
return filtered_dataset

def rebuild_with_new_data(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of things could go wrong with features because we don't do any checks. And user will not know about wrong behaviour.
Let's add on_unconsistent_features argument with options "raise" and "ignore". If "raise" is specified, then after features creation there needs to be a check that old dataset feature names are exactly the same as new feature names (and they are also in the same order). If something is not right then ValueError is raised

self,
interactions_df: pd.DataFrame,
user_features_df: tp.Optional[pd.DataFrame] = None,
cat_user_features: tp.Iterable[str] = (),
make_dense_user_features: bool = False,
item_features_df: tp.Optional[pd.DataFrame] = None,
cat_item_features: tp.Iterable[str] = (),
make_dense_item_features: bool = False,
) -> "Dataset":
"""
Create new dataset by merging user_id_map and item_id_map.
This function is useful when you want to use fit_partial.

Parameters
----------
interactions_df : pd.DataFrame
New interactions table.
The same structure as in `construct` method.
user_features_df, item_features_df : pd.DataFrame, optional
New user (item) explicit features table.
The same structure as in `construct` method.
cat_user_features, cat_item_features : tp.Iterable[str], default ``()``
List of categorical user (item) feature names for
`SparseFeatures.from_flatten` method.
Used only if `make_dense_user_features` (`make_dense_item_features`)
flag is ``False`` and `user_features_df` (`item_features_df`) is not ``None``.
make_dense_user_features, make_dense_item_features : bool, default ``False``
Create user (item) features as dense or sparse.
Used only if `user_features_df` (`item_features_df`) is not ``None``.
- if ``False``, `SparseFeatures.from_flatten` method will be used;
- if ``True``, `DenseFeatures.from_dataframe` method will be used.

Returns
-------
Dataset
New dataset with added data.
"""
self._check_columns_present(interactions_df)

old_hot_user_id_map = IdMap.from_dict(dict(zip(self.get_hot_users_external(), self.get_hot_users_internal())))
old_hot_item_id_map = IdMap.from_dict(dict(zip(self.get_hot_items_external(), self.get_hot_items_internal())))
new_user_id_map = old_hot_user_id_map.add_ids(
interactions_df[Columns.User].values, raise_if_already_present=False
)
new_item_id_map = old_hot_item_id_map.add_ids(
interactions_df[Columns.Item].values, raise_if_already_present=False
)
new_interactions = Interactions.from_raw(interactions_df, new_user_id_map, new_item_id_map)

new_user_features, new_user_id_map = self._make_features(
user_features_df,
cat_user_features,
make_dense_user_features,
new_user_id_map,
Columns.User,
"user",
)
new_item_features, new_item_id_map = self._make_features(
item_features_df,
cat_item_features,
make_dense_item_features,
new_item_id_map,
Columns.Item,
"item",
)

return Dataset(
new_user_id_map,
new_item_id_map,
new_interactions,
new_user_features,
new_item_features,
)
20 changes: 20 additions & 0 deletions rectools/models/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,26 @@ def fit(self: T, dataset: Dataset, *args: tp.Any, **kwargs: tp.Any) -> T:
def _fit(self, dataset: Dataset, *args: tp.Any, **kwargs: tp.Any) -> None:
raise NotImplementedError()

def fit_partial(self: T, dataset: Dataset, *args: tp.Any, **kwargs: tp.Any) -> T:
"""
Partial fit model.

Parameters
----------
dataset : Dataset
Dataset with input data.

Returns
-------
self
"""
self._fit_partial(dataset, *args, **kwargs)
self.is_fitted = True
return self

def _fit_partial(self, dataset: Dataset, *args: tp.Any, **kwargs: tp.Any) -> None:
raise NotImplementedError()

def _custom_transform_dataset_u2i(
self, dataset: Dataset, users: ExternalIds, on_unsupported_targets: ErrorBehaviour
) -> Dataset:
Expand Down
20 changes: 20 additions & 0 deletions rectools/models/implicit_als.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,26 @@ def _fit(self, dataset: Dataset) -> None: # type: ignore
self.verbose,
)

def _fit_partial(self, dataset: Dataset) -> None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've discussed it a lot. Let's not do ALS support for fit_partial for now.
Main reasons are:

  1. ALS wrapper has different behaviour then LightFM wrapper in current implementation. ALS cannot do fit_partial on model that has not yet been fitted. We can't leave it as it is.
  2. Features support should be here for ALS. So we're gonna have to rewrite some of our code for features support in fit to add support for fit_partial and also for epochs parameter. And it's a long story. It should be in another PR.

# Implicit ALS assumes to fit first on all data.
if not self.is_fitted:
raise NotFittedError(self.__class__.__name__)
users = dataset.get_hot_users_internal()
items = dataset.get_hot_items_internal()

ui_csr = dataset.get_user_item_matrix(
include_weights=True, include_warm_users=True, include_warm_items=True
).astype(np.float32)
iu_csr = ui_csr[:, items].T.tocsr(copy=False)

# TODO: implement partial fit for explicit features
if dataset.get_hot_item_features() or dataset.get_hot_user_features():
raise NotImplementedError("fit_partial with explicit features is not implemented")

for _ in range(self.model.iterations):
self.model.partial_fit_users(users, ui_csr[users])
self.model.partial_fit_items(items, iu_csr)

def _get_users_factors(self, dataset: Dataset) -> Factors:
return Factors(get_users_vectors(self.model))

Expand Down
75 changes: 75 additions & 0 deletions rectools/models/lightfm.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import numpy as np
from lightfm import LightFM
from scipy import sparse
from sklearn.base import clone

from rectools.dataset import Dataset, Features
from rectools.exceptions import NotFittedError
Expand Down Expand Up @@ -89,6 +90,80 @@ def _fit(self, dataset: Dataset) -> None: # type: ignore
verbose=self.verbose > 0,
)

def _fit_partial(self, dataset: Dataset, epochs: tp.Optional[int] = None) -> None:
if not self.is_fitted:
self.model = deepcopy(self._model)

ui_coo = dataset.get_user_item_matrix(include_weights=True).tocoo(copy=False)
user_features = self._prepare_features(dataset.get_hot_user_features(), dataset.n_hot_users)
item_features = self._prepare_features(dataset.get_hot_item_features(), dataset.n_hot_items)
epochs = epochs if epochs is not None else self.n_epochs
sample_weight = None if self._model.loss == "warp-kos" else ui_coo

if self.is_fitted:
self.model._check_initialized() # pylint: disable=W0212
self._resize_model(ui_coo, user_features, item_features)

Comment thread
feldlime marked this conversation as resolved.
Resolved
self.model.fit_partial(
ui_coo,
user_features=user_features,
item_features=item_features,
sample_weight=sample_weight,
epochs=epochs,
num_threads=self.n_threads,
verbose=self.verbose > 0,
)

# Based on LightFMResizable by @JohnPaton
# https://github.com/lyst/lightfm/issues/347#issuecomment-707829342
def _resize_model(
self,
interactions: sparse.coo_matrix,
user_features: tp.Optional[sparse.csr_matrix] = None,
item_features: tp.Optional[sparse.csr_matrix] = None,
) -> None:
"""Resizes the model to accommodate new users/items/features"""
no_components = self.model.no_components
no_user_features, no_item_features = interactions.shape

if user_features is not None and hasattr(user_features, "shape"):
no_user_features = user_features.shape[-1]
if item_features is not None and hasattr(item_features, "shape"):
no_item_features = item_features.shape[-1]

if (
no_user_features == self.model.user_embeddings.shape[0]
and no_item_features == self.model.item_embeddings.shape[0]
):
return

new_model = clone(self.model)
new_model._initialize(no_components, no_item_features, no_user_features) # pylint: disable=W0212

for attr in (
"item_embeddings",
"item_embedding_gradients",
"item_embedding_momentum",
"item_biases",
"item_bias_gradients",
"item_bias_momentum",
"user_embeddings",
"user_embedding_gradients",
"user_embedding_momentum",
"user_biases",
"user_bias_gradients",
"user_bias_momentum",
):
# extend attribute matrices with new rows/cols from
# freshly initialized model with right shape
old_array = getattr(self.model, attr)
old_slice = [slice(None, i) for i in old_array.shape]
new_array = getattr(new_model, attr)
new_array[tuple(old_slice)] = old_array
setattr(self.model, attr, new_array)

return

@staticmethod
def _prepare_features(features: tp.Optional[Features], n_hot: int) -> tp.Optional[sparse.csr_matrix]:
if features is None:
Expand Down
Loading