Regression Using Embeddings
Regression using the embeddings
Regression means predicting a number, rather than one of the categories. We will predict the score based on the embedding of the review's text. We split the dataset into a training and a testing set for all of the following tasks, so we can realistically evaluate performance on unseen data. The dataset is created in the Get_embeddings_from_dataset Notebook.
We're predicting the score of the review, which is a number between 1 and 5 (1-star being negative and 5-star positive).
text-embedding-3-small performance on 1k Amazon reviews: mse=0.65, mae=0.52
Dummy mean prediction performance on Amazon reviews: mse=1.73, mae=1.03
We can see that the embeddings are able to predict the scores with an average error of 0.53 per score prediction. This is roughly equivalent to predicting half of reviews perfectly, and half off by one star.
You could also train a classifier to predict the label, or use the embeddings within an existing ML model to encode free text features.