kubeflow
/
examples
mirror of https://github.com/kubeflow/examples.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
examples/h-and-m-fash-rec-kaggle-com.../h&m-fash-rec-orig.ipynb

439 lines
10 KiB
Plaintext
Raw Blame History

{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"# Kaggle Featured Prediction Competition: H&M Personalized Fashion Recommendations"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"In this [competition](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations), product recommendations have to be done based on previous purchases. There's a whole range of data available including customer meta data, product meta data, and meta data that spans from simple data, such as garment type and customer age, to text data from product descriptions, to image data from garment images."
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Install necessary packages\n",
"\n",
"We can install the necessary package by either running `pip install --user <package_name>` or include everything in a `requirements.txt` file and run `pip install --user -r requirements.txt`. We have put the dependencies in a `requirements.txt` file so we will use the former method.\n",
"\n",
"Restart the kernel after installation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# !pip install --user -r requirements.txt"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np # linear algebra\n",
"import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
"import implicit"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"path = \"data/\"\n",
"train_data_filepath = path + \"transactions_train.csv\"\n",
"article_metadata_filepath = path + \"articles.csv\"\n",
"customer_metadata_filepath = path + \"customers.csv\"\n",
"test_data_filepath = path + \"sample_submission.csv\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_data = pd.read_csv(train_data_filepath,index_col='customer_id')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_data.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_data.drop(['t_dat','sales_channel_id','price'],axis= 1, inplace = True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_data.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_data=train_data.sort_values(by=['customer_id']).reset_index()\n",
"train_data.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(\"Unique customers\",train_data['customer_id'].nunique())\n",
"print(\"Unique articles\",train_data['article_id'].nunique())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_data.info()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"X = train_data.groupby(['customer_id', 'article_id'])['article_id'].count().reset_index(name = \"purchase_count\") "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"unique_customers = X['customer_id'].unique()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"unique_articles = X['article_id'].unique()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"customer_id_dict = {unique_customers[i]:i for i in range(len(unique_customers))}"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"reverse_customer_id_dict = {i:unique_customers[i] for i in range(len(unique_customers))} "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"numeric_cus_id = []"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for i in range(len(X['customer_id'])):\n",
" numeric_cus_id.append(customer_id_dict.get(X['customer_id'][i]))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(X['customer_id'].nunique())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"len(numeric_cus_id)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"X['customer_id'] = numeric_cus_id"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"X.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"article_id_dict = {unique_articles[i]:i for i in range(len(unique_articles))}\n",
"reverse_article_id_dict = {i:unique_articles[i] for i in range(len(unique_articles))} "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"numeric_art_id = []"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for i in range(len(X['article_id'])):\n",
" numeric_art_id.append(article_id_dict.get(X['article_id'][i]))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"X['article_id'] = numeric_art_id\n",
"X.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Constructing sparse matrices for alternating least squares algorithm \n",
"import scipy.sparse as sparse\n",
"sparse_user_item_coo = sparse.coo_matrix((X.purchase_count, (X.customer_id, X.article_id)), shape = (n_customers, n_articles))\n",
"sparse_user_item_csr = sparse.csr_matrix((X['purchase_count'], (X['customer_id'], X['article_id'])), shape = (n_customers, n_articles))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# parameters for the model\n",
"als_params = dict(\n",
" factors = 200, # number of latent factors - try between 50 to 1000\n",
" regularization = 0.01, # regularization factor - try between 0.001 to 0.2\n",
" iterations = 5, # iterations - try between 2 to 100\n",
")\n",
"\n",
"# initialize a model\n",
"model = implicit.als.AlternatingLeastSquares(**als_params)\n",
"\n",
"# train the model on a sparse matrix of user/item/confidence weights \n",
"model.fit(sparse_user_item_csr)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"test_data = pd.read_csv(test_data_filepath)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"test_data.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"predictions=[]\n",
"count = 0\n",
"for cust_id in test_data.customer_id:\n",
" cust_id = customer_id_dict.get(cust_id)\n",
"# if(cust_id!=None): \n",
" recommendations = model.recommend(cust_id, sparse_user_item_csr[cust_id],10)\n",
" result=[]\n",
" for i in range(len(recommendations[0])):\n",
" val = reverse_article_id_dict.get(recommendations[0][i])\n",
" result.append(val) \n",
" predictions.append(result)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"test_data['prediction'] = predictions\n",
"test_data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"test_data.to_csv('submission.csv', index=False)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"kubeflow_notebook": {
"autosnapshot": true,
"experiment": {
"id": "",
"name": ""
},
"experiment_name": "",
"katib_metadata": {
"algorithm": {
"algorithmName": "grid"
},
"maxFailedTrialCount": 3,
"maxTrialCount": 12,
"objective": {
"objectiveMetricName": "",
"type": "minimize"
},
"parallelTrialCount": 3,
"parameters": []
},
"katib_run": false,
"pipeline_description": "",
"pipeline_name": "",
"snapshot_volumes": true,
"steps_defaults": [
"label:access-ml-pipeline:true",
"label:access-rok:true"
],
"volume_access_mode": "rwm",
"volumes": [
{
"annotations": [],
"mount_point": "/home/jovyan",
"name": "hm-fash-workspace-fhh9d",
"size": 50,
"size_type": "Gi",
"snapshot": false,
"type": "clone"
}
]
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in New Issue View Git Blame Copy Permalink