构建第一个推荐引擎（小白能看懂的笔记手册）
一、准备
本地环境MAC OS
1.1、安装R语言和
Step 1 安装R语言
安装包下载地址：
点击下载免费版（免费够用），下图红框标出。
下载之前要看下版本信息是否符合否则会容易出问题。我这里的版本是10.15所以满足，你在下载的时候要看清自己的版本哦哦。
安装完毕打开,如下图：
二、构建基础推荐引擎 2.1、载入并格式化数据
下载数据地址：
数据分为=用户名称、title=电影名称、=评分，下面是6名用户。
,title,
Jack ,Lady in the Water,3.0
Jack , on a Plane,4.0
Jack ,You Me and ,3.5
Jack , ,5.0
Jack ,The Night ,3.0
Mick ,Lady in the Water,3.0
Mick , on a Plane,4.0
Mick ,Just My Luck,2.0
Mick , ,3.0
Mick ,You Me and ,2.0
Mick ,The Night ,3.0
Puig, on a Plane,3.5
Puig,Just My Luck,3.0
Puig,You Me and ,2.5
Puig, ,4.0
Puig,The Night ,4.5
Lisa Rose,Lady in the Water,2.5
Lisa Rose, on a Plane,3.5
Lisa Rose,Just My Luck,3.0
Lisa Rose, ,3.5
Lisa Rose,The Night ,3.0
Lisa Rose,You Me and ,2.5
Toby, on a Plane,4.5
Toby, ,4.0
Toby,You Me and ,1.0
Gene ,Lady in the Water,3.0
Gene , on a Plane,3.5
Gene ,Just My Luck,1.5
Gene , ,5.0
Gene ,You Me and ,3.5
Gene ,The Night ,3.0
我们的目标是构建一个推荐引擎，用来基于相似用户的评级推荐他们还没有看过的电影。这里的move.csv就是刚才我们提到的上面的数据
安装我们需要的的包，这个包为数据结构化工具包
我们可以通过如下命令来获取结构化的数据
> movie_ratings = as.data.frame(acast(ratings,title~critic,value.var="rating"))
> View(movie_ratings)
结构化的数据展示如下图
那么我们现在的目标就是基于用户的相似度给用户推荐他们没有评级过的电影。比如：想要给Toby推荐电影时候，会基于与Toby相似的用户提供的评级。
2.2、计算用户相似度
> sim_users=cor(movie_ratings[,1:6],use="complete.obs")
> View(sim_users)
基于用户矩阵求用户相似度以后得出如下图，红色框的两个用户Toby和比较接近相似度0.99最高,然后与Toby相似度为0.92.
2.3、为用户预测未知评级
我们目的是将相似用户给出的评级为Toby推荐他没有评级过的电影。有如下几个步骤：
抽取出Toby没有评级过的电影找到所有其他用户给这些电影的评级将所有用户（不包括Toby自己）对第一步中找到的电影的评级乘以与Toby的相似度计算每一步电影作品的评级和，并除以所有相似度之和。
编码前我们要熟悉一下用到的data.table包和setDT()函数。可以切分数据级、操作数据、合并等高级操作。
2.3.1、 我们要抽取Toby没有评级过的电影
这里需要注意的是安装包以后还需要载入包，才可以使用包中的函数。
install.packages("data.table")
试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/data.table_1.14.0.tgz'
Content type 'application/x-gzip' length 2338988 bytes (2.2 MB)
==================================================
downloaded 2.2 MBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> rating_critic=setDT(movie_ratings[colnames(movie_ratings)[6]],keep.rownames=TRUE)[]
Error in setDT(movie_ratings[colnames(movie_ratings)[6]], keep.rownames = TRUE) : 没有"setDT"这个函数
> library(data.table)
data.table 1.14.0 using 1 threads (see ?getDTthreads). Latest news: r-datatable.com
**********
用中文运行data.table。软件包只提供英语支持。当在在线搜索帮助时，也要确保检查英语错误信息。这个可以通过查看软件包源文件中的po/R-zh_CN.po和po/zh_CN.po文件获得，这个文件可以并排找到母语和英语错误信息。
**********
**********
This installation of data.table has not detected OpenMP support. It should still work but in single-threaded mode.
This is a Mac. Please read https://mac.r-project.org/openmp/. Please engage with Apple and ask them for support. Check r-datatable.com for updates, and our Mac instructions here: https://github.com/Rdatatable/data.table/wiki/Installation. After several years of many reports of installation problems on Mac, it's time to gingerly point out that there have been no similar problems on Windows or Linux.
**********载入程辑包：‘data.table’The following objects are masked from ‘package:reshape2’:dcast, melt> rating_critic=setDT(movie_ratings[colnames(movie_ratings)[6]],keep.rownames=TRUE)[]
>
进行电影评级筛选
> rating_critic=setDT(movie_ratings[colnames(movie_ratings)[6]],keep.rownames=TRUE)[]
> names(rating_critic)=c('title','rating')
> View(rating_critic)
从面列表中筛选未评级的电影。is_na()函数用于筛选出NA值
> titles_na_critic=rating_critic$title[is.na(rating_critic$rating)]
> titles_na_critic
[1] "Just My Luck" "Lady in the Water" "The Night Listener"
2.3.2、 根据评级过上述电影的用户原始数据集和子集获取评级
> ratings_t=ratings[ratings$title %in% titles_na_critic,]
> View(ratings_t)
增加一个相似度
> x=(setDT(data.frame(sim_users[,6]),keep.rownames = TRUE)[])
> names(x)=c('critic','similarity')
> ratings_t=merge(x=ratings_t,y=x,by="critic",all.x = TRUE)
> View(ratings_t)
2.3.3、将相似度值乘以评级，结果作为新的变量:
> ratings_t$sim_rating=ratings_t$rating*ratings_t$similarity
> ratings_tcritic title rating similarity sim_rating
1 Claudia Puig Just My Luck 3.0 0.8934051 2.6802154
2 Claudia Puig The Night Listener 4.5 0.8934051 4.0203232
3 Gene Seymour Lady in the Water 3.0 0.3812464 1.1437393
4 Gene Seymour Just My Luck 1.5 0.3812464 0.5718696
5 Gene Seymour The Night Listener 3.0 0.3812464 1.1437393
6 Jack Matthews Lady in the Water 3.0 0.6628490 1.9885469
7 Jack Matthews The Night Listener 3.0 0.6628490 1.9885469
8 Lisa Rose Lady in the Water 2.5 0.9912407 2.4781018
9 Lisa Rose Just My Luck 3.0 0.9912407 2.9737221
10 Lisa Rose The Night Listener 3.0 0.9912407 2.9737221
11 Mick LaSalle Lady in the Water 3.0 0.9244735 2.7734204
12 Mick LaSalle Just My Luck 2.0 0.9244735 1.8489469
13 Mick LaSalle The Night Listener 3.0 0.9244735 2.7734204
2.3.4、计算每一步电影作品的评级和，并除以所有相似度之和。
对前面步骤中计算出的每部电影所有的进行累加，然后用每部电影的累加的值除以该电影用户相似度的累加值。也就是比如Just My Luck这部电影，Toby的预测评级是通过将评级过Just My Luck的所有值相加，然后除以其与所有这些用户的相似度值的总和来计算的：
> install.packages("dplyr")
试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/dplyr_1.0.5.tgz'
Content type 'application/x-gzip' length 1251016 bytes (1.2 MB)
==================================================
downloaded 1.2 MBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> library(dplyr)载入程辑包：‘dplyr’The following objects are masked from ‘package:data.table’:between, first, lastThe following objects are masked from ‘package:stats’:filter, lagThe following objects are masked from ‘package:base’:intersect, setdiff, setequal, union> result = ratings_t %>% group_by(title) %>% summarise(sum(sim_rating)/sum(similarity))
> result
# A tibble: 3 x 2title `sum(sim_rating)/sum(similarity)`<chr> <dbl>
1 Just My Luck 2.53
2 Lady in the Water 2.83
3 The Night Listener 3.35
> mean(rating_critic$rating,na.rm = T)
[1] 3.166667
整个搭建代码
> ratings = read.csv("~/Documents/知识库/推荐引擎/movie.csv")
> head(ratings)critic title rating
1 Jack Matthews Lady in the Water 3.0
2 Jack Matthews Snakes on a Plane 4.0
3 Jack Matthews You Me and Dupree 3.5
4 Jack Matthews Superman Returns 5.0
5 Jack Matthews The Night Listener 3.0
6 Mick LaSalle Lady in the Water 3.0
> dim(ratings)
[1] 31 3
> Str(ratings)
Error in Str(ratings) : 没有"Str"这个函数
> str(ratings)
'data.frame': 31 obs. of 3 variables:$ critic: chr "Jack Matthews" "Jack Matthews" "Jack Matthews" "Jack Matthews" ...$ title : chr "Lady in the Water" "Snakes on a Plane" "You Me and Dupree" "Superman Returns" ...$ rating: num 3 4 3.5 5 3 3 4 2 3 2 ...
> levels(ratings$critic)
NULL
> levels(ratings$title)
NULL
> as.data.frame(acast(ratings,title~critic,value.var="ratings"))
Error in acast(ratings, title ~ critic, value.var = "ratings") : 没有"acast"这个函数
> as.data.frame(cast(ratings,title~critic,value.var="ratings"))
Error in cast(ratings, title ~ critic, value.var = "ratings") : 没有"cast"这个函数
> library(reshape2)
Error in library(reshape2) : 不存在叫‘reshape2’这个名字的程辑包
> install.packages("dplyr")
also installing the dependencies ‘assertthat’, ‘cli’, ‘crayon’, ‘utf8’, ‘fansi’, ‘pillar’, ‘pkgconfig’, ‘purrr’, ‘digest’, ‘ellipsis’, ‘generics’, ‘glue’, ‘lifecycle’, ‘magrittr’, ‘R6’, ‘rlang’, ‘tibble’, ‘tidyselect’, ‘vctrs’试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/assertthat_0.2.1.tgz'
Content type 'application/x-gzip' length 52572 bytes (51 KB)
==================================================
downloaded 51 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/cli_2.3.1.tgz'
Content type 'application/x-gzip' length 469798 bytes (458 KB)
==================================================
downloaded 458 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/crayon_1.4.1.tgz'
Content type 'application/x-gzip' length 139916 bytes (136 KB)
==================================================
downloaded 136 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/utf8_1.1.4.tgz'
Content type 'application/x-gzip' length 195526 bytes (190 KB)
==================================================
downloaded 190 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/fansi_0.4.2.tgz'
Content type 'application/x-gzip' length 212149 bytes (207 KB)
==================================================
downloaded 207 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/pillar_1.5.1.tgz'
Content type 'application/x-gzip' length 951068 bytes (928 KB)
==================================================
downloaded 928 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/pkgconfig_2.0.3.tgz'
Content type 'application/x-gzip' length 17738 bytes (17 KB)
==================================================
downloaded 17 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/purrr_0.3.4.tgz'
Content type 'application/x-gzip' length 417900 bytes (408 KB)
==================================================
downloaded 408 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/digest_0.6.27.tgz'
Content type 'application/x-gzip' length 300368 bytes (293 KB)
==================================================
downloaded 293 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/ellipsis_0.3.1.tgz'
Content type 'application/x-gzip' length 33497 bytes (32 KB)
==================================================
downloaded 32 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/generics_0.1.0.tgz'
Content type 'application/x-gzip' length 69334 bytes (67 KB)
==================================================
downloaded 67 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/glue_1.4.2.tgz'
Content type 'application/x-gzip' length 139018 bytes (135 KB)
==================================================
downloaded 135 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/lifecycle_1.0.0.tgz'
Content type 'application/x-gzip' length 93309 bytes (91 KB)
==================================================
downloaded 91 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/magrittr_2.0.1.tgz'
Content type 'application/x-gzip' length 224854 bytes (219 KB)
==================================================
downloaded 219 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/R6_2.5.0.tgz'
Content type 'application/x-gzip' length 82447 bytes (80 KB)
==================================================
downloaded 80 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/rlang_0.4.10.tgz'
Content type 'application/x-gzip' length 1327903 bytes (1.3 MB)
==================================================
downloaded 1.3 MB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/tibble_3.1.0.tgz'
Content type 'application/x-gzip' length 803558 bytes (784 KB)
==================================================
downloaded 784 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/tidyselect_1.1.0.tgz'
Content type 'application/x-gzip' length 197492 bytes (192 KB)
==================================================
downloaded 192 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/vctrs_0.3.6.tgz'
Content type 'application/x-gzip' length 1403267 bytes (1.3 MB)
==================================================
downloaded 1.3 MB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/dplyr_1.0.5.tgz'
Content type 'application/x-gzip' length 1251016 bytes (1.2 MB)
==================================================
downloaded 1.2 MBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> library(reshape2)
Error in library(reshape2) : 不存在叫‘reshape2’这个名字的程辑包
> install.packages("dplyr")
试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/dplyr_1.0.5.tgz'
Content type 'application/x-gzip' length 1251016 bytes (1.2 MB)
==================================================
downloaded 1.2 MBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> install.packages("reshape")
also installing the dependencies ‘Rcpp’, ‘plyr’试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/Rcpp_1.0.6.tgz'
Content type 'application/x-gzip' length 3203922 bytes (3.1 MB)
==================================================
downloaded 3.1 MB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/plyr_1.8.6.tgz'
Content type 'application/x-gzip' length 1012642 bytes (988 KB)
==================================================
downloaded 988 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/reshape_0.8.8.tgz'
Content type 'application/x-gzip' length 168563 bytes (164 KB)
==================================================
downloaded 164 KBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> install.packages("reshape2")
also installing the dependencies ‘stringi’, ‘stringr’试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/stringi_1.5.3.tgz'
Content type 'application/x-gzip' length 13641892 bytes (13.0 MB)
==================================================
downloaded 13.0 MB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/stringr_1.4.0.tgz'
Content type 'application/x-gzip' length 210650 bytes (205 KB)
==================================================
downloaded 205 KB试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/reshape2_1.4.4.tgz'
Content type 'application/x-gzip' length 332035 bytes (324 KB)
==================================================
downloaded 324 KBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> library(reshape2)
> as.data.frame(acast(ratings,title~critic,value.var="ratings"))
错误: value.var (ratings) not found in input
> ratings = read.csv("~/Documents/知识库/推荐引擎/movie.csv")
> as.data.frame(acast(ratings,title~critic,value.var="ratings"))
错误: value.var (ratings) not found in input
> as.data.frame(acast(ratings,title~critic,value.var="rating"))Claudia Puig Gene Seymour Jack Matthews Lisa Rose Mick LaSalle
Just My Luck 3.0 1.5 NA 3.0 2
Lady in the Water NA 3.0 3.0 2.5 3
Snakes on a Plane 3.5 3.5 4.0 3.5 4
Superman Returns 4.0 5.0 5.0 3.5 3
The Night Listener 4.5 3.0 3.0 3.0 3
You Me and Dupree 2.5 3.5 3.5 2.5 2Toby
Just My Luck NA
Lady in the Water NA
Snakes on a Plane 4.5
Superman Returns 4.0
The Night Listener NA
You Me and Dupree 1.0
> movie_ratings = as.data.frame(acast(ratings,title~critic,value.var="rating"))
> View(movie_ratings)
> sim_users=cor(movie_ratings[,1:6],use="complete.obs")
> View(sim_users)
> rating_critic=setDT(movie_ratings[colnames(movie_ratings)[6]],keep.rownames=TRUE)[]
Error in setDT(movie_ratings[colnames(movie_ratings)[6]], keep.rownames = TRUE) : 没有"setDT"这个函数
> install.packages("data.table")
试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/data.table_1.14.0.tgz'
Content type 'application/x-gzip' length 2338988 bytes (2.2 MB)
==================================================
downloaded 2.2 MBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> rating_critic=setDT(movie_ratings[colnames(movie_ratings)[6]],keep.rownames=TRUE)[]
Error in setDT(movie_ratings[colnames(movie_ratings)[6]], keep.rownames = TRUE) : 没有"setDT"这个函数
> library(data.table)
data.table 1.14.0 using 1 threads (see ?getDTthreads). Latest news: r-datatable.com
**********
用中文运行data.table。软件包只提供英语支持。当在在线搜索帮助时，也要确保检查英语错误信息。这个可以通过查看软件包源文件中的po/R-zh_CN.po和po/zh_CN.po文件获得，这个文件可以并排找到母语和英语错误信息。
**********
**********
This installation of data.table has not detected OpenMP support. It should still work but in single-threaded mode.
This is a Mac. Please read https://mac.r-project.org/openmp/. Please engage with Apple and ask them for support. Check r-datatable.com for updates, and our Mac instructions here: https://github.com/Rdatatable/data.table/wiki/Installation. After several years of many reports of installation problems on Mac, it's time to gingerly point out that there have been no similar problems on Windows or Linux.
**********载入程辑包：‘data.table’The following objects are masked from ‘package:reshape2’:dcast, melt> rating_critic=setDT(movie_ratings[colnames(movie_ratings)[6]],keep.rownames=TRUE)[]
> names(rating_critic)=c('title','rating')
> View(rating_critic)
> titles_na_critic=rating_critic$title[is.na(rating_critic$rating)]
> titles_na_critic
[1] "Just My Luck" "Lady in the Water" "The Night Listener"
> ratings_t=ratings[ratings$title %in% titles_na_critic,]
> View(ratings_t)
> x=(setDT(data.frame(sim_users[,6]),keep.rownames = TRUE)[])
> names(x)=c('critic','similarity')
> ratings_t=merge(x=ratings_t,y=x,by="critic",all.x = TRUE)
> View(ratings_t)
> ratings_t$sim_rating=ratings_t$rating*ratings_t$similarity
> ratings_tcritic title rating similarity sim_rating
1 Claudia Puig Just My Luck 3.0 0.8934051 2.6802154
2 Claudia Puig The Night Listener 4.5 0.8934051 4.0203232
3 Gene Seymour Lady in the Water 3.0 0.3812464 1.1437393
4 Gene Seymour Just My Luck 1.5 0.3812464 0.5718696
5 Gene Seymour The Night Listener 3.0 0.3812464 1.1437393
6 Jack Matthews Lady in the Water 3.0 0.6628490 1.9885469
7 Jack Matthews The Night Listener 3.0 0.6628490 1.9885469
8 Lisa Rose Lady in the Water 2.5 0.9912407 2.4781018
9 Lisa Rose Just My Luck 3.0 0.9912407 2.9737221
10 Lisa Rose The Night Listener 3.0 0.9912407 2.9737221
11 Mick LaSalle Lady in the Water 3.0 0.9244735 2.7734204
12 Mick LaSalle Just My Luck 2.0 0.9244735 1.8489469
13 Mick LaSalle The Night Listener 3.0 0.9244735 2.7734204
> install.packages("dply")
Warning in install.packages :package ‘dply’ is not available for this version of RA version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages
> install.packages("dplyr")
试开URL’https://cran.rstudio.com/bin/macosx/contrib/4.0/dplyr_1.0.5.tgz'
Content type 'application/x-gzip' length 1251016 bytes (1.2 MB)
==================================================
downloaded 1.2 MBThe downloaded binary packages are in/var/folders/2w/tt1p_4td3yq9xlbl7c2t4jn00000gn/T//Rtmp8jyvFD/downloaded_packages
> library(dplyr)载入程辑包：‘dplyr’The following objects are masked from ‘package:data.table’:between, first, lastThe following objects are masked from ‘package:stats’:filter, lagThe following objects are masked from ‘package:base’:intersect, setdiff, setequal, union> result = ratings_t %>% group_by(title) %>% summarise(sum(sim_rating)/sum(similarity))
> result
# A tibble: 3 x 2title `sum(sim_rating)/sum(similarity)`<chr> <dbl>
1 Just My Luck 2.53
2 Lady in the Water 2.83
3 The Night Listener 3.35
> mean(rating_critic$rating,na.rm = T)
[1] 3.166667
小型推荐系统函数
generateRecommendation <- function(userId){
rating_critic=setDT(movie_ratings[colnames(movie_ratings)[userId]],keep.rownames=TRUE)[]
names(rating_critic)=c('title','rating')
titles_na_critic=rating_critic$title[is.na(rating_critic$rating)]
ratings_t=ratings[ratings$title %in% titles_na_critic,]
#每个用户添加一个新的相似变量
x=(setDT(data.frame(sim_users[,6]),keep.rownames = TRUE)[])
names(x)=c('critic','similarity')
ratings_t=merge(x=ratings_t,y=x,by="critic",all.x = TRUE)
#多个相似值
ratings_t$sim_rating=ratings_t$rating*ratings_t$similarity
#预测费评级的电影名称
result = ratings_t %>% group_by(title) %>% summarise(sum(sim_rating)/sum(similarity))
return (result)
}
根据不同的用户ID进行不同的电影预测和推荐。
