Author: Vinta
-
Updated
Build a recommender system with Spark: Content-based and Elasticsearch
在本篇文章中，我們以 Candidate Generation 階段常用的方法之一：Content-based recommendation 基於內容的推薦為例，利用 Elasticsearch 的 More Like This query 建立一個 GitHub repositories 的推薦系統，以用戶最近打星過的 repo 作為輸入數據，比對出相似的其他 repo 作為候選物品集。
-
Updated
Spark ML cookbook (Scala)
Scala is the first class citizen language for interacting with Apache Spark, but it's difficult to learn. This article is mostly about Spark ML - the new Spark Machine Learning library which was rewritten in DataFrame-based API.
-
Updated
Spark SQL cookbook (Scala)
Scala is the first class citizen language for interacting with Apache Spark, but it's difficult to learn. This article is mostly about operating DataFrame or Dataset in Spark SQL.
-
Updated
Play with GitHub Archive Dataset on BigQuery
Category: Big Data
Google BigQuery is a web service that lets you do interactive analysis of very massive datasets - analyzing billions of rows in seconds.
-
Updated
-
Updated
Setup Spark, Scala and Maven with Intellij IDEA
Category: Big Data
IntelliJ IDEA supports Scala and Apache Spark perfectly. You're able to browse a complete Spark project built with IntelliJ IDEA on GitHub: https://github.com/vinta/albedo
-
Updated
-
Updated
-
Updated
Observe system metrics, status, and logs on Linux
Category: DevOps
Linux commands that every DevOps engineer or SysAdmin should know.
-
Updated
Build a recommender system with Spark: Implicit ALS
Category: AI, Big Data, Python
在本篇文章中，我們以 Candidate Generation 階段常用的方法之一：Collaborative Filtering 協同過濾演算法為例，利用 Apache Spark 的 ALS (Alternating Least Squares) 模型建立一個 GitHub repositories 的推薦系統，以用戶對 repo 的打星紀錄作為訓練數據，推薦出用戶可能會感興趣的其他 repo 作為候選物品集。