Author: Vinta
-
Updated
Import MySQL database but ignore specific tables
You could just exclude tables you do not need.
-
Updated
ngrok: Share your localhost services with friends
Category: DevOps
Generate a https://xxx.ngrok.com URL for letting other people access your localhost services.
-
Updated
Dot notation (obj.x.y.z) for nested objects and dictionaries in Python
Category: Python
Simple implementations of nested_getattr(obj, attr, default), nested_setattr(obj, attr, val) and nested_dict_get(dictionary, dotted_key).
-
Updated
Remotely debug a Python app inside a Docker container in Visual Studio Code
Category: Python, Web Development
In this article, we are going to debug a Flask app inside a local Docker container through VS Code's fancy debugger, and simultaneously we are still able to leverage Flask's auto-reloading mechanism. It should apply to other Python apps.
-
Updated
MongoDB cookbook: Indexes
Category: Database
The order of fields in an index matters, you must consider Index Cardinality and Selectivity. Instead, the order of fields in a find() query doesn't affect whether it can use an index or not.
-
Updated
MongoDB cookbook: Queries and Aggregations
Category: Database, JavaScript, Python
Frequently accessed items are cached in memory, so that MongoDB can provide optimal response time.
-
Updated
Run a Celery task at a specific time
Category: Python, Web Development
You are able to run any Celery task at a specific time through eta (means "Estimated Time of Arrival") parameter.
-
Updated
Timezone in Python: Offset-naive and Offset-aware datetimes
Category: Python
You should always store datetimes in UTC and convert to proper timezone on display.
-
Updated
Build a recommender system with Spark: Logistic Regression
在本篇文章中，我們將以 Ranking 階段常用的方法之一：Logistic Regression 邏輯迴歸為例，利用 Apache Spark 的 Logistic Regression 模型建立一個 GitHub repositories 的推薦系統，以用戶對 repo 的打星紀錄和用戶與 repo 的各項屬性做為特徵，預測出用戶會不會打星某個 repo（分類問題）。最後訓練出來的模型就可以做為我們的推薦系統的 Ranking 模組。不過因為 LR 是線性模型，所以通常需要大量的 Feature Engineering 來習得非線性關係。所以這篇文章的重點是 Spark ML 的 Pipeline 機制和特徵工程，不會在演算法的部分著墨太多。
-
Updated
Spark best practices
Category: Big Data
A Spark application is a set of processes running on a cluster, all these processes are coordinated by the driver program. The driver program is 1) the process where the main() method of your program runs, 2) the process running the code that creates a SparkSession, RDDs, DataFrames, and stages up or sends off transformations and actions.