I Failed the Turing Test

All posts in JavaScript

AWS Lambda Cookbook

AWS Lambda Cookbook

Posted on Author VintaPosted in DevOps, JavaScript, Web Development

AWS Lambda is an event-driven service that lets you upload your code and run it on-demand without having your own servers.

ref:
http://aws.amazon.com/lambda/
http://docs.aws.amazon.com/lambda/latest/dg/limits.html

API Gateway 就是 URL routing
Lambda 則是那些 route (endpoint) 對應的 handler
如果你是用 event 或 schedule 的方式呼叫 Lambda function 的話
可以不用 API Gateway

AWS Lambda 有兩種 invocation type
一是 RequestResponse，同步（例如綁定 API Gateway 和你在 Lambda Management Console 操作的時候）
二是 Event，非同步

Runtimes

AWS Lambda supports the following runtime versions:

  • nodejs (Node v0.10)
  • nodejs4.3
  • java
  • python

ref:
http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html

Node.js

const aws = require('aws-sdk');

exports.handle = (event, context, callback) => {
  doYourShit();
  callback(null, 'DONE');
};

每個 Lambda function 會接收三個參數 event、context 和 callback

event 是從外部的 input
可能是來自 S3 object event、DynamoDB stream 或是由 API Gateway POST 進來的 JSON payload

context 則會包含當前這個 Lambda fuction 的一些 metadata
例如 context.getRemainingTimeInMillis()

callback 參數只有 Node.js runtime v4.3 才支援
v0.10 的話得用 context.succeed()、context.fail() 和 context.done()
不過誰他媽還在用 Node.js v0.10

ref:
http://docs.aws.amazon.com/lambda/latest/dg/programming-model.html
http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html
http://docs.aws.amazon.com/lambda/latest/dg/best-practices.html

Calling another Lambda function in a Lambda function.

要注意的是
你的 Lambda function 的 role 得要有 invoke 其他 Lambda function 的權限才行

const util = require('util');

const aws = require('aws-sdk');

const params = {
  FunctionName: 'LambdaBaku_syncIssue',
  InvocationType: 'Event', // means asynchronous execution
  Payload: JSON.stringify({ issue_number: curatedIssue.number }),
};

lambda.invoke(params, (err, data) => {
  if (err) {
    console.log('FAIL', params);
    console.log(util.inspect(err));
  } else {
    console.log(data);
  }
});

ref:
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
http://stackoverflow.com/questions/31714788/can-an-aws-lambda-function-call-another

完整的程式碼放在 GitHub 上
https://github.com/CodeTengu/lambdabaku

Users and Roles

如果你是用 apex 來管理 Lambda functions 的話
確保你用的 AWS credential (User) 擁有 AWSLambdaFullAccess 和 AWSLambdaRole 這兩個 permissions

以 project 為單位建立 Role 即可
例如 lambdabaku_role
你可以在 IAM Management Console 找到那些你建立的 roles
基本上用 Basic execution role 就夠了
反正之後可以隨時修改 Role 的 permission / policy
Lambda function 屬於哪個 VPC 是額外指定的
跟 Role 沒有關係
也就是說你用 Basic execution role 還是可以支援 VPC

如果想在 Lambda function 裡存取 DynamoDB
要記得在 Role 裡新增對應的設定

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Stmt1428341300017",
            "Effect": "Allow",
            "Action": [
                "dynamodb:*"
            ],
            "Resource": [
                "arn:aws:dynamodb:ap-northeast-1:004615714446:table/CodeTengu_Preference",
                "arn:aws:dynamodb:ap-northeast-1:004615714446:table/CodeTengu_WeeklyIssue",
                "arn:aws:dynamodb:ap-northeast-1:004615714446:table/CodeTengu_WeeklyPost"
            ]
        }
    ]
}

Scheduled Events

ref:
http://docs.aws.amazon.com/lambda/latest/dg/with-scheduled-events.html

API Gateway

單純一點的話
Security 可以選 Open with access key
然後到 API Gateway 介面的 API Keys 底下新增一組 access key
然後分配一個 API stage 給它

使用的時候在 HTTP header 加上 x-api-key: YOUR_API_KEY 即可

ref:
http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-api-keys.html

Related Projects

ref:
https://github.com/serverless/serverless
https://github.com/apex/apex
https://github.com/claudiajs/claudia
https://github.com/garnaat/kappa
https://github.com/Miserlou/Zappa
https://github.com/nficano/python-lambda

淺析 serverless 架構與實作
http://abalone0204.github.io/2016/05/22/serverless-simple-crud/

Deploy Lambda Functions via apex

$ curl https://raw.githubusercontent.com/apex/apex/master/install.sh | sh

$ apex deploy
$ apex invoke syncPublishedIssues --logs
$ echo -n '{"issue_number": 43}' | apex invoke syncIssue --logs

ref:
https://github.com/apex/apex
http://apex.run/

Slack: Build a chat bot with Hubot in JavaScript

Slack: Build a chat bot with Hubot in JavaScript

Posted on Author VintaPosted in DevOps, JavaScript

Hubot is an scriptable chat bot framework created by GitHub. The newer version supports JavaScript (ES6+), no more CoffeeScript!

ref:
https://hubot.github.com/
https://slack.dev/hubot-slack/

Installation

$ npm install -g yo generator-hubot

$ mkdir codetengu-bot
$ cd codetengu-bot
$ yo hubot --adapter=slack

You could find all available adapters here:
https://hubot.github.com/docs/adapters/

Slack Token

The next thing you need is a Slack Bot Token (API Token) which looks like xoxb-xxx for your chat bot app. You could create a Hubot app in your Slack workspace to request a token, for instance https://vintachen.slack.com/apps/A0F7XDU93-hubot.

Otherwise, you could also create an universal Slack app, install it to your workspace. In your app settings, under "Install App" section, you are able to find OAuth Tokens for your chat bot. See https://api.slack.com/apps.

ref:
https://api.slack.com/bot-users

Development

$ HUBOT_SLACK_TOKEN=YOUR_SLACK_BOT_TOKEN 
./bin/hubot --adapter slack

I fork a script named hubot-reload-scripts to help you reload your scripts when developing them.
https://github.com/vinta/hubot-reload-scripts

Hear And Respond Messages

Writing your own script
https://hubot.github.com/docs/scripting/

// scripts/your_script.js
// Description
//   Do your shit
//
// Author:
//   Vinta Chen
//
// Commands:
//   * restart <service>* - Restart the service
//
const _ = require('lodash');

module.exports = (robot) => {
  robot.hear(/restart ([a-z0-9_-]+)/i, (res) => {
    robot.logger.debug(Received message: ${res.message.rawText});
    const [ serviceName ] = res.match.slice(1);
    res.send(Restarting ${serviceName});
    doYourShit();
  });
};

Call Slack APIs

robot.slack.channels.info({'channel': res.message.rawMessage.channel})
  .then(apiRes => {
    const purpose = apiRes.channel.purpose.value;
    const topic = apiRes.channel.topic.value;
    res.send(purpose: ${purpose});
    res.send(topic: ${topic);
  })
  .catch(apiErr => {
    robot.logger.error('apiErr', apiErr);
  });

ref:
https://slack.dev/hubot-slack/basic_usage#using-the-slack-web-api
https://api.slack.com/methods

Parse datetime in Python and JavaScript

Parse datetime in Python and JavaScript

Posted on Author VintaPosted in JavaScript, Python, Web Development

Python

I recommend dateutil.

ref:
https://dateutil.readthedocs.org/en/latest/

import datetime
from dateutil import parser as dateutil_parser

>>> dateutil_parser.parse('2014-12-24T16:15:16')
datetime.datetime(2014, 12, 24, 16, 15, 16)

>>> datetime_obj = datetime.datetime.strptime('2014-12-24T16:15:16', '%Y-%m-%dT%H:%M:%S')
datetime.datetime(2014, 12, 24, 16, 15, 16)

>>> datetime_obj = datetime.datetime.strptime('201408282300', '%Y%m%d%H%M')
datetime.datetime(2014, 8, 28, 23, 0)

>>> datetime_obj.strftime('%Y-%m-%d %H:%M')

strftime >> datetime -> str
strptime >> str --> datetime

ref:
https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior

Django Template

class DriverInfoForm(forms.ModelForm):
    service_time_start = forms.TimeField(
        widget=forms.TimeInput(format='%H:%M'),
        input_formats=['%H:%M', ]
    )

@register.filter
def str_to_time(time_str, output_format):
    """
    把字串轉成 datetime obj
    再依據 output_format 輸出

    {{ news.modified_at|str_to_time:"%Y/%m/%d %H:%M" }}
    """

    from dateutil import parser

    datetime_obj = parser.parse(time_str, fuzzy=True)

    return datetime_obj.strftime(output_format)
日期：{{ withdraw.presented_at|date:"%Y 年 %n 月" }}
聯絡時間：{{ driver.service_time_start|date:"H:i" }} - {{ driver.service_time_end|date:"H:i" }}

要注意的是，Django 似乎不能 parse AM / PM，所以儘量用 24 小時制。

ref:
https://docs.djangoproject.com/en/dev/ref/templates/builtins/#date

JavaScript

I recommend moment.js.

ref:
https://momentjs.com/

var today = new Date().toISOString().slice(0, 10);
// 2016-05-11

var t1 = new Date('2016-05-02T03:00:00.000+01:00');
// Mon May 02 2016 10:00:00 GMT+0800 (CST)

var t1_timestamp_ms = t1.getTime();
// 要注意的是 JavaScript 的 getTime() 的單位是 ms
// 1462154400000

var t1_timestamp = t1.getTime() / 1000;
// 1462154400

var t2 = new Date(1485596172 * 1000);
// Sat Jan 28 2017 17:36:12 GMT+0800 (CST)

var t3 = moment('201408292300', 'YYYYMMDDHHmm');

var t3 = moment('2018-02-02')
var timestamp = time.unix()
// 單位是 second
// 1518192000

ref:
https://stackoverflow.com/questions/3552461/how-to-format-a-javascript-date

nvm: Node.js Version Manager

nvm: Node.js Version Manager

Posted on Author VintaPosted in JavaScript, Web Development

A simple Node.js version manager.

Install nvm

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash

ref:
https://nodejs.org/en/download/current
https://github.com/nvm-sh/nvm

Install Node.js

You could also simply run brew install node if you don't really care about what version you're installing.

# list available Node.js versions
nvm ls-remote

# list installed Node.js versions
nvm ls

# install the latest LTS version
nvm install --lts

nvm install v22.15.0 && \
nvm use v22.15.0 && \
nvm alias default v22.15.0

npm i -g [email protected]
npm i -g [email protected]

nvm uninstall v16.13.2

ref:
https://nodejs.org/en/

Install Node.js Packages

# install the package globally
npm install -g pangu

# install the package in the current folder
# which generate package.json in the same folder
npm install pangu

Override Dependency Version

Use overrides in package.json:

{
  "overrides": {
    "foo": {
      ".": "1.0.0",
      "bar": "1.0.0"
    }
  }
}

ref:
https://docs.npmjs.com/cli/v8/configuring-npm/package-json#overrides

Pin Package Versions

npm install -g npm-check-updates

npm-check-updates -u --removeRange

ref:
https://github.com/raineorshine/npm-check-updates