Slack: Build a chat bot with Hubot in JavaScript

Slack: Build a chat bot with Hubot in JavaScript

Hubot is an scriptable chat bot framework created by GitHub. The newer version supports JavaScript (ES6+), no more CoffeeScript!

ref:
https://hubot.github.com/
https://slack.dev/hubot-slack/

Installation

$ npm install -g yo generator-hubot

$ mkdir codetengu-bot
$ cd codetengu-bot
$ yo hubot --adapter=slack

You could find all available adapters here:
https://hubot.github.com/docs/adapters/

Slack Token

The next thing you need is a Slack Bot Token (API Token) which looks like xoxb-xxx for your chat bot app. You could create a Hubot app in your Slack workspace to request a token, for instance https://vintachen.slack.com/apps/A0F7XDU93-hubot.

Otherwise, you could also create an universal Slack app, install it to your workspace. In your app settings, under "Install App" section, you are able to find OAuth Tokens for your chat bot. See https://api.slack.com/apps.

ref:
https://api.slack.com/bot-users

Development

$ HUBOT_SLACK_TOKEN=YOUR_SLACK_BOT_TOKEN \
./bin/hubot --adapter slack

I fork a script named hubot-reload-scripts to help you reload your scripts when developing them.
https://github.com/vinta/hubot-reload-scripts

Hear And Respond Messages

Writing your own script
https://hubot.github.com/docs/scripting/

// scripts/your_script.js
// Description
//   Do your shit
//
// Author:
//   Vinta Chen
//
// Commands:
//   * `restart <service>`* - Restart the service
//
const _ = require('lodash');

module.exports = (robot) => {
  robot.hear(/restart ([a-z0-9_\-]+)/i, (res) => {
    robot.logger.debug(`Received message: ${res.message.rawText}`);
    const [ serviceName ] = res.match.slice(1);
    res.send(`Restarting ${serviceName}`);
    doYourShit();
  });
};

Call Slack APIs

robot.slack.channels.info({'channel': res.message.rawMessage.channel})
  .then(apiRes => {
    const purpose = apiRes.channel.purpose.value;
    const topic = apiRes.channel.topic.value;
    res.send(`purpose: ${purpose}`);
    res.send(`topic: ${topic`);
  })
  .catch(apiErr => {
    robot.logger.error('apiErr', apiErr);
  });

ref:
https://slack.dev/hubot-slack/basic_usage#using-the-slack-web-api
https://api.slack.com/methods

Python with Excel: xlrd, xlsxwriter, and xlutils

Python with Excel: xlrd, xlsxwriter, and xlutils

Libraries

xlsxwriter 的文件寫得比較好

ref:
http://www.python-excel.org/
https://xlsxwriter.readthedocs.org/en/latest/
http://openpyxl.readthedocs.org/en/latest/

Usage

row 是橫排
column 是直排

Default format

import xlsxwriter

workbook = xlsxwriter.Workbook('label_copy.xlsx')

# default cell format
workbook.formats[0].set_font_size(12)
workbook.formats[0].set_text_wrap() # 要加上這個才能正常顯示多行
workbook.formats[0].set_align('vcenter')

ref:
https://xlsxwriter.readthedocs.org/en/latest/format.html

Multiple lines

lines_format = workbook.add_format({
    'align': 'left',
    'font_size': 12,
    'text_wrap': True,
    'valign': 'vcenter',
})

# 或是用 """多行"""
content = 'first line\nsecond line'
worksheet.write(0, 0, content, lines_format)

重點是要加上 text_wrap

ref:
http://stackoverflow.com/questions/15370432/writing-multi-line-strings-into-cells-using-openpyxl

Write to existing excel files

from xlutils.copy import copy as xlutils_copy
import xlrd

rb = xlrd.open_workbook('your_file.xls', formatting_info=True)
wb = xlutils_copy(rb)
ws = wb.get_sheet(0)
ws.write(0, 0, 'Hello World')

ref:
https://stackoverflow.com/questions/2725852/writing-to-existing-workbook-using-xlwt

Examples

ref:
https://xlsxwriter.readthedocs.org/en/latest/examples.html

ipdb: The interactive Python debugger with IPython

ipdb: The interactive Python debugger with IPython

ipdb is an interactive Python Debugger with IPython integration, which features tab completion and syntax highlighting, etc. In layman's terms, ipdb is a better pdb.

ref:
https://github.com/gotcha/ipdb

Usage

$ pip install -U ipdb

ref:
https://pypi.python.org/pypi/ipdb

Add a breakpoint to any place you want to inspect, then run your code.

import ipdb; ipdb.set_trace()

If you use Sublime Text 3, try Python Breakpoints.
https://github.com/obormot/PythonBreakpoints

Useful Commands

Oldest frame is the frame in the stack where your program started; it is the oldest in time; the Newest frame, the other end of the stack, is where Python is executing code and is the current frame of execution.

# help: Print the list of all commands
h

# help: Print help about the certain command
h break

# print: Print the value of the expression
p some_obj
pp some_obj

# Print detailed information about the object
pinfo some_obj
pinfo2 some_obj

# args: Print arguments with their values of the current function
a

# list: List 11 lines of source code around the current line
l

# list: List 11 lines of source code around line 123
l 123

# longlist: List all source code for the current function or frame
ll

# jump: Jump to line 123, skip the execution of anything between
j 123

# args: List all arguments of the current function
a

# step: Execute code line by line, it may jump to another frame when a function call is encountered
s

# next: Execute code line by line, it doesn't enter functions called from the statement being executed
n

# return: Continue execution until the current function returns.
r

# continue: Continue execution, only stop when a breakpoint is encountered
c

# break: List all breakpoints
b

# break: Set a breakpoint at line 123
b 123

# break: Set a breakpoint at line 123 of file.py
b path/to/file.py:123

# break: Set a breakpoint on some_func that will be triggered if some_arg == 0
b some_func, some_arg == 0

# clear: Clear all breakpoints
clear

# where: Print a stack trace
w

# up: Move the current frame one level up in the stack trace
u

# down: Move the current frame one level down in the stack trace
d

# quit: Quit debugging
q

# use ! to run Python code that may conflict with pdb's built-in commands
!r = 123
!r = 123; c = 455

ref:
https://docs.python.org/2/library/pdb.html#debugger-commands
https://docs.python.org/3/library/pdb.html#debugger-commands
https://pymotw.com/2/pdb/
https://pymotw.com/3/pdb/
https://medium.com/instamojo-matters/become-a-pdb-power-user-e3fc4e2774b2

post_mortem

Debugging a failure after a program terminates is called post-mortem debugging.

>>> do_shit(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pdb_post_mortem.py", line 13, in go
    for i in range(self.num_loops):
AttributeError: 'MyObj' object has no attribute 'num_loops'
>>> import ipdb; ipdb.pm()
>>> w

trace

Tracing a program as it runs. In this case, it will enter ipdb when sys.path changes.

import sys

# this function will execute on every line!!!
def trace_sys_path(frame, event, arg):
    if sys.path[0].endswith('/lib'):
        ipdb.set_trace()
    return trace_sys_path

sys.settrace(trace_sys_path)

ref:
https://youtu.be/5XvAVgcbmdY?t=22m51s

Use IPython magic functions in ipdb

Because that ipdb is not a full IPython shell: actually, it provides the same Python Debugger interface as pdb, ipdb lacks many features of IPython, for instance, magic functions. You could use following code to enter a real IPython environment for debugging.

from IPython import embed; embed()

Instead of import ipdb; ipdb.set_trace().

ref:
http://stackoverflow.com/questions/16184487/use-ipython-magic-functions-in-ipdb-shell
https://github.com/gotcha/ipdb/issues/33

MkDocs: Deploy your Markdown documents on GitHub Pages

MkDocs: Deploy your Markdown documents on GitHub Pages

MkDocs is a static site generator that builds modern webpages based on your Markdown documents and a simple YAML file.

ref:
https://www.mkdocs.org/

Here is the website which is generated by MkDocs in this post:
https://awesome-python.com/
https://github.com/vinta/awesome-python

Installation

$ pip install mkdocs

Configuration

in mkdocs.yml

site_name: Awesome Python
site_url: https://awesome-python.com
site_description: A curated list of awesome Python frameworks, libraries and software
site_author: Vinta Chen
repo_name: vinta/awesome-python
repo_url: https://github.com/vinta/awesome-python
theme:
  name: material
  palette:
    primary: red
    accent: pink
extra:
  social:
    - type: github
      link: https://github.com/vinta
    - type: twitter
      link: https://twitter.com/vinta
    - type: linkedin
      link: https://www.linkedin.com/in/vinta
google_analytics:
  - UA-510626-7
  - auto
extra_css:
    - css/extra.css
nav:
  - "Life is short, you need Python.": "index.md"

There are more themes:

in Makefile

site_install:
    pip install -r requirements.txt

site_link:
    ln -sf $(CURDIR)/README.md $(CURDIR)/docs/index.md

site_preview: site_link
    mkdocs serve

site_build: site_link
    mkdocs build

site_deploy: site_link
    mkdocs gh-deploy --clean

Custom Domain for GitHub Pages

in docs/CNAME

awesome-python.com

After deploying your GitHub Page, just pointing your domain to following IPs with DNS A records:

  • 185.199.108.153
  • 185.199.109.153
  • 185.199.110.153
  • 185.199.111.153

ref:
https://help.github.com/articles/setting-up-an-apex-domain/#configuring-a-records-with-your-dns-provider
https://help.github.com/articles/troubleshooting-custom-domains/#https-errors

Automatic Deployment Via Travis CI

You need to

language: python

python:
  - "3.6"

script:
  - cp README.md docs/index.md
  - mkdocs build

deploy:
  provider: pages
  local-dir: site
  skip-cleanup: true
  keep-history: true
  github-token: $GITHUB_TOKEN
  on:
    branch: master

ref:
https://docs.travis-ci.com/user/deployment/pages/

Python decorators

Python decorators

Python 裡的所有東西都是 object
function 也是
所以你可以對 function 做任何跟 object 一樣的事
例如把一個 function 當成參數丟給另一個 function
當然也可以 decorate class

不帶參數的 decorator

第一層 def 接收 func
第二層 def 接收 func 的 *args, **kwargs

通常意義下的 decorator 是把 func(就是 something_1、something_2)丟給 decorator function
做一些額外的動作
然後回傳該 func 原本的 return
並不操作 func 本身

如果要操作 func 本身
例如幫 func 增加一個 attribute
請參考下下面的例子

def func_wrapper(func):
    def arg_wrapper(*args, **kwargs):
        print func.__name__ + ' was called'
        print args
        print kwargs

        return func(*args, **kwargs)

    return arg_wrapper

@func_wrapper
def something_1(name='[name]'):
    print 'something_1: ' + name

@func_wrapper
def something_2(name='[name]'):
    print 'something_2: ' + name

something_1('Tom')
something_2('Cathy')
something_1()
something_2()

ref:
http://www.dotblogs.com.tw/rickyteng/archive/2013/11/06/126852.aspx

帶參數的 decorator

需要傳參數給 decorator 時
要多 wrap 一次
其實就是在原本的 decorator function 的外面再多加一個 def 來接收參數

那個 model_class 就是傳給 decorator 的參數

# 跟不帶參數的 decorator 相比多了一層,用來接收 decorator 的參數
def can_access_item_required(model_class):
    def func_wrapper(func):
        def arg_wrapper(*args, **kwargs):
            request = args[0]
            item = model_class.objects.get(pk=kwargs['pk'])

            if not model_class.objects.can_access_item_by(item, request.user):
                raise PermissionDenied()

            return func(*args, **kwargs)

        return arg_wrapper

    return func_wrapper

@can_access_item_required(Label)
def label_update(request, pk):
    pass

# equals to

label_update = can_access_item_required(Artist)(label_update)

ref:
https://www.python.org/dev/peps/pep-0318/#current-syntax

decorator 修改 func 本身

不需要封裝什麼
單純地把 func 丟給另一個 function
然後再 return 那個 func 即可

in views.py

def intro_middleware_decorator(func):
    func.enable_intro_middleware = False
    return func

@intro_middleware_decorator
def intro_1(request):
    return render(request, 'dps/intro/1.html')

in middleware.py

class IntroMiddleware(object):
    """
    如果 user 沒有完成 intro 步驟,則轉址到 /intro/1/
    """
    def __init__(self):
        self.enable = True
    def process_view(self, request, view_func, view_args, view_kwargs):
        """
        要在 intro 系列的 view 加上 disable_intro_middleware_decorator
        避免無限迴圈
        """
        self.enable = getattr(view_func, 'enable_intro_middleware', True)
    def process_response(self, request, response):
        if self.enable:
            resolver = request.resolver_match
            namespace = getattr(resolver, 'namespace', None)
            # 避免 admin, login 之類的 view 也都被 redirect
            if namespace == 'dps':
                if request.user.is_authenticated():
                    profile = request.user.profile
                    if (profile.identity == ProfileIdentity.PHANTOM) and (not profile.is_ready):
                        return redirect('dps:intro-one')
                else:
                    return redirect('auth:login')
        return response

decorator in a Class

class SVAPIListView(object):

    @staticmethod
    def filter_last_modified_decorator(func):
        """
        用來讓 client 同步資料
        ?last_modified=2015-02-04T15:16:22&is_deleted=true
        可以拿到某個時間點之後,新增、修改、刪除的項目
        必須明確地指定 is_deleted=true 才會包含被刪除的項目
        """

        def _add_filters_for_syncing(self, *args, **kwargs):
            queryset = func(self, *args, **kwargs)

            last_modified = self.get_last_modified()
            if last_modified:
                queryset = queryset.filter(last_modified__gte=last_modified)

                is_deleted = self.get_is_deleted()
                target_user = self.get_target_user()
                if (self.request.user == target_user) and (is_deleted):
                    # 只有用戶本人才能拿到被刪除的 items
                    pass
                else:
                    queryset = queryset.filter(enable=True)
            else:
                queryset = queryset.filter(enable=True)

            return queryset

        return _add_filters_for_syncing

class UserAlbumList(SVAPIListView):
    serializer_class = api_serializers.AlbumListSerializer

    @UserIDAsUsernameMixin.filter_last_modified_decorator
    def get_queryset(self, profile_user):
        queryset = MusicAlbum.objects.filter(user=profile_user)
        queryset = queryset.select_related('user', 'user__profile')

        ordering = self.request.GET.get('ordering')
        if ordering == 'like_count':
            queryset = queryset.order_by('-like_count')
        else:
            queryset = queryset.order_by('-id')

        return queryset

    def get(self, request, user_id, *args, **kwargs):
        profile_user = self.get_target_user()
        queryset = self.get_queryset(profile_user=profile_user)
        data = self.get_serializer_data(queryset)

        return Response(data)

ref:
http://stackoverflow.com/questions/1263451/python-decorators-in-classes

多個 decorator 時的執行順序

如果有多個 decorator 裝飾同一個 function / class
執行的順序是由下往上的
會先執行 @decorator_1

@decorator_3
@decorator_2
@decorator_1
def function():
    pass

Class-based decorator

class RecorderDecorator(object):

    __slots__ = ['recorder', 'msg_template', 'subject', 'action']

    recorder = Recorder()

    def __init__(self, msg_template, subject=None, action=None):
        self.msg_template = msg_template
        self.subject = subject
        self.action = action

class model_recorder(RecorderDecorator):
    """
    用於 model 的 method
    接受一個 string template,會自動使用 method parameters 作為 template context
    會使用 method name 作為 action,model name 作為 subject
    會在裝飾的 method 執行完之後執行這個 decorator

    用法可以參考 Song.play()

    @recorder_decorators.model_recorder('user: {user.id}, ip: {from_ip}, full: {is_full}, embed: {is_embed}')
    def play(self, user, from_ip, is_full=False, is_embed=False, from_playlist=None):
        pass

    則 log message 會是 [song:play] user: 123, ip: 59.120.12.57, full: True, embed: False
    """

    def __call__(self, func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            func_result = func(*args, **kwargs)
            func_kwargs = inspect.getcallargs(func, *args, **kwargs)

            model_instance = func_kwargs.get('self')

            if getattr(model_instance, 'id'):
                new_msg_template = ', '.join(('item: {self.id}', self.msg_template))
            else:
                new_msg_template = self.msg_template

            if not self.subject:
                model_name = type_utils.item_type(model_instance)
                new_subject = model_name
            else:
                new_subject = self.subject

            if not self.action:
                new_action = func.__name__
            else:
                new_action = self.action

            msg = new_msg_template.format(**func_kwargs)
            self.recorder.write(new_subject, new_action, msg)

            return func_result

        return wrapper

class Song(models.Model):

    @model_recorder('user: {user.id}, ip: {from_ip}, full: {is_full}, embed: {is_embed}')
    def play(self, user, from_ip, is_full=False, is_embed=False, from_playlist=None):
        pass

ref:
https://stackoverflow.com/questions/9416947/python-class-based-decorator-with-parameters-that-can-decorate-a-method-or-a-fun