python

coverage.py: Python code coverage

2014-03-192019-10-22VintaPython

coveralls.io 跑 python project 的覆蓋率就是用 coverage.py
nose 也是用 coverage.py
基本上 Python 社群幾乎都是用這個

ref:
http://nedbatchelder.com/code/coverage/
http://www.cnblogs.com/coderzh/archive/2009/12/01/1614874.html

Install

$ pip install coverage

Configuration

in .coveragerc

[run]
branch = True
source = email_confirm_la
omit =
    email_confirm_la/migrations/*
    email_confirm_la/south_migrations/*
    email_confirm_la/tests/*

[run] 底下的 include 和 omit 是用來指定 filename pattern
而 source 是用來指定 package 或目錄

omit 是「忽略」的意思

ref:
http://nedbatchelder.com/code/coverage/config.html

Usage

# 會自動套用當前目錄下的 .coveragerc
$ coverage run setup.py test
$ coverage report
$ coverage html

# 默認會一併測試所有包含第三方 lib 在內的 Python 程式碼（如果你沒有寫 .coveragerc 的話）
$ coverage run test.py

# 你可以用 --source 指定只測試某個 packages 或目錄
$ coverage run --source=haul setup.py test
$ coverage run --source=haul tests/test.py
$ coverage run --source=. manage.py test

# show report in terminal
$ coverage report

# generate html report in the same dir
$ coverage html

ref:
http://nedbatchelder.com/code/coverage/cmd.html
http://nedbatchelder.com/code/coverage/source.html

Usage with nose

$ pip install nose nose-cov

# output terminal
$ nosetests --with-cov --cov haul tests/

# output html
$ nosetests --with-cov --cov-report html --cov haul tests/

Travis-CI for Python: a .travis.yml example

2014-02-232019-10-22VintaPython, Web Development

Configuration

in .travis.yml

language: python

# 多個 env 要寫在同一行
env:
  - DPS_ENV=test OTHER_ENV=whatever
  - DPS_ENV=development OTHER_ENV=whatever

python:
  - "2.7"
  - "3.3"

before_install:
  - sudo apt-get update -qq
  - sudo apt-get install -qq libxml2-dev libxslt1-dev

install:
  - pip install -r requirements_test.txt --use-mirrors

script:
  - coverage run --source=haul setup.py test

notifications:
  email:
    - [email protected]

after_success:
  - coveralls

要注意的是多個 environment variables 要寫在同一行
否則會被當成多個環境
每個環境都會跑一次 CI

ref:
http://docs.travis-ci.com/user/ci-environment/
http://docs.travis-ci.com/user/build-configuration/
http://docs.travis-ci.com/user/languages/python/
http://docs.travis-ci.com/user/database-setup/

in .coveralls.yml

如果是 public repo 的話，不需要特別設定什麼

ref:
https://coveralls.io/docs/python
https://github.com/coagulant/coveralls-python

How to Set a Variable in Django Templates?

2014-02-172019-10-22VintaPython, Web Development

Creating a custom template tag for assigning variables in Django template.

in app/templatetags/common.py

from django import template
register = template.Library()

class SetVarNode(template.Node):

    def __init__(self, var_name, var_value):
        self.var_name = var_name
        self.var_value = var_value

    def render(self, context):
        try:
            value = template.Variable(self.var_value).resolve(context)
        except template.VariableDoesNotExist:
            value = ""
        context[self.var_name] = value

        return u""

@register.tag(name='set')
def set_var(parser, token):
    """
    {% set some_var = '123' %}
    """
    parts = token.split_contents()
    if len(parts) < 4:
        raise template.TemplateSyntaxError("'set' tag must be of the form: {% set <var_name> = <var_value> %}")

    return SetVarNode(parts[1], parts[3])

in app/templates/xxx.html

{% load common %}

{% if work and is_public_work %}
    {% ifequal link_class 'info_box pull-right' %}
        {% set ga_label = 'card' %}
    {% else %}
        {% set ga_label = ga_label_detail %}
    {% endifequal %}
    <a class="{{ ga_label }}">123</a>
{% endif %}

lxml: Parse HTML or XML with XPath in Python

2014-02-142019-10-22VintaPython, Web Development

lxml is the most feature-rich and easy-to-use library for processing XML and HTML in Python.

import lxml
import lxml.html

# HTML
html_text = 'some html string'
doc = lxml.html.fromstring(html_text)
image_urls = doc.xpath('//img/@src')

url = 'http://www.gotceleb.com/rosie-huntington-whiteley-vogue-brazil-magazine-april-2013-2013-03-29.html'
doc = lxml.html.parse(url)
image_urls = doc.xpath('//img/@src')

# XML
xml_text = 'some xml string'
doc = lxml.etree.fromstring(xml_text)
image_urls = doc.xpath('//img/@src')

Submit your Python Library to PyPI (Python Package Index)

2014-02-142019-10-22VintaPython

Packaging Python Projects
https://packaging.python.org/tutorials/packaging-projects/

A Human's Ultimate Guide to setup.py
https://github.com/kennethreitz/setup.py

Sharing Your Labor of Love: PyPI Quick and Dirty
https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/

setup.py Example

#!/usr/bin/env python
# coding: utf-8

import os
import sys

from setuptools import find_packages
from setuptools import setup

def get_version():
    code = None
    path = os.path.join(
        os.path.dirname(os.path.abspath(__file__)),
        'jokekappa',
        '__init__.py',
    )
    with open(path) as f:
        for line in f:
            if line.startswith('__version__'):
                code = line[len('__version__ = '):]
                break
    return eval(code)

if sys.argv[-1] == 'wheel':
    os.system('rm -rf dist/*')
    os.system('pip install wheel')
    os.system('python setup.py bdist_wheel')
    sys.exit(0)

if sys.argv[-1] == 'publish':
    os.system('python setup.py wheel')
    os.system('pip install twine')
    os.system('twine upload dist/*')
    sys.exit(0)

long_description = open('README.rst').read()

requirements_lines = [line.strip() for line in open('requirements.txt').readlines()]
install_requires = list(filter(None, requirements_lines))

setup(
    name='jokekappa',
    version=get_version(),
    url='https://github.com/CodeTengu/jokekappa',
    description='A library for delivering one-line programming jokes.',
    long_description=long_description,
    keywords='jokes programming-jokes',
    author='Vinta Chen',
    author_email='[email protected]',
    license='MIT',
    install_requires=install_requires,
    scripts=['scripts/jokekappa', ],
    packages=find_packages(exclude=('tests', )),
    include_package_data=True,
    test_suite='tests',
    classifiers=(
        'Development Status :: 3 - Alpha',
        'Environment :: Console',
        'Environment :: Web Environment',
        'Intended Audience :: Developers',
        'License :: OSI Approved :: MIT License',
        'Natural Language :: English',
        'Natural Language :: Chinese (Simplified)',
        'Natural Language :: Chinese (Traditional)',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.3',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
        'Programming Language :: Python :: 3.6',
        'Topic :: Software Development :: Libraries :: Python Modules',
        'Topic :: Utilities',
    ),
)

ref:
https://github.com/pypa/twine/blob/master/setup.py
https://pypi.python.org/pypi?%3Aaction=list_classifiers

If you have exactly one Python file, for instance: pangu.py, use py_modules instead of packages in setup.py:

py_modules=['pangu', ],

ref:
https://docs.python.org/2/distutils/examples.html#pure-python-distribution-by-package

setuptools.find_packages() finds *.py files only, you're able to set include_package_data=True for including requirements.txt or *.html files. Additionally, you may specify extra files to include or exclude in MANIFEST.in:

recursive-include jokekappa/jokes *.json
include LICENSE
include README.rst
include requirements.txt

.pypirc Example

in ~/.pypirc

[pypi]
username=YOUR_USERNAME
password=YOUR_PASSWORD

ref:
https://pypi.org/account/register/

Commands

You need to register an account on Python Package Index to submit your package.

$ pip install -U pip setuptools wheel twine

$ python setup.py --help
$ python setup.py --help-commands 

# install locally
$ python setup.py install

# link your package files to site-packages, for debug
$ python setup.py develop

# you must specify `test_suite` in `setup.py`
$ python setup.py test

# pack in wheel format
$ python setup.py bdist_wheel

# pack in the traditional way
$ python setup.py sdist

# upload
$ twine upload dist/*

ref:
https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives
https://github.com/pypa/twine