Set up black formatter
Add black formatter and a corresponding pre-commit hook.
Issue: HIC-161
Change-Id: I8aed563114a6350cc5803f0c5232d3a20d6d6eb6
diff --git a/README.md b/README.md
index d9c48ed..72badc1 100644
--- a/README.md
+++ b/README.md
@@ -7,6 +7,11 @@
## Setup
+Python 3.6 is the only supported python version for the server code. Use this
+version for development. It is the default version in Ubuntu 18.04, if you
+run another OS, it is still possible to get python 3.6 (see for example
+https://askubuntu.com/a/865569).
+
Make sure you have installed `python3`, `virtualenv` and `libffi-dev`.
$ sudo apt install python3 virtualenv libffi-dev
@@ -15,10 +20,20 @@
$ git clone ssh://$USER@review.fairphone.software:29418/tools/hiccup/hiccup-server
$ cd hiccup-server
- $ virtualenv -p python3 .venv/hiccupenv
+ $ virtualenv -p python3.6 .venv/hiccupenv
$ source .venv/hiccupenv/bin/activate
(hiccupenv) $ pip install -r requirements.txt
+When using a virtualenv with pyenv (e.g. to get python3.6 on Ubuntu 16.04),
+the python executable needs to be explicitly named to make tox work (see
+https://github.com/pyenv/pyenv-virtualenv/issues/202#issuecomment-284728205).
+
+ pyenv virtualenv -p python3.6 <installed-python-version> hiccupenv
+
+i.e., because I have compiled python 3.6.6:
+
+ pyenv virtualenv -p python3.6 3.6.6 hiccupenv
+
By default Django will use a SQLite3 database (`db.sqlite3` in the base directory).
#### Using PostgreSQL Server
@@ -146,17 +161,33 @@
(hiccupenv) $ tox
-Before committing a patchset, you are kindly asked to run the linting tools.
-The flake8 tool can be run automatically for you with a (strict) git
-pre-commit hook:
-
- (hiccupenv) $ flake8 --install-hook git
- (hiccupenv) $ git config --bool flake8.strict true
-
-Also, running flake8 on only the diff with upstream:
+To run flake8 on only the diff with upstream:
(hiccupenv) $ git diff origin/master ./**/*py | flake8 --diff
+We use the [black formatter](https://github.com/ambv/black) to check the
+format of the code. To format a single file in place run:
+
+ (hiccupenv) $ black file.py
+
+To run the formatter over all python files run:
+
+ (hiccupenv) $ git ls-files '*.py' | xargs black
+
+Before committing a patchset, you are kindly asked to run the linting tools
+and format the code. For both linter and formatter, a git pre-commit hook
+can be set up. To activate, copy the pre-commit script that calls the tox
+command for the pre-commit hooks to the `.git/hooks` folder and make it
+executable:
+
+ (hiccupenv) $ cp tools/hooks/pre-commit .git/hooks/pre-commit
+ (hiccupenv) $ chmod +x .git/hooks/pre-commit
+
+To prevent commits when the flake8 check fails the *strict* option can be
+set to true:
+
+ (hiccupenv) $ git config --bool flake8.strict true
+
There is also a lint check included with tox (`tox -e linters`) but it is very
noisy at the moment since the codebase is not clean yet. Since you are already
validating the changes you are making with the git pre-commit hook, you are
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 0000000..534d054
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,5 @@
+# Configuration for Black (https://github.com/ambv/black)
+
+[tool.black]
+line-length = 80
+
diff --git a/requirements-dev.txt b/requirements-dev.txt
index 762e6f3..44621c2 100644
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -1,3 +1,4 @@
-rrequirements-dev-flake8.txt
-rrequirements-dev-pylint.txt
tox
+black
diff --git a/tools/hooks/pre-commit b/tools/hooks/pre-commit
new file mode 100644
index 0000000..3a8b557
--- /dev/null
+++ b/tools/hooks/pre-commit
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+tox -e pre-commit-hooks
\ No newline at end of file
diff --git a/tools/hooks/pre-commit.d/pre-commit-black.sh b/tools/hooks/pre-commit.d/pre-commit-black.sh
new file mode 100755
index 0000000..2d6ff86
--- /dev/null
+++ b/tools/hooks/pre-commit.d/pre-commit-black.sh
@@ -0,0 +1,84 @@
+#!/bin/bash
+
+# Copyright 2017-2018 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Adapted from
+# https://github.com/google/yapf/blob/d4933e857e85d2e685542d19d09282f59c12c6b4/plugins/pre-commit.sh
+# (using black formatter instead of yapf)
+
+# INSTALLING: This hook is only run by tox and does not need to be installed.
+#
+# Git pre-commit.d hook to check staged Python files for formatting issues with
+# black.
+#
+# This requires that black is installed and runnable in the environment running
+# the pre-commit.d hook.
+#
+# When running, this first checks for unstaged changes to staged files, and if
+# there are any, it will exit with an error. Files with unstaged changes will be
+# printed.
+#
+# If all staged files have no unstaged changes, it will run black against them,
+# leaving the formatting changes unstaged. Changed files will be printed.
+#
+# BUGS: This does not leave staged changes alone when used with the -a flag to
+# git commit, due to the fact that git stages ALL unstaged files when that flag
+# is used.
+
+# Find all staged Python files, and exit early if there aren't any.
+PYTHON_FILES=(`git diff --name-only --cached --diff-filter=AM | \
+ grep --color=never '.py$'`)
+if [ ! "$PYTHON_FILES" ]; then
+ exit 0
+fi
+
+# Verify that black is installed; if not, warn and exit.
+if [ -z $(which black) ]; then
+ echo 'black not on path; can not format. Please run using tox:'
+ echo ' tox -e pre-commit-hooks'
+ exit 2
+fi
+
+# Check for unstaged changes to files in the index.
+CHANGED_FILES=(`git diff --name-only ${PYTHON_FILES[@]}`)
+if [ "$CHANGED_FILES" ]; then
+ echo 'You have unstaged changes to some files in your commit; skipping '
+ echo 'auto-format. Please stage, stash, or revert these changes. You may '
+ echo 'find `git stash -k` helpful here.'
+ echo
+ echo 'Files with unstaged changes:'
+ for file in ${CHANGED_FILES[@]}; do
+ echo " $file"
+ done
+ exit 1
+fi
+
+# Format all staged files, then exit with an error code if any have uncommitted
+# changes.
+echo 'Formatting staged Python files . . .'
+black ${PYTHON_FILES[@]}
+
+CHANGED_FILES=(`git diff --name-only ${PYTHON_FILES[@]}`)
+if [ "$CHANGED_FILES" ]; then
+ echo 'Reformatted staged files. Please review and stage the changes.'
+ echo
+ echo 'Files updated:'
+ for file in ${CHANGED_FILES[@]}; do
+ echo " $file"
+ done
+ exit 1
+else
+ exit 0
+fi
diff --git a/tools/hooks/pre-commit.d/pre-commit-flake8.py b/tools/hooks/pre-commit.d/pre-commit-flake8.py
new file mode 100755
index 0000000..3eca277
--- /dev/null
+++ b/tools/hooks/pre-commit.d/pre-commit-flake8.py
@@ -0,0 +1,10 @@
+#!/usr/bin/env python
+"""Pre-commit hook to check code with flake8."""
+import sys
+
+from flake8.main import git
+
+if __name__ == "__main__":
+ sys.exit(
+ git.hook(strict=git.config_for("strict"), lazy=git.config_for("lazy"))
+ )
diff --git a/tox.ini b/tox.ini
index b0b15d2..650641f 100644
--- a/tox.ini
+++ b/tox.ini
@@ -4,7 +4,7 @@
# and then run "tox" from this directory.
[tox]
-envlist = py35
+envlist = py36
# There is no proper way to install the app for now (i.e. setup.py)
skipsdist = True
@@ -37,6 +37,15 @@
{[testenv:flake8]commands}
{[testenv:pylint]commands}
+# Git pre-commit hooks: Run formatter and linters
+[testenv:pre-commit-hooks]
+deps =
+ -rrequirements-dev.txt
+passenv = GIT_INDEX_FILE
+commands =
+ {toxinidir}/tools/hooks/pre-commit.d/pre-commit-black.sh
+ python {toxinidir}/tools/hooks/pre-commit.d/pre-commit-flake8.py
+
# Flake8 configuration
[flake8]
format = ${cyan}%(path)s${reset}:${yellow_bold}%(row)d${reset}:${green_bold}%(col)d${reset}: ${red_bold}%(code)s${reset} %(text)s