allow bytes-like for key/iv/data for symmetric encryption (#4621)
* allow bytearrays for key/iv for symmetric encryption
* bump pypy/cffi requirements
* update docs, fix some tests
* old openssl is naught but pain
* revert a typo
* use trusty for old pypy
* better error msg again
* restore match
diff --git a/.travis.yml b/.travis.yml
index 2a636ad..b8ecb7d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -31,9 +31,9 @@
env: TOXENV=py37
- python: 3.7
env: TOXENV=py37-idna
- - python: pypy-5.3
+ - python: pypy-5.4
env: TOXENV=pypy-nocoverage
- # PyPy 5.3 isn't available for xenial
+ # PyPy 5.4 isn't available for xenial
dist: trusty
- python: pypy2.7-5.10.0
env: TOXENV=pypy-nocoverage
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 25c7c8c..54004b4 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -12,6 +12,7 @@
version 2.1, but this version removes the default ``idna`` dependency as
well. If you still need this deprecated path please install cryptography
with the ``idna`` extra: ``pip install cryptography[idna]``.
+* **BACKWARDS INCOMPATIBLE:** The minimum supported PyPy version is now 5.4.
* Added support for :class:`~cryptography.hazmat.primitives.hashes.SHA512_224`
and :class:`~cryptography.hazmat.primitives.hashes.SHA512_256` when using
OpenSSL 1.1.1.
diff --git a/docs/glossary.rst b/docs/glossary.rst
index ce08dba..95b893c 100644
--- a/docs/glossary.rst
+++ b/docs/glossary.rst
@@ -94,6 +94,11 @@
bit key, you can calculate the number of bytes by dividing by 8. 128
divided by 8 is 16, so a 128 bit key is a 16 byte key.
+ bytes-like
+ A bytes-like object contains binary data and supports the
+ `buffer protocol`_. This includes ``bytes``, ``bytearray``, and
+ ``memoryview`` objects.
+
U-label
The presentational unicode form of an internationalized domain
name. U-labels use unicode characters outside the ASCII range and
@@ -101,3 +106,4 @@
.. _`hardware security module`: https://en.wikipedia.org/wiki/Hardware_security_module
.. _`idna`: https://pypi.org/project/idna/
+.. _`buffer protocol`: https://docs.python.org/3/c-api/buffer.html
diff --git a/docs/hazmat/primitives/symmetric-encryption.rst b/docs/hazmat/primitives/symmetric-encryption.rst
index 31f240c..4924ddd 100644
--- a/docs/hazmat/primitives/symmetric-encryption.rst
+++ b/docs/hazmat/primitives/symmetric-encryption.rst
@@ -92,8 +92,9 @@
AES is both fast, and cryptographically strong. It is a good default
choice for encryption.
- :param bytes key: The secret key. This must be kept secret. Either ``128``,
+ :param key: The secret key. This must be kept secret. Either ``128``,
``192``, or ``256`` :term:`bits` long.
+ :type key: :term:`bytes-like`
.. class:: Camellia(key)
@@ -101,8 +102,9 @@
It is considered to have comparable security and performance to AES but
is not as widely studied or deployed.
- :param bytes key: The secret key. This must be kept secret. Either ``128``,
+ :param key: The secret key. This must be kept secret. Either ``128``,
``192``, or ``256`` :term:`bits` long.
+ :type key: :term:`bytes-like`
.. class:: ChaCha20(key)
@@ -120,15 +122,17 @@
ChaCha20 is a stream cipher used in several IETF protocols. It is
standardized in :rfc:`7539`.
- :param bytes key: The secret key. This must be kept secret. ``256``
+ :param key: The secret key. This must be kept secret. ``256``
:term:`bits` (32 bytes) in length.
+ :type key: :term:`bytes-like`
- :param bytes nonce: Should be unique, a :term:`nonce`. It is
+ :param nonce: Should be unique, a :term:`nonce`. It is
critical to never reuse a ``nonce`` with a given key. Any reuse of a
nonce with the same key compromises the security of every message
encrypted with that key. The nonce does not need to be kept secret
and may be included with the ciphertext. This must be ``128``
:term:`bits` in length.
+ :type nonce: :term:`bytes-like`
.. note::
@@ -161,12 +165,13 @@
Nonetheless, Triple DES is not recommended for new applications because it
is incredibly slow; old applications should consider moving away from it.
- :param bytes key: The secret key. This must be kept secret. Either ``64``,
+ :param key: The secret key. This must be kept secret. Either ``64``,
``128``, or ``192`` :term:`bits` long. DES only uses ``56``, ``112``,
or ``168`` bits of the key as there is a parity byte in each component
of the key. Some writing refers to there being up to three separate
keys that are each ``56`` bits long, they can simply be concatenated
to produce the full key.
+ :type key: :term:`bytes-like`
.. class:: CAST5(key)
@@ -177,8 +182,9 @@
a variable key length cipher and supports keys from 40-128 :term:`bits` in
length.
- :param bytes key: The secret key, This must be kept secret. 40 to 128
+ :param key: The secret key, This must be kept secret. 40 to 128
:term:`bits` in length in increments of 8 bits.
+ :type key: :term:`bytes-like`
.. class:: SEED(key)
@@ -188,8 +194,9 @@
(KISA). It is defined in :rfc:`4269` and is used broadly throughout South
Korean industry, but rarely found elsewhere.
- :param bytes key: The secret key. This must be kept secret. ``128``
+ :param key: The secret key. This must be kept secret. ``128``
:term:`bits` in length.
+ :type key: :term:`bytes-like`
Weak ciphers
------------
@@ -206,8 +213,9 @@
susceptible to attacks when using weak keys. The author has recommended
that users of Blowfish move to newer algorithms such as :class:`AES`.
- :param bytes key: The secret key. This must be kept secret. 32 to 448
+ :param key: The secret key. This must be kept secret. 32 to 448
:term:`bits` in length in increments of 8 bits.
+ :type key: :term:`bytes-like`
.. class:: ARC4(key)
@@ -215,9 +223,10 @@
initial stream output. Its use is strongly discouraged. ARC4 does not use
mode constructions.
- :param bytes key: The secret key. This must be kept secret. Either ``40``,
+ :param key: The secret key. This must be kept secret. Either ``40``,
``56``, ``64``, ``80``, ``128``, ``192``, or ``256`` :term:`bits` in
length.
+ :type key: :term:`bytes-like`
.. doctest::
@@ -238,8 +247,9 @@
is susceptible to attacks when using weak keys. It is recommended that you
do not use this cipher for new applications.
- :param bytes key: The secret key. This must be kept secret. ``128``
+ :param key: The secret key. This must be kept secret. ``128``
:term:`bits` in length.
+ :type key: :term:`bytes-like`
.. _symmetric-encryption-modes:
@@ -256,13 +266,14 @@
**Padding is required when using this mode.**
- :param bytes initialization_vector: Must be :doc:`random bytes
+ :param initialization_vector: Must be :doc:`random bytes
</random-numbers>`. They do not need to be kept secret and they can be
included in a transmitted message. Must be the same number of bytes as
the ``block_size`` of the cipher. Each time something is encrypted a
new ``initialization_vector`` should be generated. Do not reuse an
``initialization_vector`` with a given ``key``, and particularly do not
use a constant ``initialization_vector``.
+ :type initialization_vector: :term:`bytes-like`
A good construction looks like:
@@ -295,12 +306,13 @@
**This mode does not require padding.**
- :param bytes nonce: Should be unique, a :term:`nonce`. It is
+ :param nonce: Should be unique, a :term:`nonce`. It is
critical to never reuse a ``nonce`` with a given key. Any reuse of a
nonce with the same key compromises the security of every message
encrypted with that key. Must be the same number of bytes as the
``block_size`` of the cipher with a given key. The nonce does not need
to be kept secret and may be included with the ciphertext.
+ :type nonce: :term:`bytes-like`
.. class:: OFB(initialization_vector)
@@ -309,11 +321,12 @@
**This mode does not require padding.**
- :param bytes initialization_vector: Must be :doc:`random bytes
+ :param initialization_vector: Must be :doc:`random bytes
</random-numbers>`. They do not need to be kept secret and they can be
included in a transmitted message. Must be the same number of bytes as
the ``block_size`` of the cipher. Do not reuse an
``initialization_vector`` with a given ``key``.
+ :type initialization_vector: :term:`bytes-like`
.. class:: CFB(initialization_vector)
@@ -322,11 +335,12 @@
**This mode does not require padding.**
- :param bytes initialization_vector: Must be :doc:`random bytes
+ :param initialization_vector: Must be :doc:`random bytes
</random-numbers>`. They do not need to be kept secret and they can be
included in a transmitted message. Must be the same number of bytes as
the ``block_size`` of the cipher. Do not reuse an
``initialization_vector`` with a given ``key``.
+ :type initialization_vector: :term:`bytes-like`
.. class:: CFB8(initialization_vector)
@@ -336,11 +350,12 @@
**This mode does not require padding.**
- :param bytes initialization_vector: Must be :doc:`random bytes
+ :param initialization_vector: Must be :doc:`random bytes
</random-numbers>`. They do not need to be kept secret and they can be
included in a transmitted message. Must be the same number of bytes as
the ``block_size`` of the cipher. Do not reuse an
``initialization_vector`` with a given ``key``.
+ :type initialization_vector: :term:`bytes-like`
.. class:: GCM(initialization_vector, tag=None, min_tag_length=16)
@@ -368,12 +383,13 @@
**This mode does not require padding.**
- :param bytes initialization_vector: Must be unique, a :term:`nonce`.
+ :param initialization_vector: Must be unique, a :term:`nonce`.
They do not need to be kept secret and they can be included in a
transmitted message. NIST `recommends a 96-bit IV length`_ for
performance critical situations but it can be up to 2\ :sup:`64` - 1
:term:`bits`. Do not reuse an ``initialization_vector`` with a given
``key``.
+ :type initialization_vector: :term:`bytes-like`
.. note::
@@ -495,11 +511,11 @@
**This mode does not require padding.**
- :param bytes tweak: The tweak is a 16 byte value typically derived from
+ :param tweak: The tweak is a 16 byte value typically derived from
something like the disk sector number. A given ``(tweak, key)`` pair
should not be reused, although doing so is less catastrophic than
in CTR mode.
-
+ :type tweak: :term:`bytes-like`
Insecure modes
--------------
@@ -544,7 +560,8 @@
.. method:: update(data)
- :param bytes data: The data you wish to pass into the context.
+ :param data: The data you wish to pass into the context.
+ :type data: :term:`bytes-like`
:return bytes: Returns the data that was encrypted or decrypted.
:raises cryptography.exceptions.AlreadyFinalized: See :meth:`finalize`
@@ -567,7 +584,8 @@
requirement and you will be making many small calls to
``update_into``.
- :param bytes data: The data you wish to pass into the context.
+ :param data: The data you wish to pass into the context.
+ :type data: :term:`bytes-like`
:param buf: A writable Python buffer that the data will be written
into. This buffer should be ``len(data) + n - 1`` bytes where ``n``
is the block size (in bytes) of the cipher and mode combination.
@@ -629,7 +647,8 @@
.. method:: authenticate_additional_data(data)
- :param bytes data: Any data you wish to authenticate but not encrypt.
+ :param data: Any data you wish to authenticate but not encrypt.
+ :type data: :term:`bytes-like`
:raises: :class:`~cryptography.exceptions.AlreadyFinalized`
.. class:: AEADEncryptionContext
@@ -747,7 +766,7 @@
.. attribute:: initialization_vector
- :type: bytes
+ :type: :term:`bytes-like`
Exact requirements of the initialization are described by the
documentation of individual modes.
@@ -759,7 +778,7 @@
.. attribute:: nonce
- :type: bytes
+ :type: :term:`bytes-like`
Exact requirements of the nonce are described by the documentation of
individual modes.
@@ -771,7 +790,7 @@
.. attribute:: tag
- :type: bytes
+ :type: :term:`bytes-like`
Exact requirements of the tag are described by the documentation of
individual modes.
@@ -785,7 +804,7 @@
.. attribute:: tweak
- :type: bytes
+ :type: :term:`bytes-like`
Exact requirements of the tweak are described by the documentation of
individual modes.
diff --git a/docs/installation.rst b/docs/installation.rst
index 2d9db66..5b2854d 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -11,7 +11,7 @@
-------------------
Currently we test ``cryptography`` on Python 2.7, 3.4+, and
-PyPy 5.3+ on these operating systems.
+PyPy 5.4+ on these operating systems.
* x86-64 CentOS 7.x
* macOS 10.12 Sierra, 10.11 El Capitan
diff --git a/pyproject.toml b/pyproject.toml
index 461675f..7d64f99 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -3,5 +3,5 @@
requires = [
"setuptools>=18.5",
"wheel",
- "cffi>=1.7,!=1.11.3; python_implementation != 'PyPy'",
+ "cffi>=1.8,!=1.11.3; python_implementation != 'PyPy'",
]
diff --git a/setup.py b/setup.py
index bd98fd5..5b29d32 100644
--- a/setup.py
+++ b/setup.py
@@ -44,12 +44,12 @@
VECTORS_DEPENDENCY = "cryptography_vectors=={0}".format(about['__version__'])
# `setup_requirements` must be kept in sync with `pyproject.toml`
-setup_requirements = ["cffi>=1.7,!=1.11.3"]
+setup_requirements = ["cffi>=1.8,!=1.11.3"]
if platform.python_implementation() == "PyPy":
- if sys.pypy_version_info < (5, 3):
+ if sys.pypy_version_info < (5, 4):
raise RuntimeError(
- "cryptography is not compatible with PyPy < 5.3. Please upgrade "
+ "cryptography is not compatible with PyPy < 5.4. Please upgrade "
"PyPy to use this library."
)
diff --git a/src/cryptography/hazmat/backends/openssl/ciphers.py b/src/cryptography/hazmat/backends/openssl/ciphers.py
index e0ee06e..fe5715b 100644
--- a/src/cryptography/hazmat/backends/openssl/ciphers.py
+++ b/src/cryptography/hazmat/backends/openssl/ciphers.py
@@ -56,13 +56,15 @@
)
if isinstance(mode, modes.ModeWithInitializationVector):
- iv_nonce = mode.initialization_vector
+ iv_nonce = self._backend._ffi.from_buffer(
+ mode.initialization_vector
+ )
elif isinstance(mode, modes.ModeWithTweak):
- iv_nonce = mode.tweak
+ iv_nonce = self._backend._ffi.from_buffer(mode.tweak)
elif isinstance(mode, modes.ModeWithNonce):
- iv_nonce = mode.nonce
+ iv_nonce = self._backend._ffi.from_buffer(mode.nonce)
elif isinstance(cipher, modes.ModeWithNonce):
- iv_nonce = cipher.nonce
+ iv_nonce = self._backend._ffi.from_buffer(cipher.nonce)
else:
iv_nonce = self._backend._ffi.NULL
# begin init with cipher and operation type
@@ -105,7 +107,7 @@
ctx,
self._backend._ffi.NULL,
self._backend._ffi.NULL,
- cipher.key,
+ self._backend._ffi.from_buffer(cipher.key),
iv_nonce,
operation
)
@@ -131,8 +133,10 @@
"unsigned char *", self._backend._ffi.from_buffer(buf)
)
outlen = self._backend._ffi.new("int *")
- res = self._backend._lib.EVP_CipherUpdate(self._ctx, buf, outlen,
- data, len(data))
+ res = self._backend._lib.EVP_CipherUpdate(
+ self._ctx, buf, outlen,
+ self._backend._ffi.from_buffer(data), len(data)
+ )
self._backend.openssl_assert(res != 0)
return outlen[0]
@@ -215,7 +219,8 @@
def authenticate_additional_data(self, data):
outlen = self._backend._ffi.new("int *")
res = self._backend._lib.EVP_CipherUpdate(
- self._ctx, self._backend._ffi.NULL, outlen, data, len(data)
+ self._ctx, self._backend._ffi.NULL, outlen,
+ self._backend._ffi.from_buffer(data), len(data)
)
self._backend.openssl_assert(res != 0)
diff --git a/src/cryptography/hazmat/primitives/ciphers/algorithms.py b/src/cryptography/hazmat/primitives/ciphers/algorithms.py
index 21d9ecf..1f49fd9 100644
--- a/src/cryptography/hazmat/primitives/ciphers/algorithms.py
+++ b/src/cryptography/hazmat/primitives/ciphers/algorithms.py
@@ -13,7 +13,7 @@
def _verify_key_size(algorithm, key):
# Verify that the key is instance of bytes
- utils._check_bytes("key", key)
+ utils._check_byteslike("key", key)
# Verify that the key size matches the expected key size
if len(key) * 8 not in algorithm.key_sizes:
@@ -153,7 +153,7 @@
def __init__(self, key, nonce):
self.key = _verify_key_size(self, key)
- utils._check_bytes("nonce", nonce)
+ utils._check_byteslike("nonce", nonce)
if len(nonce) != 16:
raise ValueError("nonce must be 128-bits (16 bytes)")
diff --git a/src/cryptography/hazmat/primitives/ciphers/modes.py b/src/cryptography/hazmat/primitives/ciphers/modes.py
index d244458..ad91a6e 100644
--- a/src/cryptography/hazmat/primitives/ciphers/modes.py
+++ b/src/cryptography/hazmat/primitives/ciphers/modes.py
@@ -88,7 +88,7 @@
name = "CBC"
def __init__(self, initialization_vector):
- utils._check_bytes("initialization_vector", initialization_vector)
+ utils._check_byteslike("initialization_vector", initialization_vector)
self._initialization_vector = initialization_vector
initialization_vector = utils.read_only_property("_initialization_vector")
@@ -101,7 +101,7 @@
name = "XTS"
def __init__(self, tweak):
- utils._check_bytes("tweak", tweak)
+ utils._check_byteslike("tweak", tweak)
if len(tweak) != 16:
raise ValueError("tweak must be 128-bits (16 bytes)")
@@ -131,7 +131,7 @@
name = "OFB"
def __init__(self, initialization_vector):
- utils._check_bytes("initialization_vector", initialization_vector)
+ utils._check_byteslike("initialization_vector", initialization_vector)
self._initialization_vector = initialization_vector
initialization_vector = utils.read_only_property("_initialization_vector")
@@ -144,7 +144,7 @@
name = "CFB"
def __init__(self, initialization_vector):
- utils._check_bytes("initialization_vector", initialization_vector)
+ utils._check_byteslike("initialization_vector", initialization_vector)
self._initialization_vector = initialization_vector
initialization_vector = utils.read_only_property("_initialization_vector")
@@ -157,7 +157,7 @@
name = "CFB8"
def __init__(self, initialization_vector):
- utils._check_bytes("initialization_vector", initialization_vector)
+ utils._check_byteslike("initialization_vector", initialization_vector)
self._initialization_vector = initialization_vector
initialization_vector = utils.read_only_property("_initialization_vector")
@@ -170,7 +170,7 @@
name = "CTR"
def __init__(self, nonce):
- utils._check_bytes("nonce", nonce)
+ utils._check_byteslike("nonce", nonce)
self._nonce = nonce
nonce = utils.read_only_property("_nonce")
@@ -195,7 +195,7 @@
# len(initialization_vector) must in [1, 2 ** 64), but it's impossible
# to actually construct a bytes object that large, so we don't check
# for it
- utils._check_bytes("initialization_vector", initialization_vector)
+ utils._check_byteslike("initialization_vector", initialization_vector)
if len(initialization_vector) == 0:
raise ValueError("initialization_vector must be at least 1 byte")
self._initialization_vector = initialization_vector
diff --git a/src/cryptography/utils.py b/src/cryptography/utils.py
index 3d45a77..65a4ee7 100644
--- a/src/cryptography/utils.py
+++ b/src/cryptography/utils.py
@@ -30,6 +30,13 @@
raise TypeError("{0} must be bytes".format(name))
+def _check_byteslike(name, value):
+ try:
+ memoryview(value)
+ except TypeError:
+ raise TypeError("{0} must be bytes-like".format(name))
+
+
def read_only_property(name):
return property(lambda self: getattr(self, name))
diff --git a/tests/hazmat/primitives/test_aes.py b/tests/hazmat/primitives/test_aes.py
index 4ceccf1..90a6b3b 100644
--- a/tests/hazmat/primitives/test_aes.py
+++ b/tests/hazmat/primitives/test_aes.py
@@ -455,3 +455,45 @@
).decryptor()
with pytest.raises(ValueError):
decryptor.finalize_with_tag(b"tagtooshort")
+
+ def test_buffer_protocol(self, backend):
+ data = bytearray(b"helloworld")
+ enc = base.Cipher(
+ algorithms.AES(bytearray(b"\x00" * 16)),
+ modes.GCM(bytearray(b"\x00" * 12)),
+ backend
+ ).encryptor()
+ enc.authenticate_additional_data(bytearray(b"foo"))
+ ct = enc.update(data) + enc.finalize()
+ dec = base.Cipher(
+ algorithms.AES(bytearray(b"\x00" * 16)),
+ modes.GCM(bytearray(b"\x00" * 12), enc.tag),
+ backend
+ ).decryptor()
+ dec.authenticate_additional_data(bytearray(b"foo"))
+ pt = dec.update(ct) + dec.finalize()
+ assert pt == data
+
+
+@pytest.mark.parametrize(
+ "mode",
+ [
+ modes.CBC(bytearray(b"\x00" * 16)),
+ modes.CTR(bytearray(b"\x00" * 16)),
+ modes.OFB(bytearray(b"\x00" * 16)),
+ modes.CFB(bytearray(b"\x00" * 16)),
+ modes.CFB8(bytearray(b"\x00" * 16)),
+ modes.XTS(bytearray(b"\x00" * 16)),
+ ]
+)
+@pytest.mark.requires_backend_interface(interface=CipherBackend)
+def test_buffer_protocol_alternate_modes(mode, backend):
+ data = bytearray(b"sixteen_byte_msg")
+ cipher = base.Cipher(
+ algorithms.AES(bytearray(b"\x00" * 32)), mode, backend
+ )
+ enc = cipher.encryptor()
+ ct = enc.update(data) + enc.finalize()
+ dec = cipher.decryptor()
+ pt = dec.update(ct) + dec.finalize()
+ assert pt == data
diff --git a/tests/hazmat/primitives/test_chacha20.py b/tests/hazmat/primitives/test_chacha20.py
index 33730d9..7c475c0 100644
--- a/tests/hazmat/primitives/test_chacha20.py
+++ b/tests/hazmat/primitives/test_chacha20.py
@@ -44,6 +44,18 @@
computed_ct = encryptor.update(pt) + encryptor.finalize()
assert binascii.hexlify(computed_ct) == vector["ciphertext"]
+ def test_buffer_protocol(self, backend):
+ key = bytearray(os.urandom(32))
+ nonce = bytearray(os.urandom(16))
+ cipher = Cipher(
+ algorithms.ChaCha20(key, nonce), None, backend
+ )
+ enc = cipher.encryptor()
+ ct = enc.update(bytearray(b"hello")) + enc.finalize()
+ dec = cipher.decryptor()
+ pt = dec.update(ct) + dec.finalize()
+ assert pt == b"hello"
+
def test_key_size(self):
chacha = algorithms.ChaCha20(b"0" * 32, b"0" * 16)
assert chacha.key_size == 256