7ebc5a3570
* Switched from links to some external docs to using footnotes where necessary. * Ideally the howto should be as stand alone as possible. * Also it makes it difficult to convert to another format for proof-reading if there are links that the conversion can't find.
569 lines
17 KiB
Org Mode
569 lines
17 KiB
Org Mode
#+TITLE: GNU Privacy Guard (GnuPG) Made Easy Python Bindings HOWTO (English)
|
|
#+LATEX_COMPILER: xelatex
|
|
#+LATEX_CLASS: article
|
|
#+LATEX_CLASS_OPTIONS: [12pt]
|
|
#+LATEX_HEADER: \usepackage{xltxtra}
|
|
#+LATEX_HEADER: \usepackage[margin=1in]{geometry}
|
|
#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman}
|
|
#+LATEX_HEADER: \author{Ben McGinnes <ben@gnupg.org>}
|
|
|
|
|
|
* Introduction
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: intro
|
|
:END:
|
|
|
|
Version: 0.0.1-alpha [2018-03-07 Wed]
|
|
Author: Ben McGinnes <ben@gnupg.org>
|
|
Author GPG Key: DB4724E6FA4286C92B4E55C4321E4E2373590E5D
|
|
|
|
This document provides basic instruction in how to use the GPGME
|
|
Python bindings to programmatically leverage the GPGME library.
|
|
|
|
|
|
* GPGME Concepts
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: gpgme-concepts
|
|
:END:
|
|
|
|
** A C API
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: gpgme-c-api
|
|
:END:
|
|
|
|
Unlike many modern APIs with which programmers will be more
|
|
familiar with these days, the GPGME API is a C API. The API is
|
|
intended for use by C coders who would be able to access its
|
|
features by including the =gpgme.h= header file eith their own C
|
|
source code and then access its functions just as they would any
|
|
other C headers.
|
|
|
|
This is a very effective method of gaining complete access to the
|
|
API and in the most efficient manner possible. It does, however,
|
|
have the drawback that it cannot be directly used by other
|
|
languages without some means of providing an interface to those
|
|
languages. This is where the need for bindings in various
|
|
languages stems.
|
|
|
|
** Python bindings
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: gpgme-python-bindings
|
|
:END:
|
|
|
|
The Python bindings for GPGME provide a higher level means of
|
|
accessing the complete feature set of GPGME itself. It also
|
|
provides a more pythonic means of calling these API functions.
|
|
|
|
The bindings are generated dynamically with SWIG and the copy of =gpgme.h= generated when GPGME is compiled.
|
|
|
|
This means that a version of the Python bindings is fundamentally
|
|
tied to the exact same version of GPGME used to gemerate that copy
|
|
of =gpgme.h=.
|
|
|
|
** Difference between the Python bindings and other GnuPG Python packages
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: gpgme-python-bindings-diffs
|
|
:END:
|
|
|
|
There have been numerous attempts to add GnuPG support to Python
|
|
over the years. Some of the most well known are listed here, along
|
|
with what differentiates them.
|
|
|
|
*** The python-gnupg package maintained by Vinay Sajip
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: diffs-python-gnupg
|
|
:END:
|
|
|
|
This is arguably the most popular means of integrating GPG with
|
|
Python. The package utilises the =subprocess= module to implement
|
|
wrappers for the =gpg= and =gpg2= executables normally invoked on
|
|
the command line (=gpg.exe= and =gpg2.exe= on Windows).
|
|
|
|
The popularity of this package stemmed from its ease of use and
|
|
capability in providing the most commonly required features.
|
|
|
|
Unfortunately it has been beset by a number of security issues,
|
|
most of which stemmed from using unsafe methods of accessing the
|
|
command line via the =subprocess= calls.
|
|
|
|
The python-gnupg package is available under the MIT license.
|
|
|
|
*** The gnupg package created and maintained by Isis Lovecruft
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: diffs-isis-gnupg
|
|
:END:
|
|
|
|
In 2015 Isis Lovecruft from the Tor Project forked and then
|
|
re-implemented the python-gnupg package as just gnupg. This new
|
|
package also relied on subprocess to call the =gpg= or =gpg2=
|
|
binaries, but did so somewhat more securely.
|
|
|
|
However the naming and version numbering selected for this package
|
|
resulted in conflicts with the original python-gnupg and since its
|
|
functions were called in a different manner, the release of this
|
|
package also resulted in a great deal of consternation when people
|
|
installed what they thought was an upgrade that subsequently broke
|
|
the code relying on it.
|
|
|
|
The gnupg package is available under the GNU General Public
|
|
License version 3.0 (or later).
|
|
|
|
*** The PyME package maintained by Martin Albrecht
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: diffs-pyme
|
|
:END:
|
|
|
|
This package is the origin of these bindings, though they are
|
|
somewhat different now. For details of when and how the PyME
|
|
package was folded back into GPGME itself see the /Short History/
|
|
document[fn:1] in this Python bindings =docs= directory.[fn:2]
|
|
|
|
The PyME package was first released in 2002 and was also the first
|
|
attempt to implement a low level binding to GPGME. In doing so it
|
|
provided access to considerably more functionality than either the =python-gnupg= or =gnupg= packages.
|
|
|
|
The PyME package is only available for Python 2.6 and 2.7.
|
|
|
|
Porting the PyME package to Python 3.4 in 2015 is what resulted in
|
|
it being folded into the GPGME project and the current bindings
|
|
are the end result of that effort.
|
|
|
|
The PyME package is available under the same dual licensing as
|
|
GPGME itself: the GNU General Public License version 2.0 (or any
|
|
later version) and the GNU Lesser Public License version 2.1 (or
|
|
any later version).
|
|
|
|
|
|
* GPGME Python bindings installation
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: gpgme-python-install
|
|
:END:
|
|
|
|
** No PyPI
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: do-not-use-pypi
|
|
:END:
|
|
|
|
Most third-party Python packages and modules are available and
|
|
distributed through the Python Package Installer, known as PyPI.
|
|
|
|
Due to the nature of what these bindings are and how they work, it
|
|
is infeasible to install the GPGME Python bindings in the same way.
|
|
|
|
** Requirements
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: gpgme-python-requirements
|
|
:END:
|
|
|
|
The GPGME Python bindings only have three requirements:
|
|
|
|
1. A suitable version of Python 2 or Python 3. With Python 2 that
|
|
means Python 2.7 and with Python 3 that means Python 3.4 or
|
|
higher.
|
|
2. SWIG.
|
|
3. GPGME itself. Which also means that all of GPGME's dependencies
|
|
must be installed too.
|
|
|
|
** Installation
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: installation
|
|
:END:
|
|
|
|
Installing the Python bindings is effectively achieved by compiling
|
|
and installing GPGME itself.
|
|
|
|
Once SWIG is installed with Python and all the dependencies for
|
|
GPGME are installed you only need to confirm that the version(s) of
|
|
Python you want the bindings installed for are in your =$PATH=.
|
|
|
|
By default GPGME will attempt to install the bindings for the most
|
|
recent or highest version number of Python 2 and Python 3 it
|
|
detects in =$PATH=. It specifically checks for the =python= and =python3= executabled first and then checks for specific version
|
|
numbers.
|
|
|
|
For Python 2 it checks for these executables in this order: =python=, =python2= and =python2.7=.
|
|
|
|
For Python 3 it checks for these executables in this order: =python3=, =python3.6=, =python3.5= and =python3.4=.
|
|
|
|
*** Installing GPGME
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: install-gpgme
|
|
:END:
|
|
|
|
See the GPGME =README= file for details of how to install GPGME from
|
|
source.
|
|
|
|
|
|
* Fundamentals
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-fund-a-mental
|
|
:END:
|
|
|
|
Before we can get to the fun stuff, there are a few matters
|
|
regarding GPGME's design which hold true whether you're dealing with
|
|
the C code directly or these Python bindings.
|
|
|
|
** No REST
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: no-rest-for-the-wicked
|
|
:END:
|
|
|
|
The first part of which is or will be fairly blatantly obvious upon
|
|
viewing the first example, but it's worth reiterating anyway. That
|
|
being that this API is /*not*/ a REST API. Nor indeed could it
|
|
ever be one.
|
|
|
|
Most, if not all, Python programmers (and not just Python
|
|
programmers) know how easy it is to work with a RESTful API. In
|
|
fact they've become so popular that many other APIs attempt to
|
|
emulate REST-like behaviour as much as they are able. Right down
|
|
to the use of JSON formatted output to facilitate the use of their
|
|
API without having to retrain developers.
|
|
|
|
This API does not do that. It would not be able to do that and
|
|
also provide access to the entire C API on which it's built. It
|
|
does, however, provide a very pythonic interface on top of the
|
|
direct bindings and it's this pythonic layer with which this HOWTO
|
|
deals with.
|
|
|
|
** Context
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-get-context
|
|
:END:
|
|
|
|
One of the reasons which prevents this API from being RESTful is
|
|
that most operations require more than one instruction to the API
|
|
to perform the task. Sure, there are certain functions which can
|
|
be performed simultaneously, particularly if the result known or
|
|
strongly anticipated (e.g selecting and encrypting to a key known
|
|
to be in the public keybox).
|
|
|
|
There are many more, however, which cannot be manipulated so
|
|
readily: they must be performed in a specific sequence and the
|
|
result of one operation has a direct bearing on the outcome of
|
|
subsequent operations. Not merely by generating an error either.
|
|
|
|
When dealing with this type of persistant state on the web, full of
|
|
both the RESTful and REST-like, it's most commonly referred to as a
|
|
session. In GPGME, however, it is called a context and every
|
|
operation type has one.
|
|
|
|
|
|
* Basic Functions
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-the-basics
|
|
:END:
|
|
|
|
The most frequently called features of any cryptographic library
|
|
will be the most fundamental tasks for enxryption software. In this
|
|
section we will look at how to programmatically encrypt data,
|
|
decrypt it, sign it and verify signatures.
|
|
|
|
** Encryption
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-basic-encryption
|
|
:END:
|
|
|
|
Encrypting is very straight forward. In the first example below
|
|
the message, =text=, is encrypted to a single recipient's key. In
|
|
the second example the message will be encrypted to multiple
|
|
recipients.
|
|
|
|
*** Encrypting to one key
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-basic-encryption-single
|
|
:END:
|
|
|
|
The text is then encapsulated in a GPGME Data object as =plain= and
|
|
the =cipher= object is created with another Data object. Then we
|
|
create the Context as =c= and set it to use the ASCII armoured
|
|
OpenPGP format. In later examples there will be alternative
|
|
methods of setting the OpenPGP output to be ASCII armoured.
|
|
|
|
Next we prepare a keylist object in our Context and follow it with
|
|
specifying the recipients as =r=. Note that the configuration in
|
|
one's =gpg.conf= file is honoured, so if you have the options set
|
|
to encrypt to one key or to a default key, that will be included
|
|
with this operation.
|
|
|
|
This is followed by a quick check to be sure that the recipient is
|
|
actually selected and that the key is available. Assuming it is,
|
|
the encryption can proceed, but if not a message will print stating
|
|
the key was not found.
|
|
|
|
The encryption operation is invoked within the Context with the =c.op_encrypt= function, loading the recipien (=r=), the message
|
|
(=plain=) and the =cipher=. The =cipher.seek= uses =os.SEEK_SET=
|
|
to set the data to the correct byte format for GPGME to use it.
|
|
|
|
At this point we no longer need the plaintext material, so we
|
|
delete both the =text= and the =plain= objects. Then we write the
|
|
encrypted data out to a file, =secret_plans.txt.asc=.
|
|
|
|
#+begin_src python
|
|
import gpg
|
|
import os
|
|
|
|
rkey = "0x12345678DEADBEEF"
|
|
text = """
|
|
Some plain text to test with. Obtained from any input source Python can read.
|
|
|
|
It makes no difference whether it is string or bytes, but the bindings always
|
|
produce byte output data. Which is useful to know when writing out either the
|
|
encrypted or decrypted results.
|
|
|
|
"""
|
|
|
|
plain = gpg.core.Data(text)
|
|
cipher = gpg.core.Data()
|
|
c = gpg.core.Context()
|
|
c.set_armor(1)
|
|
|
|
c.op_keylist_start(rkey, 0)
|
|
r = c.op_keylist_next()
|
|
|
|
if r == None:
|
|
print("""The key for user "{0}" was not found""".format(rkey))
|
|
else:
|
|
try:
|
|
c.op_encrypt([r], 1, plain, cipher)
|
|
cipher.seek(0, os.SEEK_SET)
|
|
del(text)
|
|
del(plain)
|
|
afile = open("secret_plans.txt.asc", "wb")
|
|
afile.write(cipher.read())
|
|
afile.close()
|
|
except gpg.errors.GPGMEError as ex:
|
|
print(ex.getstring())
|
|
#+end_src
|
|
|
|
*** Encrypting to multiple keys
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-basic-encryption-multiple
|
|
:END:
|
|
|
|
|
|
** Decryption
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-basic-encryption
|
|
:END:
|
|
|
|
Decrypting something encrypted to a key in one's secret keyring
|
|
(will display some extra data you normally wouldn't show, but which
|
|
may be of use):
|
|
|
|
#+begin_src python
|
|
import os.path
|
|
import gpg
|
|
|
|
if os.path.exists("/path/to/secret_plans.txt.asc") is True:
|
|
ciphertext = "/path/to/secret_plans.txt.asc"
|
|
elif os.path.exists("/path/to/secret_plans.txt.gpg") is True:
|
|
ciphertext = "/path/to/secret_plans.txt.gpg"
|
|
else:
|
|
ciphertext = None
|
|
|
|
if ciphertext is not None:
|
|
afile = open(ciphertext, "rb")
|
|
plaintext = gpg.Context().decrypt(afile)
|
|
afile.close()
|
|
newfile = open("/path/to/secret_plans.txt", "wb")
|
|
newfile.write(plaintext[0])
|
|
newfile.close()
|
|
print(plaintext[0])
|
|
plaintext[1]
|
|
plaintext[2]
|
|
del(plaintext)
|
|
else:
|
|
pass
|
|
#+end_src
|
|
|
|
|
|
** Signing text
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-basic-signing
|
|
:END:
|
|
|
|
Need to determine whether or not to include clearsigning and
|
|
detached signing here or give them separate sections.
|
|
|
|
#+begin_src python
|
|
import gpg
|
|
|
|
text = """Declaration of ... something.
|
|
|
|
"""
|
|
|
|
c = gpg.Context()
|
|
c.armor = True
|
|
signed = c.sign(text, mode=mode.NORMAL)
|
|
|
|
afile = open("/path/to/statement.txt.asc", "w")
|
|
for i in range(len(signed[0].splitlines())):
|
|
afile.write("{0}\n".format(signed[0].splitlines()[i].decode('utf-8')))
|
|
afile.close()
|
|
#+end_src
|
|
|
|
Clearsigning:
|
|
|
|
#+begin_src python
|
|
import gpg
|
|
|
|
text = """Declaration of ... something.
|
|
|
|
"""
|
|
|
|
c = gpg.Context()
|
|
c.armor = True
|
|
signed = c.sign(text, mode=mode.CLEAR)
|
|
|
|
afile = open("/path/to/statement.txt.asc", "w")
|
|
for i in range(len(signed[0].splitlines())):
|
|
afile.write("{0}\n".format(signed[0].splitlines()[i].decode('utf-8')))
|
|
afile.close()
|
|
#+end_src
|
|
|
|
Detached ASCII Armoured signing:
|
|
|
|
#+begin_src python
|
|
import gpg
|
|
|
|
text = """Declaration of ... something.
|
|
|
|
"""
|
|
|
|
c = gpg.Context()
|
|
c.armor = True
|
|
signed = c.sign(text, mode=mode.DETACH)
|
|
|
|
afile = open("/path/to/statement.txt.asc", "w")
|
|
for i in range(len(signed[0].splitlines())):
|
|
afile.write("{0}\n".format(signed[0].splitlines()[i].decode('utf-8')))
|
|
afile.close()
|
|
#+end_src
|
|
|
|
Detached binary signing of a file.
|
|
|
|
#+begin_src python
|
|
import gpg
|
|
|
|
tfile = open("/path/to/statement.txt", "r")
|
|
text = tfile.read()
|
|
tfile.close()
|
|
|
|
c = gpg.Context()
|
|
c.armor = True
|
|
signed = c.sign(text, mode=mode.DETACH)
|
|
|
|
afile = open("/path/to/statement.txt.sig", "wb")
|
|
afile.write(signed[0])
|
|
afile.close()
|
|
#+end_src
|
|
|
|
|
|
** Signature verification
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-basic-verification
|
|
:END:
|
|
|
|
Verify a signed file, both detached and not:
|
|
|
|
#+begin_src python
|
|
import gpg
|
|
import sys
|
|
import time
|
|
|
|
c = gpg.Context()
|
|
|
|
data, result = c.verify(open(filename),
|
|
open(detached_sig_filename)
|
|
if detached_sig_filename else None)
|
|
|
|
for index, sign in enumerate(result.signatures):
|
|
print("signature", index, ":")
|
|
print(" summary: %#0x" % (sign.summary))
|
|
print(" status: %#0x" % (sign.status))
|
|
print(" timestamp: ", sign.timestamp)
|
|
print(" timestamp: ", time.ctime(sign.timestamp))
|
|
print(" fingerprint:", sign.fpr)
|
|
print(" uid: ", c.get_key(sign.fpr).uids[0].uid)
|
|
|
|
if data:
|
|
sys.stdout.buffer.write(data)
|
|
#+end_src
|
|
|
|
|
|
* Working with keys
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-keys
|
|
:END:
|
|
|
|
** Counting keys
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: howto-basic-verification
|
|
:END:
|
|
|
|
Counting the number of keys in your public keybox (=pubring.kbx=),
|
|
the format shich has superceded the old keyring format
|
|
(=pubring.gpg= and =secring.gpg=) is a very simple task.
|
|
|
|
#+begin_src python
|
|
import gpg
|
|
|
|
c = gpg.Context()
|
|
seckeys = c.keylist(pattern=None, secret=True)
|
|
pubkeys = c.keylist(pattern=None, secret=False)
|
|
|
|
seclist = list(seckeys)
|
|
secnum = len(seclist)
|
|
|
|
publist = list(pubkeys)
|
|
pubnum = len(publist)
|
|
|
|
print("""
|
|
Number of secret keys: {0}
|
|
Number of public keys: {1}
|
|
""".format(secnum, pubnum)
|
|
#+end_src
|
|
|
|
* Copyright and Licensing
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: copyright-and-license
|
|
:END:
|
|
|
|
** Copyright (C) The GnuPG Project, 2018
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: copyright
|
|
:END:
|
|
|
|
Copyright © The GnuPG Project, 2018.
|
|
|
|
** License GPL compatible
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: license
|
|
:END:
|
|
|
|
This file is free software; as a special exception the author gives
|
|
unlimited permission to copy and/or distribute it, with or without
|
|
modifications, as long as this notice is preserved.
|
|
|
|
This file is distributed in the hope that it will be useful, but
|
|
WITHOUT ANY WARRANTY, to the extent permitted by law; without even
|
|
the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
|
|
PURPOSE.
|
|
|
|
|
|
* Footnotes
|
|
:PROPERTIES:
|
|
:CUSTOM_ID: footnotes
|
|
:END:
|
|
|
|
[fn:1] Short_History.org and/or Short_History.html.
|
|
|
|
[fn:2] The =lang/python/docs/= directory in the GPGME source.
|