gpgme/lang/python/doc/rst/gpgme-python-howto.rst

.. _intro:

Introduction
============

+-----------------------------------+-----------------------------------+
| Version:                          | 0.1.4                             |
+-----------------------------------+-----------------------------------+
| GPGME Version:                    | 1.12.1                            |
+-----------------------------------+-----------------------------------+
| Author:                           | `Ben                              |
|                                   | McGinnes <https://gnupg.org/peopl |
|                                   | e/index.html#sec-1-5>`__          |
|                                   | <ben@gnupg.org>                   |
+-----------------------------------+-----------------------------------+
| Author GPG Key:                   | DB4724E6FA4286C92B4E55C4321E4E237 |
|                                   | 3590E5D                           |
+-----------------------------------+-----------------------------------+
| Language:                         | Australian English, British       |
|                                   | English                           |
+-----------------------------------+-----------------------------------+
| Language codes:                   | en-AU, en-GB, en                  |
+-----------------------------------+-----------------------------------+

This document provides basic instruction in how to use the GPGME Python
bindings to programmatically leverage the GPGME library.

.. _py2-vs-py3:

Python 2 versus Python 3
------------------------

Though the GPGME Python bindings themselves provide support for both
Python 2 and 3, the focus is unequivocally on Python 3 and specifically
from Python 3.4 and above. As a consequence all the examples and
instructions in this guide use Python 3 code.

Much of it will work with Python 2, but much of it also deals with
Python 3 byte literals, particularly when reading and writing data.
Developers concentrating on Python 2.7, and possibly even 2.6, will need
to make the appropriate modifications to support the older string and
unicode types as opposed to bytes.

There are multiple reasons for concentrating on Python 3; some of which
relate to the immediate integration of these bindings, some of which
relate to longer term plans for both GPGME and the python bindings and
some of which relate to the impending EOL period for Python 2.7.
Essentially, though, there is little value in tying the bindings to a
version of the language which is a dead end and the advantages offered
by Python 3 over Python 2 make handling the data types with which GPGME
deals considerably easier.

.. _howto-python3-examples:

Examples
--------

All of the examples found in this document can be found as Python 3
scripts in the ``lang/python/examples/howto`` directory.

Unofficial Drafts
-----------------

In addition to shipping with each release of GPGME, there is a section
on locations to read or download `draft editions <#draft-editions>`__ of
this document from at the end of it. These are unofficial versions
produced in between major releases.

.. _new-stuff:

What\'s New
-----------

Full details of what is new are now available in the `What\'s
New <what-is-new.org>`__ file and archives of the preceding *What\'s
New* sections are available in the `What Was New <what-was-new>`__ file.

.. _new-stuff-1-13-0:

New in GPGME 1·13·0
~~~~~~~~~~~~~~~~~~~

See the `What\'s New <what-is-new#new-stuff-1-13-0>`__ document for what
is new in version 1.13.0.

.. _new-stuff-1-12-0:

New in GPGME 1·12·0
~~~~~~~~~~~~~~~~~~~

See the `What Was New <what-was-new#new-stuff-1-12-0>`__ document for
what was new in version 1.12.0.

GPGME Concepts
==============

.. _gpgme-c-api:

A C API
-------

Unlike many modern APIs with which programmers will be more familiar
with these days, the GPGME API is a C API. The API is intended for use
by C coders who would be able to access its features by including the
``gpgme.h`` header file with their own C source code and then access its
functions just as they would any other C headers.

This is a very effective method of gaining complete access to the API
and in the most efficient manner possible. It does, however, have the
drawback that it cannot be directly used by other languages without some
means of providing an interface to those languages. This is where the
need for bindings in various languages stems.

.. _gpgme-python-bindings:

Python bindings
---------------

The Python bindings for GPGME provide a higher level means of accessing
the complete feature set of GPGME itself. It also provides a more
pythonic means of calling these API functions.

The bindings are generated dynamically with SWIG and the copy of
``gpgme.h`` generated when GPGME is compiled.

This means that a version of the Python bindings is fundamentally tied
to the exact same version of GPGME used to generate that copy of
``gpgme.h``.

.. _gpgme-python-bindings-diffs:

Difference between the Python bindings and other GnuPG Python packages
----------------------------------------------------------------------

There have been numerous attempts to add GnuPG support to Python over
the years. Some of the most well known are listed here, along with what
differentiates them.

.. _diffs-python-gnupg:

The python-gnupg package maintained by Vinay Sajip
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is arguably the most popular means of integrating GPG with Python.
The package utilises the ``subprocess`` module to implement wrappers for
the ``gpg`` and ``gpg2`` executables normally invoked on the command
line (``gpg.exe`` and ``gpg2.exe`` on Windows).

The popularity of this package stemmed from its ease of use and
capability in providing the most commonly required features.

Unfortunately it has been beset by a number of security issues in the
past; most of which stemmed from using unsafe methods of accessing the
command line via the ``subprocess`` calls. While some effort has been
made over the last two to three years (as of 2018) to mitigate this,
particularly by no longer providing shell access through those
subprocess calls, the wrapper is still somewhat limited in the scope of
its GnuPG features coverage.

The python-gnupg package is available under the MIT license.

.. _diffs-isis-gnupg:

The gnupg package created and maintained by Isis Lovecruft
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In 2015 Isis Lovecruft from the Tor Project forked and then
re-implemented the python-gnupg package as just gnupg. This new package
also relied on subprocess to call the ``gpg`` or ``gpg2`` binaries, but
did so somewhat more securely.

The naming and version numbering selected for this package, however,
resulted in conflicts with the original python-gnupg and since its
functions were called in a different manner to python-gnupg, the release
of this package also resulted in a great deal of consternation when
people installed what they thought was an upgrade that subsequently
broke the code relying on it.

The gnupg package is available under the GNU General Public License
version 3.0 (or any later version).

.. _diffs-pyme:

The PyME package maintained by Martin Albrecht
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This package is the origin of these bindings, though they are somewhat
different now. For details of when and how the PyME package was folded
back into GPGME itself see the `Short History <short-history.org>`__
document. [1]_

The PyME package was first released in 2002 and was also the first
attempt to implement a low level binding to GPGME. In doing so it
provided access to considerably more functionality than either the
``python-gnupg`` or ``gnupg`` packages.

The PyME package is only available for Python 2.6 and 2.7.

Porting the PyME package to Python 3.4 in 2015 is what resulted in it
being folded into the GPGME project and the current bindings are the end
result of that effort.

The PyME package is available under the same dual licensing as GPGME
itself: the GNU General Public License version 2.0 (or any later
version) and the GNU Lesser General Public License version 2.1 (or any
later version).

.. _gpgme-python-install:

GPGME Python bindings installation
==================================

.. _do-not-use-pypi:

No PyPI
-------

Most third-party Python packages and modules are available and
distributed through the Python Package Installer, known as PyPI.

Due to the nature of what these bindings are and how they work, it is
infeasible to install the GPGME Python bindings in the same way.

This is because the bindings use SWIG to dynamically generate C bindings
against ``gpgme.h`` and ``gpgme.h`` is generated from ``gpgme.h.in`` at
compile time when GPGME is built from source. Thus to include a package
in PyPI which actually built correctly would require either statically
built libraries for every architecture bundled with it or a full
implementation of C for each architecture.

See the additional notes regarding `CFFI and SWIG <#snafu-cffi>`__ at
the end of this section for further details.

.. _gpgme-python-requirements:

Requirements
------------

The GPGME Python bindings only have three requirements:

#. A suitable version of Python 2 or Python 3. With Python 2 that means
   CPython 2.7 and with Python 3 that means CPython 3.4 or higher.
#. `SWIG <https://www.swig.org>`__.
#. GPGME itself. Which also means that all of GPGME\'s dependencies must
   be installed too.

.. _gpgme-python-recommendations:

Recommended Additions
~~~~~~~~~~~~~~~~~~~~~

Though none of the following are absolute requirements, they are all
recommended for use with the Python bindings. In some cases these
recommendations refer to which version(s) of CPython to use the bindings
with, while others refer to third party modules which provide a
significant advantage in some way.

#. If possible, use Python 3 instead of 2.
#. Favour a more recent version of Python since even 3.4 is due to reach
   EOL soon. In production systems and services, Python 3.6 should be
   robust enough to be relied on.
#. If possible add the following Python modules which are not part of
   the standard library:
   `Requests <http://docs.python-requests.org/en/latest/index.html>`__,
   `Cython <https://cython.org/>`__,
   `Pendulum <https://pendulum.eustace.io/>`__ and
   `hkp4py <https://github.com/Selfnet/hkp4py>`__.

Chances are quite high that at least the first one and maybe two of
those will already be installed.

Note that, as with Cython, some of advanced use case scenarios will
bring with them additional requirements. Most of these will be fairly
well known and commonly installed ones, however, which are in many cases
likely to have already been installed on many systems or be familiar to
Python programmers.

Installation
------------

Installing the Python bindings is effectively achieved by compiling and
installing GPGME itself.

Once SWIG is installed with Python and all the dependencies for GPGME
are installed you only need to confirm that the version(s) of Python you
want the bindings installed for are in your ``$PATH``.

By default GPGME will attempt to install the bindings for the most
recent or highest version number of Python 2 and Python 3 it detects in
``$PATH``. It specifically checks for the ``python`` and ``python3``
executables first and then checks for specific version numbers.

For Python 2 it checks for these executables in this order: ``python``,
``python2`` and ``python2.7``.

For Python 3 it checks for these executables in this order: ``python3``,
``python3.7``, ``python3.6``, ``python3.5`` and ``python3.4``. [2]_

On systems where ``python`` is actually ``python3`` and not ``python2``
it may be possible that ``python2`` may be overlooked, but there have
been no reports of that actually occurring as yet.

In the three months or so since the release of Python 3.7.0 there has
been extensive testing and work with these bindings with no issues
specifically relating to the new version of Python or any of the new
features of either the language or the bindings. This has also been the
case with Python 3.7.1rc1. With that in mind and given the release of
Python 3.7.1 is scheduled for around the same time as GPGME 1.12.0, the
order of preferred Python versions has been changed to move Python 3.7
ahead of Python 3.6.

.. _install-gpgme:

Installing GPGME
~~~~~~~~~~~~~~~~

See the GPGME ``README`` file for details of how to install GPGME from
source.

.. _snafu:

Known Issues
------------

There are a few known issues with the current build process and the
Python bindings. For the most part these are easily addressed should
they be encountered.

.. _snafu-a-swig-of-this-builds-character:

Breaking Builds
~~~~~~~~~~~~~~~

Occasionally when installing GPGME with the Python bindings included it
may be observed that the ``make`` portion of that process induces a
large very number of warnings and, eventually errors which end that part
of the build process. Yet following that with ``make check`` and
``make install`` appears to work seamlessly.

The cause of this is related to the way SWIG needs to be called to
dynamically generate the C bindings for GPGME in the first place. So the
entire process will always produce ``lang/python/python2-gpg/`` and
``lang/python/python3-gpg/`` directories. These should contain the build
output generated during compilation, including the complete bindings and
module installed into ``site-packages``.

Occasionally the errors in the early part or some other conflict (e.g.
not installing as **root** or **su**) may result in nothing being
installed to the relevant ``site-packages`` directory and the build
directory missing a lot of expected files. Even when this occurs, the
solution is actually quite simple and will always work.

That solution is simply to run the following commands as either the
**root** user or prepended with ``sudo -H``\  [3]_ in the
``lang/python/`` directory:

.. code:: shell

   /path/to/pythonX.Y setup.py build
   /path/to/pythonX.Y setup.py build
   /path/to/pythonX.Y setup.py install

Yes, the build command does need to be run twice. Yes, you still need to
run the potentially failing or incomplete steps during the
``configure``, ``make`` and ``make install`` steps with installing
GPGME. This is because those steps generate a lot of essential files
needed, both by and in order to create, the bindings (including both the
``setup.py`` and ``gpgme.h`` files).

#. IMPORTANT Note

   If specifying a selected number of languages to create bindings for,
   try to leave Python last. Currently the majority of the other
   language bindings are also preceding Python of either version when
   listed alphabetically and so that just happens by default currently.

   If Python is set to precede one of the other languages then it is
   possible that the errors described here may interrupt the build
   process before generating bindings for those other languages. In
   these cases it may be preferable to configure all preferred language
   bindings separately with alternative ``configure`` steps for GPGME
   using the ``--enable-languages=$LANGUAGE`` option.

.. _snafu-lessons-for-the-lazy:

Reinstalling Responsibly
~~~~~~~~~~~~~~~~~~~~~~~~

Regardless of whether you\'re installing for one version of Python or
several, there will come a point where reinstallation is required. With
most Python module installations, the installed files go into the
relevant site-packages directory and are then forgotten about. Then the
module is upgraded, the new files are copied over the old and that\'s
the end of the matter.

While the same is true of these bindings, there have been intermittent
issues observed on some platforms which have benefited significantly
from removing all the previous installations of the bindings before
installing the updated versions.

Removing the previous version(s) is simply a matter of changing to the
relevant ``site-packages`` directory for the version of Python in
question and removing the ``gpg/`` directory and any accompanying
egg-info files for that module.

In most cases this will require root or administration privileges on the
system, but the same is true of installing the module in the first
place.

.. _snafu-the-full-monty:

Multiple installations
~~~~~~~~~~~~~~~~~~~~~~

For a variety of reasons it may be either necessary or just preferable
to install the bindings to alternative installed Python versions which
meet the requirements of these bindings.

On POSIX systems this will generally be most simply achieved by running
the manual installation commands (build, build, install) as described in
the previous section for each Python installation the bindings need to
be installed to.

As per the SWIG documentation: the compilers, libraries and runtime used
to build GPGME and the Python Bindings **must** match those used to
compile Python itself, including the version number(s) (at least going
by major version numbers and probably minor numbers too).

On most POSIX systems, including OS X, this will very likely be the case
in most, if not all, cases.

Note that from GPGME
`1.12.1 <https://dev.gnupg.org/rMff6ff616aea6f59b7f2ce1176492850ecdf3851e>`__
the default installation installs to each version of Python it can find
first. That is that it will currently install for the first copies of
Python versions 2.7, 3.4, 3.5, 3.6, 3.7 and 3.8 (dev branch) that it
finds. Usually this will be in the same prefix as GPGME itself, but is
dictated by the ``$PATH`` when the installation is performed. The above
instructions can still be performed on other python installations which
the installer does not find, including alternative prefixes.

.. _snafu-runtime-not-funtime:

Won\'t Work With Windows
~~~~~~~~~~~~~~~~~~~~~~~~

There are semi-regular reports of Windows users having considerable
difficulty in installing and using the Python bindings at all. Very
often, possibly even always, these reports come from Cygwin users and/or
MinGW users and/or Msys2 users. Though not all of them have been
confirmed, it appears that these reports have also come from people who
installed Python using the Windows installer files from the `Python
website <https://python.org>`__ (i.e. mostly MSI installers, sometimes
self-extracting ``.exe`` files).

The Windows versions of Python are not built using Cygwin, MinGW or
Msys2; they\'re built using Microsoft Visual Studio. Furthermore the
version used is *considerably* more advanced than the version which
MinGW obtained a small number of files from many years ago in order to
be able to compile anything at all. Not only that, but there are changes
to the version of Visual Studio between some micro releases, though that
is is particularly the case with Python 2.7, since it has been kept
around far longer than it should have been.

There are two theoretical solutions to this issue:

#. Compile and install the GnuPG stack, including GPGME and the Python
   bibdings using the same version of Microsoft Visual Studio used by
   the Python Foundation to compile the version of Python installed.

   If there are multiple versions of Python then this will need to be
   done with each different version of Visual Studio used.

#. Compile and install Python using the same tools used by choice, such
   as MinGW or Msys2.

Do **not** use the official Windows installer for Python unless
following the first method.

In this type of situation it may even be for the best to accept that
there are less limitations on permissive software than free software and
simply opt to use a recent version of the Community Edition of Microsoft
Visual Studio to compile and build all of it, no matter what.

Investigations into the extent or the limitations of this issue are
ongoing.

.. _snafu-cffi:

CFFI is the Best™ and GPGME should use it instead of SWIG
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

There are many reasons for favouring
`CFFI <https://cffi.readthedocs.io/en/latest/overview.html>`__ and
proponents of it are quite happy to repeat these things as if all it
would take to switch from SWIG to CFFI is repeating that list as if it
were a new concept.

The fact is that there are things which Python\'s CFFI implementation
cannot handle in the GPGME C code. Beyond that there are features of
SWIG which are simply not available with CFFI at all. SWIG generates the
bindings to Python using the ``gpgme.h`` file, but that file is not a
single version shipped with each release, it too is generated when GPGME
is compiled.

CFFI is currently unable to adapt to such a potentially mutable
codebase. If there were some means of applying SWIG\'s dynamic code
generation to produce the Python/CFFI API modes of accessing the GPGME
libraries (or the source source code directly), but such a thing does
not exist yet either and it currently appears that work is needed in at
least one of CFFI\'s dependencies before any of this can be addressed.

So if you\'re a massive fan of CFFI; that\'s great, but if you want this
project to switch to CFFI then rather than just insisting that it
should, I\'d suggest you volunteer to bring CFFI up to the level this
project needs.

If you\'re actually seriously considering doing so, then I\'d suggest
taking the ``gpgme-tool.c`` file in the GPGME ``src/`` directory and
getting that to work with any of the CFFI API methods (not the ABI
methods, they\'ll work with pretty much anything). When you start
running into trouble with \"ifdefs\" then you\'ll know what sort of
things are lacking. That doesn\'t even take into account the amount of
work saved via SWIG\'s code generation techniques either.

.. _snafu-venv:

Virtualised Environments
~~~~~~~~~~~~~~~~~~~~~~~~

It is fairly common practice amongst Python developers to, as much as
possible, use packages like virtualenv to keep various things that are
to be installed from interfering with each other. Given how much of the
GPGME bindings is often at odds with the usual pythonic way of doing
things, it stands to reason that this would be called into question too.

As it happens the answer as to whether or not the bindings can be used
with virtualenv, the answer is both yes and no.

In general we recommend installing to the relevant path and matching
prefix of GPGME itself. Which means that when GPGME, and ideally the
rest of the GnuPG stack, is installed to a prefix like ``/usr/local`` or
``/opt/local`` then the bindings would need to be installed to the main
Python installation and not a virtualised abstraction. Attempts to
separate the two in the past have been known to cause weird and
intermittent errors ranging from minor annoyances to complete failures
in the build process.

As a consequence we only recommend building with and installing to the
main Python installations within the same prefix as GPGME is installed
to or which are found by GPGME\'s configuration stage immediately prior
to running the make commands. Which is exactly what the compiling and
installing process of GPGME does by default.

Once that is done, however, it appears that a copy of the compiled
module may be installed into a virtualenv of the same major and minor
version matching the build. Alternatively it is possible to utilise a
``sites.pth`` file in the ``site-packages/`` directory of a virtualenv
installation, which links back to the system installations corresponding
directory in order to import anything installed system wide. This may or
may not be appropriate on a case by case basis.

Though extensive testing of either of these options is not yet complete,
preliminary testing of them indicates that both are viable as long as
the main installation is complete. Which means that certain other
options normally restricted to virtual environments are also available,
including integration with pythonic test suites (e.g.
`pytest <https://docs.pytest.org/en/latest/index.html>`__) and other
large projects.

That said, it is worth reiterating the warning regarding non-standard
installations. If one were to attempt to install the bindings only to a
virtual environment without somehow also including the full GnuPG stack
(or enough of it as to include GPGME) then it is highly likely that
errors would be encountered at some point and more than a little likely
that the build process itself would break.

If a degree of separation from the main operating system is still
required in spite of these warnings, then consider other forms of
virtualisation. Either a virtual machine (e.g.
`VirtualBox <https://www.virtualbox.org/>`__), a hardware emulation
layer (e.g. `QEMU <https://www.qemu.org/>`__) or an application
container (e.g. `Docker <https://www.docker.com/why-docker>`__).

Finally it should be noted that the limited tests conducted thus far
have been using the ``virtualenv`` command in a new directory to create
the virtual python environment. As opposed to the standard ``python3
-m venv`` and it is possible that this will make a difference depending
on the system and version of Python in use. Another option is to run the
command ``python3 -m virtualenv /path/to/install/virtual/thingy``
instead.

.. _howto-fund-a-mental:

Fundamentals
============

Before we can get to the fun stuff, there are a few matters regarding
GPGME\'s design which hold true whether you\'re dealing with the C code
directly or these Python bindings.

.. _no-rest-for-the-wicked:

No REST
-------

The first part of which is or will be fairly blatantly obvious upon
viewing the first example, but it\'s worth reiterating anyway. That
being that this API is **not** a REST API. Nor indeed could it ever be
one.

Most, if not all, Python programmers (and not just Python programmers)
know how easy it is to work with a RESTful API. In fact they\'ve become
so popular that many other APIs attempt to emulate REST-like behaviour
as much as they are able. Right down to the use of JSON formatted output
to facilitate the use of their API without having to retrain developers.

This API does not do that. It would not be able to do that and also
provide access to the entire C API on which it\'s built. It does,
however, provide a very pythonic interface on top of the direct bindings
and it\'s this pythonic layer that this HOWTO deals with.

.. _howto-get-context:

Context
-------

One of the reasons which prevents this API from being RESTful is that
most operations require more than one instruction to the API to perform
the task. Sure, there are certain functions which can be performed
simultaneously, particularly if the result known or strongly anticipated
(e.g. selecting and encrypting to a key known to be in the public
keybox).

There are many more, however, which cannot be manipulated so readily:
they must be performed in a specific sequence and the result of one
operation has a direct bearing on the outcome of subsequent operations.
Not merely by generating an error either.

When dealing with this type of persistent state on the web, full of both
the RESTful and REST-like, it\'s most commonly referred to as a session.
In GPGME, however, it is called a context and every operation type has
one.

.. _howto-keys:

Working with keys
=================

.. _howto-keys-selection:

Key selection
-------------

Selecting keys to encrypt to or to sign with will be a common occurrence
when working with GPGMe and the means available for doing so are quite
simple.

They do depend on utilising a Context; however once the data is recorded
in another variable, that Context does not need to be the same one which
subsequent operations are performed.

The easiest way to select a specific key is by searching for that key\'s
key ID or fingerprint, preferably the full fingerprint without any
spaces in it. A long key ID will probably be okay, but is not advised
and short key IDs are already a problem with some being generated to
match specific patterns. It does not matter whether the pattern is upper
or lower case.

So this is the best method:

.. code:: python

   import gpg

   k = gpg.Context().keylist(pattern="258E88DCBD3CD44D8E7AB43F6ECB6AF0DEADBEEF")
   keys = list(k)

This is passable and very likely to be common:

.. code:: python

   import gpg

   k = gpg.Context().keylist(pattern="0x6ECB6AF0DEADBEEF")
   keys = list(k)

And this is a really bad idea:

.. code:: python

   import gpg

   k = gpg.Context().keylist(pattern="0xDEADBEEF")
   keys = list(k)

Alternatively it may be that the intention is to create a list of keys
which all match a particular search string. For instance all the
addresses at a particular domain, like this:

.. code:: python

   import gpg

   ncsc = gpg.Context().keylist(pattern="ncsc.mil")
   nsa = list(ncsc)

.. _howto-keys-counting:

Counting keys
~~~~~~~~~~~~~

Counting the number of keys in your public keybox (``pubring.kbx``), the
format which has superseded the old keyring format (``pubring.gpg`` and
``secring.gpg``), or the number of secret keys is a very simple task.

.. code:: python

   import gpg

   c = gpg.Context()
   seckeys = c.keylist(pattern=None, secret=True)
   pubkeys = c.keylist(pattern=None, secret=False)

   seclist = list(seckeys)
   secnum = len(seclist)

   publist = list(pubkeys)
   pubnum = len(publist)

   print("""
     Number of secret keys:  {0}
     Number of public keys:  {1}
   """.format(secnum, pubnum))

NOTE: The `Cython <#cython>`__ introduction in the `Advanced and
Experimental <#advanced-use>`__ section uses this same key counting code
with Cython to demonstrate some areas where Cython can improve
performance even with the bindings. Users with large public keyrings or
keyboxes, for instance, should consider these options if they are
comfortable with using Cython.

.. _howto-get-key:

Get key
-------

An alternative method of getting a single key via its fingerprint is
available directly within a Context with ``Context().get_key``. This is
the preferred method of selecting a key in order to modify it, sign or
certify it and for obtaining relevant data about a single key as a part
of other functions; when verifying a signature made by that key, for
instance.

By default this method will select public keys, but it can select secret
keys as well.

This first example demonstrates selecting the current key of Werner
Koch, which is due to expire at the end of 2018:

.. code:: python

   import gpg

   fingerprint = "80615870F5BAD690333686D0F2AD85AC1E42B367"
   key = gpg.Context().get_key(fingerprint)

Whereas this example demonstrates selecting the author\'s current key
with the ``secret`` key word argument set to ``True``:

.. code:: python

   import gpg

   fingerprint = "DB4724E6FA4286C92B4E55C4321E4E2373590E5D"
   key = gpg.Context().get_key(fingerprint, secret=True)

It is, of course, quite possible to select expired, disabled and revoked
keys with this function, but only to effectively display information
about those keys.

It is also possible to use both unicode or string literals and byte
literals with the fingerprint when getting a key in this way.

.. _howto-import-key:

Importing keys
--------------

Importing keys is possible with the ``key_import()`` method and takes
one argument which is a bytes literal object containing either the
binary or ASCII armoured key data for one or more keys.

The following example retrieves one or more keys from the SKS keyservers
via the web using the requests module. Since requests returns the
content as a bytes literal object, we can then use that directly to
import the resulting data into our keybox.

.. code:: python

   import gpg
   import os.path
   import requests

   c = gpg.Context()
   url = "https://sks-keyservers.net/pks/lookup"
   pattern = input("Enter the pattern to search for key or user IDs: ")
   payload = {"op": "get", "search": pattern}

   r = requests.get(url, verify=True, params=payload)
   result = c.key_import(r.content)

   if result is not None and hasattr(result, "considered") is False:
       print(result)
   elif result is not None and hasattr(result, "considered") is True:
       num_keys = len(result.imports)
       new_revs = result.new_revocations
       new_sigs = result.new_signatures
       new_subs = result.new_sub_keys
       new_uids = result.new_user_ids
       new_scrt = result.secret_imported
       nochange = result.unchanged
       print("""
     The total number of keys considered for import was:  {0}

        Number of keys revoked:  {1}
      Number of new signatures:  {2}
         Number of new subkeys:  {3}
        Number of new user IDs:  {4}
     Number of new secret keys:  {5}
      Number of unchanged keys:  {6}

     The key IDs for all considered keys were:
   """.format(num_keys, new_revs, new_sigs, new_subs, new_uids, new_scrt,
              nochange))
       for i in range(num_keys):
           print("{0}\n".format(result.imports[i].fpr))
   else:
       pass

NOTE: When searching for a key ID of any length or a fingerprint
(without spaces), the SKS servers require the the leading ``0x``
indicative of hexadecimal be included. Also note that the old short key
IDs (e.g. ``0xDEADBEEF``) should no longer be used due to the relative
ease by which such key IDs can be reproduced, as demonstrated by the
Evil32 Project in 2014 (which was subsequently exploited in 2016).

.. _import-protonmail:

Working with ProtonMail
~~~~~~~~~~~~~~~~~~~~~~~

Here is a variation on the example above which checks the constrained
ProtonMail keyserver for ProtonMail public keys.

.. code:: python

   import gpg
   import requests
   import sys

   print("""
   This script searches the ProtonMail key server for the specified key and
   imports it.
   """)

   c = gpg.Context(armor=True)
   url = "https://api.protonmail.ch/pks/lookup"
   ksearch = []

   if len(sys.argv) >= 2:
       keyterm = sys.argv[1]
   else:
       keyterm = input("Enter the key ID, UID or search string: ")

   if keyterm.count("@") == 2 and keyterm.startswith("@") is True:
       ksearch.append(keyterm[1:])
       ksearch.append(keyterm[1:])
       ksearch.append(keyterm[1:])
   elif keyterm.count("@") == 1 and keyterm.startswith("@") is True:
       ksearch.append("{0}@protonmail.com".format(keyterm[1:]))
       ksearch.append("{0}@protonmail.ch".format(keyterm[1:]))
       ksearch.append("{0}@pm.me".format(keyterm[1:]))
   elif keyterm.count("@") == 0:
       ksearch.append("{0}@protonmail.com".format(keyterm))
       ksearch.append("{0}@protonmail.ch".format(keyterm))
       ksearch.append("{0}@pm.me".format(keyterm))
   elif keyterm.count("@") == 2 and keyterm.startswith("@") is False:
       uidlist = keyterm.split("@")
       for uid in uidlist:
           ksearch.append("{0}@protonmail.com".format(uid))
           ksearch.append("{0}@protonmail.ch".format(uid))
           ksearch.append("{0}@pm.me".format(uid))
   elif keyterm.count("@") > 2:
       uidlist = keyterm.split("@")
       for uid in uidlist:
           ksearch.append("{0}@protonmail.com".format(uid))
           ksearch.append("{0}@protonmail.ch".format(uid))
           ksearch.append("{0}@pm.me".format(uid))
   else:
       ksearch.append(keyterm)

   for k in ksearch:
       payload = {"op": "get", "search": k}
       try:
           r = requests.get(url, verify=True, params=payload)
           if r.ok is True:
               result = c.key_import(r.content)
           elif r.ok is False:
               result = r.content
       except Exception as e:
           result = None

       if result is not None and hasattr(result, "considered") is False:
           print("{0} for {1}".format(result.decode(), k))
       elif result is not None and hasattr(result, "considered") is True:
           num_keys = len(result.imports)
           new_revs = result.new_revocations
           new_sigs = result.new_signatures
           new_subs = result.new_sub_keys
           new_uids = result.new_user_ids
           new_scrt = result.secret_imported
           nochange = result.unchanged
           print("""
   The total number of keys considered for import was:  {0}

   With UIDs wholely or partially matching the following string:

           {1}

      Number of keys revoked:  {2}
    Number of new signatures:  {3}
       Number of new subkeys:  {4}
      Number of new user IDs:  {5}
   Number of new secret keys:  {6}
    Number of unchanged keys:  {7}

   The key IDs for all considered keys were:
   """.format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt,
              nochange))
           for i in range(num_keys):
               print(result.imports[i].fpr)
           print("")
       elif result is None:
           print(e)

Both the above example,
`pmkey-import.py <../examples/howto/pmkey-import.py>`__, and a version
which prompts for an alternative GnuPG home directory,
`pmkey-import-alt.py <../examples/howto/pmkey-import-alt.py>`__, are
available with the other examples and are executable scripts.

Note that while the ProtonMail servers are based on the SKS servers,
their server is related more to their API and is not feature complete by
comparison to the servers in the SKS pool. One notable difference being
that the ProtonMail server does not permit non ProtonMail users to
update their own keys, which could be a vector for attacking ProtonMail
users who may not receive a key\'s revocation if it had been
compromised.

.. _import-hkp4py:

Importing with HKP for Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Performing the same tasks with the `hkp4py
module <https://github.com/Selfnet/hkp4py>`__ (available via PyPI) is
not too much different, but does provide a number of options of benefit
to end users. Not least of which being the ability to perform some
checks on a key before importing it or not. For instance it may be the
policy of a site or project to only import keys which have not been
revoked. The hkp4py module permits such checks prior to the importing of
the keys found.

.. code:: python

   import gpg
   import hkp4py
   import sys

   c = gpg.Context()
   server = hkp4py.KeyServer("hkps://hkps.pool.sks-keyservers.net")
   results = []

   if len(sys.argv) > 2:
       pattern = " ".join(sys.argv[1:])
   elif len(sys.argv) == 2:
       pattern = sys.argv[1]
   else:
       pattern = input("Enter the pattern to search for keys or user IDs: ")

   try:
       keys = server.search(pattern)
       print("Found {0} key(s).".format(len(keys)))
   except Exception as e:
       keys = []
       for logrus in pattern.split():
           if logrus.startswith("0x") is True:
               key = server.search(logrus)
           else:
               key = server.search("0x{0}".format(logrus))
           keys.append(key[0])
       print("Found {0} key(s).".format(len(keys)))

   for key in keys:
       import_result = c.key_import(key.key_blob)
       results.append(import_result)

   for result in results:
       if result is not None and hasattr(result, "considered") is False:
           print(result)
       elif result is not None and hasattr(result, "considered") is True:
           num_keys = len(result.imports)
           new_revs = result.new_revocations
           new_sigs = result.new_signatures
           new_subs = result.new_sub_keys
           new_uids = result.new_user_ids
           new_scrt = result.secret_imported
           nochange = result.unchanged
           print("""
   The total number of keys considered for import was:  {0}

      Number of keys revoked:  {1}
    Number of new signatures:  {2}
       Number of new subkeys:  {3}
      Number of new user IDs:  {4}
   Number of new secret keys:  {5}
    Number of unchanged keys:  {6}

   The key IDs for all considered keys were:
   """.format(num_keys, new_revs, new_sigs, new_subs, new_uids, new_scrt,
              nochange))
           for i in range(num_keys):
               print(result.imports[i].fpr)
           print("")
       else:
           pass

Since the hkp4py module handles multiple keys just as effectively as one
(``keys`` is a list of responses per matching key), the example above is
able to do a little bit more with the returned data before anything is
actually imported.

.. _import-protonmail-hkp4py:

Importing from ProtonMail with HKP for Python
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Though this can provide certain benefits even when working with
ProtonMail, the scope is somewhat constrained there due to the
limitations of the ProtonMail keyserver.

For instance, searching the SKS keyserver pool for the term \"gnupg\"
produces hundreds of results from any time the word appears in any part
of a user ID. Performing the same search on the ProtonMail keyserver
returns zero results, even though there are at least two test accounts
which include it as part of the username.

The cause of this discrepancy is the deliberate configuration of that
server by ProtonMail to require an exact match of the full email address
of the ProtonMail user whose key is being requested. Presumably this is
intended to reduce breaches of privacy of their users as an email
address must already be known before a key for that address can be
obtained.

#. Import from ProtonMail via HKP for Python Example no. 1

   The following script is available with the rest of the examples under
   the somewhat less than original name, ``pmkey-import-hkp.py``.

   .. code:: python

      import gpg
      import hkp4py
      import os.path
      import sys

      print("""
      This script searches the ProtonMail key server for the specified key and
      imports it.

      Usage:  pmkey-import-hkp.py [search strings]
      """)

      c = gpg.Context(armor=True)
      server = hkp4py.KeyServer("hkps://api.protonmail.ch")
      keyterms = []
      ksearch = []
      allkeys = []
      results = []
      paradox = []
      homeless = None

      if len(sys.argv) > 2:
          keyterms = sys.argv[1:]
      elif len(sys.argv) == 2:
          keyterm = sys.argv[1]
          keyterms.append(keyterm)
      else:
          key_term = input("Enter the key ID, UID or search string: ")
          keyterms = key_term.split()

      for keyterm in keyterms:
          if keyterm.count("@") == 2 and keyterm.startswith("@") is True:
              ksearch.append(keyterm[1:])
              ksearch.append(keyterm[1:])
              ksearch.append(keyterm[1:])
          elif keyterm.count("@") == 1 and keyterm.startswith("@") is True:
              ksearch.append("{0}@protonmail.com".format(keyterm[1:]))
              ksearch.append("{0}@protonmail.ch".format(keyterm[1:]))
              ksearch.append("{0}@pm.me".format(keyterm[1:]))
          elif keyterm.count("@") == 0:
              ksearch.append("{0}@protonmail.com".format(keyterm))
              ksearch.append("{0}@protonmail.ch".format(keyterm))
              ksearch.append("{0}@pm.me".format(keyterm))
          elif keyterm.count("@") == 2 and keyterm.startswith("@") is False:
              uidlist = keyterm.split("@")
              for uid in uidlist:
                  ksearch.append("{0}@protonmail.com".format(uid))
                  ksearch.append("{0}@protonmail.ch".format(uid))
                  ksearch.append("{0}@pm.me".format(uid))
          elif keyterm.count("@") > 2:
              uidlist = keyterm.split("@")
              for uid in uidlist:
                  ksearch.append("{0}@protonmail.com".format(uid))
                  ksearch.append("{0}@protonmail.ch".format(uid))
                  ksearch.append("{0}@pm.me".format(uid))
          else:
              ksearch.append(keyterm)

      for k in ksearch:
          print("Checking for key for: {0}".format(k))
          try:
              keys = server.search(k)
              if isinstance(keys, list) is True:
                  for key in keys:
                      allkeys.append(key)
                      try:
                          import_result = c.key_import(key.key_blob)
                      except Exception as e:
                          import_result = c.key_import(key.key)
              else:
                  paradox.append(keys)
                  import_result = None
          except Exception as e:
              import_result = None
          results.append(import_result)

      for result in results:
          if result is not None and hasattr(result, "considered") is False:
              print("{0} for {1}".format(result.decode(), k))
          elif result is not None and hasattr(result, "considered") is True:
              num_keys = len(result.imports)
              new_revs = result.new_revocations
              new_sigs = result.new_signatures
              new_subs = result.new_sub_keys
              new_uids = result.new_user_ids
              new_scrt = result.secret_imported
              nochange = result.unchanged
              print("""
      The total number of keys considered for import was:  {0}

      With UIDs wholely or partially matching the following string:

              {1}

         Number of keys revoked:  {2}
       Number of new signatures:  {3}
          Number of new subkeys:  {4}
         Number of new user IDs:  {5}
      Number of new secret keys:  {6}
       Number of unchanged keys:  {7}

      The key IDs for all considered keys were:
      """.format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt,
                 nochange))
              for i in range(num_keys):
                  print(result.imports[i].fpr)
              print("")
          elif result is None:
              pass

#. Import from ProtonMail via HKP for Python Example no. 2

   Like its counterpart above, this script can also be found with the
   rest of the examples, by the name pmkey-import-hkp-alt.py.

   With this script a modicum of effort has been made to treat anything
   passed as a ``homedir`` which either does not exist or which is not a
   directory, as also being a pssible user ID to check for. It\'s not
   guaranteed to pick up on all such cases, but it should cover most of
   them.

   .. code:: python

      import gpg
      import hkp4py
      import os.path
      import sys

      print("""
      This script searches the ProtonMail key server for the specified key and
      imports it.  Optionally enables specifying a different GnuPG home directory.

      Usage:  pmkey-import-hkp.py [homedir] [search string]
         or:  pmkey-import-hkp.py [search string]
      """)

      c = gpg.Context(armor=True)
      server = hkp4py.KeyServer("hkps://api.protonmail.ch")
      keyterms = []
      ksearch = []
      allkeys = []
      results = []
      paradox = []
      homeless = None

      if len(sys.argv) > 3:
          homedir = sys.argv[1]
          keyterms = sys.argv[2:]
      elif len(sys.argv) == 3:
          homedir = sys.argv[1]
          keyterm = sys.argv[2]
          keyterms.append(keyterm)
      elif len(sys.argv) == 2:
          homedir = ""
          keyterm = sys.argv[1]
          keyterms.append(keyterm)
      else:
          keyterm = input("Enter the key ID, UID or search string: ")
          homedir = input("Enter the GPG configuration directory path (optional): ")
          keyterms.append(keyterm)

      if len(homedir) == 0:
          homedir = None
          homeless = False

      if homedir is not None:
          if homedir.startswith("~"):
              if os.path.exists(os.path.expanduser(homedir)) is True:
                  if os.path.isdir(os.path.expanduser(homedir)) is True:
                      c.home_dir = os.path.realpath(os.path.expanduser(homedir))
                  else:
                      homeless = True
              else:
                  homeless = True
          elif os.path.exists(os.path.realpath(homedir)) is True:
              if os.path.isdir(os.path.realpath(homedir)) is True:
                  c.home_dir = os.path.realpath(homedir)
              else:
                  homeless = True
          else:
              homeless = True

      # First check to see if the homedir really is a homedir and if not, treat it as
      # a search string.
      if homeless is True:
          keyterms.append(homedir)
          c.home_dir = None
      else:
          pass

      for keyterm in keyterms:
          if keyterm.count("@") == 2 and keyterm.startswith("@") is True:
              ksearch.append(keyterm[1:])
              ksearch.append(keyterm[1:])
              ksearch.append(keyterm[1:])
          elif keyterm.count("@") == 1 and keyterm.startswith("@") is True:
              ksearch.append("{0}@protonmail.com".format(keyterm[1:]))
              ksearch.append("{0}@protonmail.ch".format(keyterm[1:]))
              ksearch.append("{0}@pm.me".format(keyterm[1:]))
          elif keyterm.count("@") == 0:
              ksearch.append("{0}@protonmail.com".format(keyterm))
              ksearch.append("{0}@protonmail.ch".format(keyterm))
              ksearch.append("{0}@pm.me".format(keyterm))
          elif keyterm.count("@") == 2 and keyterm.startswith("@") is False:
              uidlist = keyterm.split("@")
              for uid in uidlist:
                  ksearch.append("{0}@protonmail.com".format(uid))
                  ksearch.append("{0}@protonmail.ch".format(uid))
                  ksearch.append("{0}@pm.me".format(uid))
          elif keyterm.count("@") > 2:
              uidlist = keyterm.split("@")
              for uid in uidlist:
                  ksearch.append("{0}@protonmail.com".format(uid))
                  ksearch.append("{0}@protonmail.ch".format(uid))
                  ksearch.append("{0}@pm.me".format(uid))
          else:
              ksearch.append(keyterm)

      for k in ksearch:
          print("Checking for key for: {0}".format(k))
          try:
              keys = server.search(k)
              if isinstance(keys, list) is True:
                  for key in keys:
                      allkeys.append(key)
                      try:
                          import_result = c.key_import(key.key_blob)
                      except Exception as e:
                          import_result = c.key_import(key.key)
              else:
                  paradox.append(keys)
                  import_result = None
          except Exception as e:
              import_result = None
          results.append(import_result)

      for result in results:
          if result is not None and hasattr(result, "considered") is False:
              print("{0} for {1}".format(result.decode(), k))
          elif result is not None and hasattr(result, "considered") is True:
              num_keys = len(result.imports)
              new_revs = result.new_revocations
              new_sigs = result.new_signatures
              new_subs = result.new_sub_keys
              new_uids = result.new_user_ids
              new_scrt = result.secret_imported
              nochange = result.unchanged
              print("""
      The total number of keys considered for import was:  {0}

      With UIDs wholely or partially matching the following string:

              {1}

         Number of keys revoked:  {2}
       Number of new signatures:  {3}
          Number of new subkeys:  {4}
         Number of new user IDs:  {5}
      Number of new secret keys:  {6}
       Number of unchanged keys:  {7}

      The key IDs for all considered keys were:
      """.format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt,
                 nochange))
              for i in range(num_keys):
                  print(result.imports[i].fpr)
              print("")
          elif result is None:
              pass

.. _howto-export-key:

Exporting keys
--------------

Exporting keys remains a reasonably simple task, but has been separated
into three different functions for the OpenPGP cryptographic engine. Two
of those functions are for exporting public keys and the third is for
exporting secret keys.

.. _howto-export-public-key:

Exporting public keys
~~~~~~~~~~~~~~~~~~~~~

There are two methods of exporting public keys, both of which are very
similar to the other. The default method, ``key_export()``, will export
a public key or keys matching a specified pattern as normal. The
alternative, the ``key_export_minimal()`` method, will do the same thing
except producing a minimised output with extra signatures and third
party signatures or certifications removed.

.. code:: python

   import gpg
   import os.path
   import sys

   print("""
   This script exports one or more public keys.
   """)

   c = gpg.Context(armor=True)

   if len(sys.argv) >= 4:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = sys.argv[3]
   elif len(sys.argv) == 3:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = input("Enter the GPG configuration directory path (optional): ")
   elif len(sys.argv) == 2:
       keyfile = sys.argv[1]
       logrus = input("Enter the UID matching the key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")
   else:
       keyfile = input("Enter the path and filename to save the secret key to: ")
       logrus = input("Enter the UID matching the key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")

   if homedir.startswith("~"):
       if os.path.exists(os.path.expanduser(homedir)) is True:
           c.home_dir = os.path.expanduser(homedir)
       else:
           pass
   elif os.path.exists(homedir) is True:
       c.home_dir = homedir
   else:
       pass

   try:
       result = c.key_export(pattern=logrus)
   except:
       result = c.key_export(pattern=None)

   if result is not None:
       with open(keyfile, "wb") as f:
           f.write(result)
   else:
       pass

It should be noted that the result will only return ``None`` when a
search pattern has been entered, but has not matched any keys. When the
search pattern itself is set to ``None`` this triggers the exporting of
the entire public keybox.

.. code:: python

   import gpg
   import os.path
   import sys

   print("""
   This script exports one or more public keys in minimised form.
   """)

   c = gpg.Context(armor=True)

   if len(sys.argv) >= 4:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = sys.argv[3]
   elif len(sys.argv) == 3:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = input("Enter the GPG configuration directory path (optional): ")
   elif len(sys.argv) == 2:
       keyfile = sys.argv[1]
       logrus = input("Enter the UID matching the key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")
   else:
       keyfile = input("Enter the path and filename to save the secret key to: ")
       logrus = input("Enter the UID matching the key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")

   if homedir.startswith("~"):
       if os.path.exists(os.path.expanduser(homedir)) is True:
           c.home_dir = os.path.expanduser(homedir)
       else:
           pass
   elif os.path.exists(homedir) is True:
       c.home_dir = homedir
   else:
       pass

   try:
       result = c.key_export_minimal(pattern=logrus)
   except:
       result = c.key_export_minimal(pattern=None)

   if result is not None:
       with open(keyfile, "wb") as f:
           f.write(result)
   else:
       pass

.. _howto-export-secret-key:

Exporting secret keys
~~~~~~~~~~~~~~~~~~~~~

Exporting secret keys is, functionally, very similar to exporting public
keys; save for the invocation of ``pinentry`` via ``gpg-agent`` in order
to securely enter the key\'s passphrase and authorise the export.

The following example exports the secret key to a file which is then set
with the same permissions as the output files created by the command
line secret key export options.

.. code:: python

   import gpg
   import os
   import os.path
   import sys

   print("""
   This script exports one or more secret keys.

   The gpg-agent and pinentry are invoked to authorise the export.
   """)

   c = gpg.Context(armor=True)

   if len(sys.argv) >= 4:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = sys.argv[3]
   elif len(sys.argv) == 3:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = input("Enter the GPG configuration directory path (optional): ")
   elif len(sys.argv) == 2:
       keyfile = sys.argv[1]
       logrus = input("Enter the UID matching the secret key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")
   else:
       keyfile = input("Enter the path and filename to save the secret key to: ")
       logrus = input("Enter the UID matching the secret key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")

   if len(homedir) == 0:
       homedir = None
   elif homedir.startswith("~"):
       userdir = os.path.expanduser(homedir)
       if os.path.exists(userdir) is True:
           homedir = os.path.realpath(userdir)
       else:
           homedir = None
   else:
       homedir = os.path.realpath(homedir)

   if os.path.exists(homedir) is False:
       homedir = None
   else:
       if os.path.isdir(homedir) is False:
           homedir = None
       else:
           pass

   if homedir is not None:
       c.home_dir = homedir
   else:
       pass

   try:
       result = c.key_export_secret(pattern=logrus)
   except:
       result = c.key_export_secret(pattern=None)

   if result is not None:
       with open(keyfile, "wb") as f:
           f.write(result)
       os.chmod(keyfile, 0o600)
   else:
       pass

Alternatively the approach of the following script can be used. This
longer example saves the exported secret key(s) in files in the GnuPG
home directory, in addition to setting the file permissions as only
readable and writable by the user. It also exports the secret key(s)
twice in order to output both GPG binary (``.gpg``) and ASCII armoured
(``.asc``) files.

.. code:: python

   import gpg
   import os
   import os.path
   import subprocess
   import sys

   print("""
   This script exports one or more secret keys as both ASCII armored and binary
   file formats, saved in files within the user's GPG home directory.

   The gpg-agent and pinentry are invoked to authorise the export.
   """)

   if sys.platform == "win32":
       gpgconfcmd = "gpgconf.exe --list-dirs homedir"
   else:
       gpgconfcmd = "gpgconf --list-dirs homedir"

   a = gpg.Context(armor=True)
   b = gpg.Context()
   c = gpg.Context()

   if len(sys.argv) >= 4:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = sys.argv[3]
   elif len(sys.argv) == 3:
       keyfile = sys.argv[1]
       logrus = sys.argv[2]
       homedir = input("Enter the GPG configuration directory path (optional): ")
   elif len(sys.argv) == 2:
       keyfile = sys.argv[1]
       logrus = input("Enter the UID matching the secret key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")
   else:
       keyfile = input("Enter the filename to save the secret key to: ")
       logrus = input("Enter the UID matching the secret key(s) to export: ")
       homedir = input("Enter the GPG configuration directory path (optional): ")

   if len(homedir) == 0:
       homedir = None
   elif homedir.startswith("~"):
       userdir = os.path.expanduser(homedir)
       if os.path.exists(userdir) is True:
           homedir = os.path.realpath(userdir)
       else:
           homedir = None
   else:
       homedir = os.path.realpath(homedir)

   if os.path.exists(homedir) is False:
       homedir = None
   else:
       if os.path.isdir(homedir) is False:
           homedir = None
       else:
           pass

   if homedir is not None:
       c.home_dir = homedir
   else:
       pass

   if c.home_dir is not None:
       if c.home_dir.endswith("/"):
           gpgfile = "{0}{1}.gpg".format(c.home_dir, keyfile)
           ascfile = "{0}{1}.asc".format(c.home_dir, keyfile)
       else:
           gpgfile = "{0}/{1}.gpg".format(c.home_dir, keyfile)
           ascfile = "{0}/{1}.asc".format(c.home_dir, keyfile)
   else:
       if os.path.exists(os.environ["GNUPGHOME"]) is True:
           hd = os.environ["GNUPGHOME"]
       else:
           try:
               hd = subprocess.getoutput(gpgconfcmd)
           except:
               process = subprocess.Popen(gpgconfcmd.split(),
                                          stdout=subprocess.PIPE)
               procom = process.communicate()
               if sys.version_info[0] == 2:
                   hd = procom[0].strip()
               else:
                   hd = procom[0].decode().strip()
       gpgfile = "{0}/{1}.gpg".format(hd, keyfile)
       ascfile = "{0}/{1}.asc".format(hd, keyfile)

   try:
       a_result = a.key_export_secret(pattern=logrus)
       b_result = b.key_export_secret(pattern=logrus)
   except:
       a_result = a.key_export_secret(pattern=None)
       b_result = b.key_export_secret(pattern=None)

   if a_result is not None:
       with open(ascfile, "wb") as f:
           f.write(a_result)
       os.chmod(ascfile, 0o600)
   else:
       pass

   if b_result is not None:
       with open(gpgfile, "wb") as f:
           f.write(b_result)
       os.chmod(gpgfile, 0o600)
   else:
       pass

.. _howto-send-public-key:

Sending public keys to the SKS Keyservers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As with the previous section on importing keys, the ``hkp4py`` module
adds another option with exporting keys in order to send them to the
public keyservers.

The following example demonstrates how this may be done.

.. code:: python

   import gpg
   import hkp4py
   import os.path
   import sys

   print("""
   This script sends one or more public keys to the SKS keyservers and is
   essentially a slight variation on the export-key.py script.
   """)

   c = gpg.Context(armor=True)
   server = hkp4py.KeyServer("hkps://hkps.pool.sks-keyservers.net")

   if len(sys.argv) > 2:
       logrus = " ".join(sys.argv[1:])
   elif len(sys.argv) == 2:
       logrus = sys.argv[1]
   else:
       logrus = input("Enter the UID matching the key(s) to send: ")

   if len(logrus) > 0:
       try:
           export_result = c.key_export(pattern=logrus)
       except Exception as e:
           print(e)
           export_result = None
   else:
       export_result = c.key_export(pattern=None)

   if export_result is not None:
       try:
           try:
               send_result = server.add(export_result)
           except:
               send_result = server.add(export_result.decode())
           if send_result is not None:
               print(send_result)
           else:
               pass
       except Exception as e:
           print(e)
   else:
       pass

An expanded version of this script with additional functions for
specifying an alternative homedir location is in the examples directory
as ``send-key-to-keyserver.py``.

The ``hkp4py`` module appears to handle both string and byte literal
text data equally well, but the GPGME bindings deal primarily with byte
literal data only and so this script sends in that format first, then
tries the string literal form.

.. _howto-the-basics:

Basic Functions
===============

The most frequently called features of any cryptographic library will be
the most fundamental tasks for encryption software. In this section we
will look at how to programmatically encrypt data, decrypt it, sign it
and verify signatures.

.. _howto-basic-encryption:

Encryption
----------

Encrypting is very straight forward. In the first example below the
message, ``text``, is encrypted to a single recipient\'s key. In the
second example the message will be encrypted to multiple recipients.

.. _howto-basic-encryption-single:

Encrypting to one key
~~~~~~~~~~~~~~~~~~~~~

Once the the Context is set the main issues with encrypting data is
essentially reduced to key selection and the keyword arguments specified
in the ``gpg.Context().encrypt()`` method.

Those keyword arguments are: ``recipients``, a list of keys encrypted to
(covered in greater detail in the following section); ``sign``, whether
or not to sign the plaintext data, see subsequent sections on signing
and verifying signatures below (defaults to ``True``); ``sink``, to
write results or partial results to a secure sink instead of returning
it (defaults to ``None``); ``passphrase``, only used when utilising
symmetric encryption (defaults to ``None``); ``always_trust``, used to
override the trust model settings for recipient keys (defaults to
``False``); ``add_encrypt_to``, utilises any preconfigured
``encrypt-to`` or ``default-key`` settings in the user\'s ``gpg.conf``
file (defaults to ``False``); ``prepare``, prepare for encryption
(defaults to ``False``); ``expect_sign``, prepare for signing (defaults
to ``False``); ``compress``, compresses the plaintext prior to
encryption (defaults to ``True``).

.. code:: python

   import gpg

   a_key = "0x12345678DEADBEEF"
   text = b"""Some text to test with.

   Since the text in this case must be bytes, it is most likely that
   the input form will be a separate file which is opened with "rb"
   as this is the simplest method of obtaining the correct data format.
   """

   c = gpg.Context(armor=True)
   rkey = list(c.keylist(pattern=a_key, secret=False))
   ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, sign=False)

   with open("secret_plans.txt.asc", "wb") as afile:
       afile.write(ciphertext)

Though this is even more likely to be used like this; with the plaintext
input read from a file, the recipient keys used for encryption
regardless of key trust status and the encrypted output also encrypted
to any preconfigured keys set in the ``gpg.conf`` file:

.. code:: python

   import gpg

   a_key = "0x12345678DEADBEEF"

   with open("secret_plans.txt", "rb") as afile:
       text = afile.read()

   c = gpg.Context(armor=True)
   rkey = list(c.keylist(pattern=a_key, secret=False))
   ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, sign=True,
                                               always_trust=True,
                                               add_encrypt_to=True)

   with open("secret_plans.txt.asc", "wb") as afile:
       afile.write(ciphertext)

If the ``recipients`` parameter is empty then the plaintext is encrypted
symmetrically. If no ``passphrase`` is supplied as a parameter or via a
callback registered with the ``Context()`` then an out-of-band prompt
for the passphrase via pinentry will be invoked.

.. _howto-basic-encryption-multiple:

Encrypting to multiple keys
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Encrypting to multiple keys essentially just expands upon the key
selection process and the recipients from the previous examples.

The following example encrypts a message (``text``) to everyone with an
email address on the ``gnupg.org`` domain, [4]_ but does *not* encrypt
to a default key or other key which is configured to normally encrypt
to.

.. code:: python

   import gpg

   text = b"""Oh look, another test message.

   The same rules apply as with the previous example and more likely
   than not, the message will actually be drawn from reading the
   contents of a file or, maybe, from entering data at an input()
   prompt.

   Since the text in this case must be bytes, it is most likely that
   the input form will be a separate file which is opened with "rb"
   as this is the simplest method of obtaining the correct data
   format.
   """

   c = gpg.Context(armor=True)
   rpattern = list(c.keylist(pattern="@gnupg.org", secret=False))
   logrus = []

   for i in range(len(rpattern)):
       if rpattern[i].can_encrypt == 1:
           logrus.append(rpattern[i])

   ciphertext, result, sign_result = c.encrypt(text, recipients=logrus,
                                               sign=False, always_trust=True)

   with open("secret_plans.txt.asc", "wb") as afile:
       afile.write(ciphertext)

All it would take to change the above example to sign the message and
also encrypt the message to any configured default keys would be to
change the ``c.encrypt`` line to this:

.. code:: python

   ciphertext, result, sign_result = c.encrypt(text, recipients=logrus,
                                               always_trust=True,
                                               add_encrypt_to=True)

The only keyword arguments requiring modification are those for which
the default values are changing. The default value of ``sign`` is
``True``, the default of ``always_trust`` is ``False``, the default of
``add_encrypt_to`` is ``False``.

If ``always_trust`` is not set to ``True`` and any of the recipient keys
are not trusted (e.g. not signed or locally signed) then the encryption
will raise an error. It is possible to mitigate this somewhat with
something more like this:

.. code:: python

   import gpg

   with open("secret_plans.txt.asc", "rb") as afile:
       text = afile.read()

   c = gpg.Context(armor=True)
   rpattern = list(c.keylist(pattern="@gnupg.org", secret=False))
   logrus = []

   for i in range(len(rpattern)):
       if rpattern[i].can_encrypt == 1:
           logrus.append(rpattern[i])

       try:
           ciphertext, result, sign_result = c.encrypt(text, recipients=logrus,
                                                       add_encrypt_to=True)
       except gpg.errors.InvalidRecipients as e:
           for i in range(len(e.recipients)):
               for n in range(len(logrus)):
                   if logrus[n].fpr == e.recipients[i].fpr:
                       logrus.remove(logrus[n])
                   else:
                       pass
           try:
               ciphertext, result, sign_result = c.encrypt(text,
                                                           recipients=logrus,
                                                           add_encrypt_to=True)
               with open("secret_plans.txt.asc", "wb") as afile:
                   afile.write(ciphertext)
           except:
               pass

This will attempt to encrypt to all the keys searched for, then remove
invalid recipients if it fails and try again.

.. _howto-basic-decryption:

Decryption
----------

Decrypting something encrypted to a key in one\'s secret keyring is
fairly straight forward.

In this example code, however, preconfiguring either ``gpg.Context()``
or ``gpg.core.Context()`` as ``c`` is unnecessary because there is no
need to modify the Context prior to conducting the decryption and since
the Context is only used once, setting it to ``c`` simply adds lines for
no gain.

.. code:: python

   import gpg

   ciphertext = input("Enter path and filename of encrypted file: ")
   newfile = input("Enter path and filename of file to save decrypted data to: ")

   with open(ciphertext, "rb") as cfile:
       try:
           plaintext, result, verify_result = gpg.Context().decrypt(cfile)
       except gpg.errors.GPGMEError as e:
           plaintext = None
           print(e)

   if plaintext is not None:
       with open(newfile, "wb") as nfile:
           nfile.write(plaintext)
       else:
           pass

The data available in ``plaintext`` in this example is the decrypted
content as a byte object, the recipient key IDs and algorithms in
``result`` and the results of verifying any signatures of the data in
``verify_result``.

.. _howto-basic-signing:

Signing text and files
----------------------

The following sections demonstrate how to specify keys to sign with.

.. _howto-basic-signing-signers:

Signing key selection
~~~~~~~~~~~~~~~~~~~~~

By default GPGME and the Python bindings will use the default key
configured for the user invoking the GPGME API. If there is no default
key specified and there is more than one secret key available it may be
necessary to specify the key or keys with which to sign messages and
files.

.. code:: python

   import gpg

   logrus = input("Enter the email address or string to match signing keys to: ")
   hancock = gpg.Context().keylist(pattern=logrus, secret=True)
   sig_src = list(hancock)

The signing examples in the following sections include the explicitly
designated ``signers`` parameter in two of the five examples; once where
the resulting signature would be ASCII armoured and once where it would
not be armoured.

While it would be possible to enter a key ID or fingerprint here to
match a specific key, it is not possible to enter two fingerprints and
match two keys since the patten expects a string, bytes or None and not
a list. A string with two fingerprints won\'t match any single key.

.. _howto-basic-signing-normal:

Normal or default signing messages or files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The normal or default signing process is essentially the same as is most
often invoked when also encrypting a message or file. So when the
encryption component is not utilised, the result is to produce an
encoded and signed output which may or may not be ASCII armoured and
which may or may not also be compressed.

By default compression will be used unless GnuPG detects that the
plaintext is already compressed. ASCII armouring will be determined
according to the value of ``gpg.Context().armor``.

The compression algorithm is selected in much the same way as the
symmetric encryption algorithm or the hash digest algorithm is when
multiple keys are involved; from the preferences saved into the key
itself or by comparison with the preferences with all other keys
involved.

.. code:: python

   import gpg

   text0 = """Declaration of ... something.

   """
   text = text0.encode()

   c = gpg.Context(armor=True, signers=sig_src)
   signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL)

   with open("/path/to/statement.txt.asc", "w") as afile:
       afile.write(signed_data.decode())

Though everything in this example is accurate, it is more likely that
reading the input data from another file and writing the result to a new
file will be performed more like the way it is done in the next example.
Even if the output format is ASCII armoured.

.. code:: python

   import gpg

   with open("/path/to/statement.txt", "rb") as tfile:
       text = tfile.read()

   c = gpg.Context()
   signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL)

   with open("/path/to/statement.txt.sig", "wb") as afile:
       afile.write(signed_data)

.. _howto-basic-signing-detached:

Detached signing messages and files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Detached signatures will often be needed in programmatic uses of GPGME,
either for signing files (e.g. tarballs of code releases) or as a
component of message signing (e.g. PGP/MIME encoded email).

.. code:: python

   import gpg

   text0 = """Declaration of ... something.

   """
   text = text0.encode()

   c = gpg.Context(armor=True)
   signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH)

   with open("/path/to/statement.txt.asc", "w") as afile:
       afile.write(signed_data.decode())

As with normal signatures, detached signatures are best handled as byte
literals, even when the output is ASCII armoured.

.. code:: python

   import gpg

   with open("/path/to/statement.txt", "rb") as tfile:
       text = tfile.read()

   c = gpg.Context(signers=sig_src)
   signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH)

   with open("/path/to/statement.txt.sig", "wb") as afile:
       afile.write(signed_data)

.. _howto-basic-signing-clear:

Clearsigning messages or text
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Though PGP/in-line messages are no longer encouraged in favour of
PGP/MIME, there is still sometimes value in utilising in-line
signatures. This is where clear-signed messages or text is of value.

.. code:: python

   import gpg

   text0 = """Declaration of ... something.

   """
   text = text0.encode()

   c = gpg.Context()
   signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR)

   with open("/path/to/statement.txt.asc", "w") as afile:
       afile.write(signed_data.decode())

In spite of the appearance of a clear-signed message, the data handled
by GPGME in signing it must still be byte literals.

.. code:: python

   import gpg

   with open("/path/to/statement.txt", "rb") as tfile:
       text = tfile.read()

   c = gpg.Context()
   signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR)

   with open("/path/to/statement.txt.asc", "wb") as afile:
       afile.write(signed_data)

.. _howto-basic-verification:

Signature verification
----------------------

Essentially there are two principal methods of verification of a
signature. The first of these is for use with the normal or default
signing method and for clear-signed messages. The second is for use with
files and data with detached signatures.

The following example is intended for use with the default signing
method where the file was not ASCII armoured:

.. code:: python

   import gpg
   import time

   filename = "statement.txt"
   gpg_file = "statement.txt.gpg"

   c = gpg.Context()

   try:
       data, result = c.verify(open(gpg_file))
       verified = True
   except gpg.errors.BadSignatures as e:
       verified = False
       print(e)

   if verified is True:
       for i in range(len(result.signatures)):
           sign = result.signatures[i]
           print("""Good signature from:
   {0}
   with key {1}
   made at {2}
   """.format(c.get_key(sign.fpr).uids[0].uid, sign.fpr,
              time.ctime(sign.timestamp)))
   else:
       pass

Whereas this next example, which is almost identical would work with
normal ASCII armoured files and with clear-signed files:

.. code:: python

   import gpg
   import time

   filename = "statement.txt"
   asc_file = "statement.txt.asc"

   c = gpg.Context()

   try:
       data, result = c.verify(open(asc_file))
       verified = True
   except gpg.errors.BadSignatures as e:
       verified = False
       print(e)

   if verified is True:
       for i in range(len(result.signatures)):
           sign = result.signatures[i]
           print("""Good signature from:
   {0}
   with key {1}
   made at {2}
   """.format(c.get_key(sign.fpr).uids[0].uid, sign.fpr,
              time.ctime(sign.timestamp)))
   else:
       pass

In both of the previous examples it is also possible to compare the
original data that was signed against the signed data in ``data`` to see
if it matches with something like this:

.. code:: python

   with open(filename, "rb") as afile:
       text = afile.read()

   if text == data:
       print("Good signature.")
   else:
       pass

The following two examples, however, deal with detached signatures. With
his method of verification the data that was signed does not get
returned since it is already being explicitly referenced in the first
argument of ``c.verify``. So ``data`` is ``None`` and only the
information in ``result`` is available.

.. code:: python

   import gpg
   import time

   filename = "statement.txt"
   sig_file = "statement.txt.sig"

   c = gpg.Context()

   try:
       data, result = c.verify(open(filename), open(sig_file))
       verified = True
   except gpg.errors.BadSignatures as e:
       verified = False
       print(e)

   if verified is True:
       for i in range(len(result.signatures)):
           sign = result.signatures[i]
           print("""Good signature from:
   {0}
   with key {1}
   made at {2}
   """.format(c.get_key(sign.fpr).uids[0].uid, sign.fpr,
              time.ctime(sign.timestamp)))
   else:
       pass

.. code:: python

   import gpg
   import time

   filename = "statement.txt"
   asc_file = "statement.txt.asc"

   c = gpg.Context()

   try:
       data, result = c.verify(open(filename), open(asc_file))
       verified = True
   except gpg.errors.BadSignatures as e:
       verified = False
       print(e)

   if verified is True:
       for i in range(len(result.signatures)):
           sign = result.signatures[i]
           print("""Good signature from:
   {0}
   with key {1}
   made at {2}
   """.format(c.get_key(sign.fpr).uids[0].uid, sign.fpr,
              time.ctime(sign.timestamp)))
   else:
       pass

.. _key-generation:

Creating keys and subkeys
=========================

The one thing, aside from GnuPG itself, that GPGME depends on, of
course, is the keys themselves. So it is necessary to be able to
generate them and modify them by adding subkeys, revoking or disabling
them, sometimes deleting them and doing the same for user IDs.

In the following examples a key will be created for the world\'s
greatest secret agent, Danger Mouse. Since Danger Mouse is a secret
agent he needs to be able to protect information to ``SECRET`` level
clearance, so his keys will be 3072-bit keys.

The pre-configured ``gpg.conf`` file which sets cipher, digest and other
preferences contains the following configuration parameters:

.. code:: conf

   expert
   allow-freeform-uid
   allow-secret-key-import
   trust-model tofu+pgp
   tofu-default-policy unknown
   enable-large-rsa
   enable-dsa2
   cert-digest-algo SHA512
   default-preference-list TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1 ZLIB BZIP2 ZIP Uncompressed
   personal-cipher-preferences TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES
   personal-digest-preferences SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1
   personal-compress-preferences ZLIB BZIP2 ZIP Uncompressed

.. _keygen-primary:

Primary key
-----------

Generating a primary key uses the ``create_key`` method in a Context. It
contains multiple arguments and keyword arguments, including:
``userid``, ``algorithm``, ``expires_in``, ``expires``, ``sign``,
``encrypt``, ``certify``, ``authenticate``, ``passphrase`` and
``force``. The defaults for all of those except ``userid``,
``algorithm``, ``expires_in``, ``expires`` and ``passphrase`` is
``False``. The defaults for ``algorithm`` and ``passphrase`` is
``None``. The default for ``expires_in`` is ``0``. The default for
``expires`` is ``True``. There is no default for ``userid``.

If ``passphrase`` is left as ``None`` then the key will not be generated
with a passphrase, if ``passphrase`` is set to a string then that will
be the passphrase and if ``passphrase`` is set to ``True`` then
gpg-agent will launch pinentry to prompt for a passphrase. For the sake
of convenience, these examples will keep ``passphrase`` set to ``None``.

.. code:: python

   import gpg

   c = gpg.Context()

   c.home_dir = "~/.gnupg-dm"
   userid = "Danger Mouse <dm@secret.example.net>"

   dmkey = c.create_key(userid, algorithm="rsa3072", expires_in=31536000,
                        sign=True, certify=True)

One thing to note here is the use of setting the ``c.home_dir``
parameter. This enables generating the key or keys in a different
location. In this case to keep the new key data created for this example
in a separate location rather than adding it to existing and active key
store data. As with the default directory, ``~/.gnupg``, any temporary
or separate directory needs the permissions set to only permit access by
the directory owner. On posix systems this means setting the directory
permissions to 700.

The ``temp-homedir-config.py`` script in the HOWTO examples directory
will create an alternative homedir with these configuration options
already set and the correct directory and file permissions.

The successful generation of the key can be confirmed via the returned
``GenkeyResult`` object, which includes the following data:

.. code:: python

   print("""
    Fingerprint:  {0}
    Primary Key:  {1}
     Public Key:  {2}
     Secret Key:  {3}
    Sub Key:  {4}
   User IDs:  {5}
   """.format(dmkey.fpr, dmkey.primary, dmkey.pubkey, dmkey.seckey, dmkey.sub,
              dmkey.uid))

Alternatively the information can be confirmed using the command line
program:

.. code:: shell

   bash-4.4$ gpg --homedir ~/.gnupg-dm -K
   ~/.gnupg-dm/pubring.kbx
   ----------------------
   sec   rsa3072 2018-03-15 [SC] [expires: 2019-03-15]
     177B7C25DB99745EE2EE13ED026D2F19E99E63AA
   uid           [ultimate] Danger Mouse <dm@secret.example.net>

   bash-4.4$

As with generating keys manually, to preconfigure expanded preferences
for the cipher, digest and compression algorithms, the ``gpg.conf`` file
must contain those details in the home directory in which the new key is
being generated. I used a cut down version of my own ``gpg.conf`` file
in order to be able to generate this:

.. code:: shell

   bash-4.4$ gpg --homedir ~/.gnupg-dm --edit-key 177B7C25DB99745EE2EE13ED026D2F19E99E63AA showpref quit
   Secret key is available.

   sec  rsa3072/026D2F19E99E63AA
        created: 2018-03-15  expires: 2019-03-15  usage: SC
        trust: ultimate      validity: ultimate
   [ultimate] (1). Danger Mouse <dm@secret.example.net>

   [ultimate] (1). Danger Mouse <dm@secret.example.net>
        Cipher: TWOFISH, CAMELLIA256, AES256, CAMELLIA192, AES192, CAMELLIA128, AES, BLOWFISH, IDEA, CAST5, 3DES
        Digest: SHA512, SHA384, SHA256, SHA224, RIPEMD160, SHA1
        Compression: ZLIB, BZIP2, ZIP, Uncompressed
        Features: MDC, Keyserver no-modify

   bash-4.4$

.. _keygen-subkeys:

Subkeys
-------

Adding subkeys to a primary key is fairly similar to creating the
primary key with the ``create_subkey`` method. Most of the arguments are
the same, but not quite all. Instead of the ``userid`` argument there is
now a ``key`` argument for selecting which primary key to add the subkey
to.

In the following example an encryption subkey will be added to the
primary key. Since Danger Mouse is a security conscious secret agent,
this subkey will only be valid for about six months, half the length of
the primary key.

.. code:: python

   import gpg

   c = gpg.Context()
   c.home_dir = "~/.gnupg-dm"

   key = c.get_key(dmkey.fpr, secret=True)
   dmsub = c.create_subkey(key, algorithm="rsa3072", expires_in=15768000,
                           encrypt=True)

As with the primary key, the results here can be checked with:

.. code:: python

   print("""
    Fingerprint:  {0}
    Primary Key:  {1}
     Public Key:  {2}
     Secret Key:  {3}
    Sub Key:  {4}
   User IDs:  {5}
   """.format(dmsub.fpr, dmsub.primary, dmsub.pubkey, dmsub.seckey, dmsub.sub,
              dmsub.uid))

As well as on the command line with:

.. code:: shell

   bash-4.4$ gpg --homedir ~/.gnupg-dm -K
   ~/.gnupg-dm/pubring.kbx
   ----------------------
   sec   rsa3072 2018-03-15 [SC] [expires: 2019-03-15]
     177B7C25DB99745EE2EE13ED026D2F19E99E63AA
   uid           [ultimate] Danger Mouse <dm@secret.example.net>
   ssb   rsa3072 2018-03-15 [E] [expires: 2018-09-13]

   bash-4.4$

.. _keygen-uids:

User IDs
--------

.. _keygen-uids-add:

Adding User IDs
~~~~~~~~~~~~~~~

By comparison to creating primary keys and subkeys, adding a new user ID
to an existing key is much simpler. The method used to do this is
``key_add_uid`` and the only arguments it takes are for the ``key`` and
the new ``uid``.

.. code:: python

   import gpg

   c = gpg.Context()
   c.home_dir = "~/.gnupg-dm"

   dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA"
   key = c.get_key(dmfpr, secret=True)
   uid = "Danger Mouse <danger.mouse@secret.example.net>"

   c.key_add_uid(key, uid)

Unsurprisingly the result of this is:

.. code:: shell

   bash-4.4$ gpg --homedir ~/.gnupg-dm -K
   ~/.gnupg-dm/pubring.kbx
   ----------------------
   sec   rsa3072 2018-03-15 [SC] [expires: 2019-03-15]
     177B7C25DB99745EE2EE13ED026D2F19E99E63AA
   uid           [ultimate] Danger Mouse <danger.mouse@secret.example.net>
   uid           [ultimate] Danger Mouse <dm@secret.example.net>
   ssb   rsa3072 2018-03-15 [E] [expires: 2018-09-13]

   bash-4.4$

.. _keygen-uids-revoke:

Revoking User IDs
~~~~~~~~~~~~~~~~~

Revoking a user ID is a fairly similar process, except that it uses the
``key_revoke_uid`` method.

.. code:: python

   import gpg

   c = gpg.Context()
   c.home_dir = "~/.gnupg-dm"

   dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA"
   key = c.get_key(dmfpr, secret=True)
   uid = "Danger Mouse <danger.mouse@secret.example.net>"

   c.key_revoke_uid(key, uid)

.. _key-sign:

Key certification
-----------------

Since key certification is more frequently referred to as key signing,
the method used to perform this function is ``key_sign``.

The ``key_sign`` method takes four arguments: ``key``, ``uids``,
``expires_in`` and ``local``. The default value of ``uids`` is ``None``
and which results in all user IDs being selected. The default value of
both ``expires_in`` and ``local`` is ``False``; which results in the
signature never expiring and being able to be exported.

The ``key`` is the key being signed rather than the key doing the
signing. To change the key doing the signing refer to the signing key
selection above for signing messages and files.

If the ``uids`` value is not ``None`` then it must either be a string to
match a single user ID or a list of strings to match multiple user IDs.
In this case the matching of those strings must be precise and it is
case sensitive.

To sign Danger Mouse\'s key for just the initial user ID with a
signature which will last a little over a month, do this:

.. code:: python

   import gpg

   c = gpg.Context()
   uid = "Danger Mouse <dm@secret.example.net>"

   dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA"
   key = c.get_key(dmfpr, secret=True)
   c.key_sign(key, uids=uid, expires_in=2764800)

.. _key-sign-verify:

Verifying key certifications
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: python

   import gpg
   import time

   c = gpg.Context()
   dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA"
   keys = list(c.keylist(pattern=dmuid, mode=gpg.constants.keylist.mode.SIGS))
   key = keys[0]

   for user in key.uids:
       for sig in user.signatures:
           print("0x{0}".format(sig.keyid), "", time.ctime(sig.timestamp), "",
                 sig.uid)

Which for Danger Mouse displays the following:

::

   0x92E3F6115435C65A  Thu Mar 15 13:17:44 2018  Danger Mouse <dm@secret.example.net>
   0x321E4E2373590E5D  Mon Nov 26 12:46:05 2018  Ben McGinnes <ben@adversary.org>

The two key signatures listed are for the self-certification of Danger
Mouse\'s key made when the key was created in March, 2018; and the
second is a signature made by the author and set to expire at the end of
the year. Note that the second signature was made with the following
code (including the preceding code to display the output of the
certifications or key signatures):

.. code:: python

   import gpg
   import math
   import pendulum
   import time

   hd = "/home/dm/.gnupg"
   c = gpg.Context()
   d = gpg.Context(home_dir=hd)
   dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA"
   dmuid = "Danger Mouse <dm@secret.example.net>"
   dkeys = list(c.keylist(pattern=dmuid))
   dmkey = dkeys[0]

   c.key_import(d.key_export(pattern=None))

   tp = pendulum.period(pendulum.now(tz="local"), pendulum.datetime(2019, 1, 1))
   ts = tp.total_seconds()
   total_secs = math.ceil(ts)
   c.key_sign(dmkey, uids=dmuid, expires_in=total_secs)

   d.key_import(c.key_export(pattern=dmuid))
   keys = list(c.keylist(pattern=dmuid, mode=gpg.constants.keylist.mode.SIGS))
   key = keys[0]

   for user in key.uids:
       for sig in user.signatures:
           print("0x{0}".format(sig.keyid), "", time.ctime(sig.timestamp), "",
                 sig.uid)

Note that this final code block includes the use of a module which is
*not* part of Python\'s standard library, the `pendulum
module <https://pendulum.eustace.io/>`__. Unlike the standard datetime
module, pendulum makes working with dates and times significantly easier
in Python; just as the requests module makes working with HTTP and HTTPS
easier than the builtin modules do.

Though neither requests nor pendulum are required modules for using the
GPGME Python bindings, they are both highly recommended more generally.

.. _advanced-use:

Advanced or Experimental Use Cases
==================================

.. _cython:

C plus Python plus SWIG plus Cython
-----------------------------------

In spite of the apparent incongruence of using Python bindings to a C
interface only to generate more C from the Python; it is in fact quite
possible to use the GPGME bindings with
`Cython <http://docs.cython.org/en/latest/index.html>`__. Though in many
cases the benefits may not be obvious since the most computationally
intensive work never leaves the level of the C code with which GPGME
itself is interacting with.

Nevertheless, there are some situations where the benefits are
demonstrable. One of the better and easier examples being the one of the
early examples in this HOWTO, the `key
counting <#howto-keys-counting>`__ code. Running that example as an
executable Python script, ``keycount.py`` (available in the
``examples/howto/`` directory), will take a noticeable amount of time to
run on most systems where the public keybox or keyring contains a few
thousand public keys.

Earlier in the evening, prior to starting this section, I ran that
script on my laptop; as I tend to do periodically and timed it using
``time`` utility, with the following results:

.. code:: shell

   bash-4.4$ time keycount.py

   Number of secret keys:  23
   Number of public keys:  12112


   real  11m52.945s
   user  0m0.913s
   sys   0m0.752s

   bash-4.4$

Sometime after that I imported another key and followed it with a little
test of Cython. This test was kept fairly basic, essentially lifting the
material from the `Cython Basic
Tutorial <http://docs.cython.org/en/latest/src/tutorial/cython_tutorial.html>`__
to demonstrate compiling Python code to C. The first step was to take
the example key counting code quoted previously, essentially from the
importing of the ``gpg`` module to the end of the script:

.. code:: python

   import gpg

   c = gpg.Context()
   seckeys = c.keylist(pattern=None, secret=True)
   pubkeys = c.keylist(pattern=None, secret=False)

   seclist = list(seckeys)
   secnum = len(seclist)

   publist = list(pubkeys)
   pubnum = len(publist)

   print("""
       Number of secret keys:  {0}
       Number of public keys:  {1}

   """.format(secnum, pubnum))

Save that into a file called ``keycount.pyx`` and then create a
``setup.py`` file which contains this:

.. code:: python

   from distutils.core import setup
   from Cython.Build import cythonize

   setup(
       ext_modules = cythonize("keycount.pyx")
   )

Compile it:

.. code:: shell

   bash-4.4$ python setup.py build_ext --inplace
   bash-4.4$

Then run it in a similar manner to ``keycount.py``:

.. code:: shell

   bash-4.4$ time python3.7 -c "import keycount"

   Number of secret keys:  23
   Number of public keys:  12113


   real  6m47.905s
   user  0m0.785s
   sys   0m0.331s

   bash-4.4$

Cython turned ``keycount.pyx`` into an 81KB ``keycount.o`` file in the
``build/`` directory, a 24KB ``keycount.cpython-37m-darwin.so`` file to
be imported into Python 3.7 and a 113KB ``keycount.c`` generated C
source code file of nearly three thousand lines. Quite a bit bigger than
the 314 bytes of the ``keycount.pyx`` file or the full 1,452 bytes of
the full executable ``keycount.py`` example script.

On the other hand it ran in nearly half the time; taking 6 minutes and
47.905 seconds to run. As opposed to the 11 minutes and 52.945 seconds
which the CPython script alone took.

The ``keycount.pyx`` and ``setup.py`` files used to generate this
example have been added to the ``examples/howto/advanced/cython/``
directory The example versions include some additional options to
annotate the existing code and to detect Cython\'s use. The latter comes
from the `Magic
Attributes <http://docs.cython.org/en/latest/src/tutorial/pure.html#magic-attributes-within-the-pxd>`__
section of the Cython documentation.

.. _cheats-and-hacks:

Miscellaneous extras and work-arounds
=====================================

Most of the things in the following sections are here simply because
there was no better place to put them, even though some are only
peripherally related to the GPGME Python bindings. Some are also
workarounds for functions not integrated with GPGME as yet. This is
especially true of the first of these, dealing with `group
lines <#group-lines>`__.

Group lines
-----------

There is not yet an easy way to access groups configured in the gpg.conf
file from within GPGME. As a consequence these central groupings of keys
cannot be shared amongst multiple programs, such as MUAs readily.

The following code, however, provides a work-around for obtaining this
information in Python.

.. code:: python

   import subprocess
   import sys

   if sys.platform == "win32":
       gpgconfcmd = "gpgconf.exe --list-options gpg"
   else:
       gpgconfcmd = "gpgconf --list-options gpg"

   try:
       lines = subprocess.getoutput(gpgconfcmd).splitlines()
   except:
       process = subprocess.Popen(gpgconfcmd.split(), stdout=subprocess.PIPE)
       procom = process.communicate()
       if sys.version_info[0] == 2:
           lines = procom[0].splitlines()
       else:
           lines = procom[0].decode().splitlines()

   for i in range(len(lines)):
       if lines[i].startswith("group") is True:
           line = lines[i]
       else:
           pass

   groups = line.split(":")[-1].replace('"', '').split(',')

   group_lines = []
   group_lists = []

   for i in range(len(groups)):
       group_lines.append(groups[i].split("="))
       group_lists.append(groups[i].split("="))

   for i in range(len(group_lists)):
       group_lists[i][1] = group_lists[i][1].split()

The result of that code is that ``group_lines`` is a list of lists where
``group_lines[i][0]`` is the name of the group and ``group_lines[i][1]``
is the key IDs of the group as a string.

The ``group_lists`` result is very similar in that it is a list of
lists. The first part, ``group_lists[i][0]`` matches
``group_lines[i][0]`` as the name of the group, but
``group_lists[i][1]`` is the key IDs of the group as a list.

A demonstration of using the ``groups.py`` module is also available in
the form of the executable ``mutt-groups.py`` script. This second script
reads all the group entries in a user\'s ``gpg.conf`` file and converts
them into crypt-hooks suitable for use with the Mutt and Neomutt mail
clients.

.. _hkp4py:

Keyserver access for Python
---------------------------

The `hkp4py <https://github.com/Selfnet/hkp4py>`__ module by Marcel Fest
was originally a port of the old
`python-hkp <https://github.com/dgladkov/python-hkp>`__ module from
Python 2 to Python 3 and updated to use the
`requests <http://docs.python-requests.org/en/latest/index.html>`__
module instead. It has since been modified to provide support for Python
2.7 as well and is available via PyPI.

Since it rewrites the ``hkp`` protocol prefix as ``http`` and ``hkps``
as ``https``, the module is able to be used even with servers which do
not support the full scope of keyserver functions. [5]_ It also works
quite readily when incorporated into a `Cython <#cython>`__ generated
and compiled version of any code.

.. _hkp4py-strings:

Key import format
~~~~~~~~~~~~~~~~~

The hkp4py module returns key data via requests as string literals
(``r.text``) instead of byte literals (``r.content``). This means that
the retrurned key data must be encoded to UTF-8 when importing that key
material using a ``gpg.Context().key_import()`` method.

For this reason an alternative method has been added to the ``search``
function of ``hkp4py.KeyServer()`` which returns the key in the correct
format as expected by ``key_import``. When importing using this module,
it is now possible to import with this:

.. code:: python

   for key in keys:
       if key.revoked is False:
           gpg.Context().key_import(key.key_blob)
       else:
           pass

Without that recent addition it would have been necessary to encode the
contents of each ``hkp4py.KeyServer().search()[i].key`` in
``hkp4py.KeyServer().search()`` before trying to import it.

An example of this is included in the `Importing
Keys <#howto-import-key>`__ section of this HOWTO and the corresponding
executable version of that example is available in the
``lang/python/examples/howto`` directory as normal; the executable
version is the ``import-keys-hkp.py`` file.

.. _gpgme-version-check:

GPGME version checking
----------------------

For various reasons it may be necessary to check which version of GPGME
the bindings have been built against; including whether a minimum
required version of GPGME is in use.

For the most part the ``gpg.version.versionstr`` and
``gpg.version.versionlist`` methods have been quite sufficient. The
former returns the same string as ``gpgme-config --version``, while the
latter returns the major, minor and patch values in a list.

To check if the installed bindings have actually been built against the
current installed libgpgme version, this check can be performed:

.. code:: python

   import gpg
   import subprocess
   import sys

   gpgme_version_call = subprocess.Popen(["gpgme-config", "--version"],
                                         stdout=subprocess.PIPE,
                                         stderr=subprocess.PIPE)
   gpgme_version_str = gpgme_version_call.communicate()

   if sys.version_info[0] == 2:
       gpgme_version = gpgme_version_str[0].strip()
   elif sys.version_info[0] >= 3:
       gpgme_version = gpgme_version_str[0].decode().strip()
   else:
       gpgme_version = None

   if gpgme_version is not None:
       if gpgme_version == gpg.version.versionstr:
           print("The GPGME Python bindings match libgpgme.")
       else:
           print("The GPGME Python bindings do NOT match libgpgme.")
   else:
       print("Upgrade Python and reinstall the GPGME Python bindings.")

For many developers, however, the preferred checking means checking for
a minimum version or point release. This is now readily available via
the ``gpg.version.versionintlist`` method (added in version
``1.12.1-beta79``). It is also now possible to easily check whether the
installed GPGME Python bindings were built from a development or beta
branch of the GPGME source code.

The following code demonstrates how both of those methods may be used:

.. code:: python

   import gpg

   try:
       if gpg.version.is_beta is True:
           print("The installed GPGME Python bindings were built from beta code.")
       else:
           print("The installed GPGME Python bindings are a released version.")
   except Exception as e:
       print(e)

   try:
       if gpg.version.versionintlist[0] == 1:
           if gpg.version.versionintlist[1] == 12:
               if gpg.version.versionintlist[2] == 1:
                   print("This is the minimum version for using versionintlist.")
               elif gpg.version.versionintlist[2] > 1:
                   print("The versionintlist method is available.")
               else:
                   pass
           elif gpg.version.versionintlist[1] > 12:
               print("The versionintlist method is available.")
           else:
               pass
       elif gpg.version.versionintlist[0] > 1:
           print("The versionintlist method is available.")
       else:
           pass
   except Exception as e:
       print(e)

The points where ``pass`` is used in the above example will most likely
also produce an ``Exception`` error since those results should only
occur in versions which do not have the ``gpgme.version.is_beta`` and
``gpgme.version.versionintlist`` methods available.

.. _copyright-and-license:

Copyright and Licensing
=======================

Copyright
---------

Copyright © The GnuPG Project, 2018.

Copyright (C) The GnuPG Project, 2018.

.. _draft-editions:

Draft Editions of this HOWTO
----------------------------

Draft editions of this HOWTO may be periodically available directly from
the author at any of the following URLs:

-  `GPGME Python Bindings HOWTO draft (XHTML AWS S3
   SSL) <https://files.au.adversary.org/crypto/gpgme-python-howto.html>`__
-  `GPGME Python Bindings HOWTO draft (XHTML AWS S3 no
   SSL) <http://files.au.adversary.org/crypto/gpgme-python-howto.html>`__
-  `GPGME Python Bindings HOWTO draft (Texinfo file AWS S3
   SSL) <https://files.au.adversary.org/crypto/gpgme-python-howto.texi>`__
-  `GPGME Python Bindings HOWTO draft (Texinfo file AWS S3 no
   SSL) <http://files.au.adversary.org/crypto/gpgme-python-howto.texi>`__
-  `GPGME Python Bindings HOWTO draft (Info file AWS S3
   SSL) <https://files.au.adversary.org/crypto/gpgme-python-howto.info>`__
-  `GPGME Python Bindings HOWTO draft (Info file AWS S3 no
   SSL) <http://files.au.adversary.org/crypto/gpgme-python-howto.info>`__
-  `GPGME Python Bindings HOWTO draft (reST file AWS S3
   SSL) <https://files.au.adversary.org/crypto/gpgme-python-howto.rst>`__
-  `GPGME Python Bindings HOWTO draft (reST file AWS S3 no
   SSL) <http://files.au.adversary.org/crypto/gpgme-python-howto.rst>`__

All of these draft versions except for one have been generated from this
document via Emacs `Org mode <https://orgmode.org/>`__ and `GNU
Texinfo <https://www.gnu.org/software/texinfo/>`__. Though it is likely
that the specific
`file <https://files.au.adversary.org/crypto/gpgme-python-howto>`__
`version <http://files.au.adversary.org/crypto/gpgme-python-howto.org>`__
used will be on the same server with the generated output formats.

The one exception is the reStructuredText version, which was converted
using the latest version of Pandoc from the Org mode source file using
either of the following two commands:

.. code:: shell

   pandoc -f org -t rst+smart -o gpgme-python-howto.rst gpgme-python-howto.org
   pandoc -f org -t rst+smart -o gpgme-python-howto.rst gpgme-python-howto

In addition to these there is a significantly less frequently updated
version as a HTML `WebHelp
site <https://files.au.adversary.org/crypto/gpgme-python-howto/webhelp/index.html>`__
(AWS S3 SSL); generated from DITA XML source files, which can be found
in `an alternative
branch <https://dev.gnupg.org/source/gpgme/browse/ben%252Fhowto-dita/>`__
of the GPGME git repository.

These draft editions are not official documents and the version of
documentation in the master branch or which ships with released versions
is the only official documentation. Nevertheless, these draft editions
may occasionally be of use by providing more accessible web versions
which are updated between releases. They are provided on the
understanding that they may contain errors or may contain content
subject to change prior to an official release.

.. _license:

License GPL compatible
----------------------

This file is free software; as a special exception the author gives
unlimited permission to copy and/or distribute it, with or without
modifications, as long as this notice is preserved.

This file is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY, to the extent permitted by law; without even the implied
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Footnotes
=========

.. [1]
   ``short-history`` and/or ``short-history.html``.

.. [2]
   With no issues reported specific to Python 3.7, the release of Python
   3.7.1 at around the same time as GPGME 1.12.0 and the testing with
   Python 3.7.1rc1, there is no reason to delay moving 3.7 ahead of 3.6
   now. Production environments with more conservative requirements will
   always enforce their own policies anyway and installation to each
   supported minor release is quite possible too.

.. [3]
   Yes, even if you use virtualenv with everything you do in Python. If
   you want to install this module as just your user account then you
   will need to manually configure, compile and install the *entire*
   GnuPG stack as that user as well. This includes libraries which are
   not often installed that way. It can be done and there are
   circumstances under which it is worthwhile, but generally only on
   POSIX systems which utilise single user mode (some even require it).

.. [4]
   You probably don\'t really want to do this. Searching the keyservers
   for \"gnupg.org\" produces over 400 results, the majority of which
   aren\'t actually at the gnupg.org domain, but just included a comment
   regarding the project in their key somewhere.

.. [5]
   Such as with ProtonMail servers. This also means that restricted
   servers which only advertise either HTTP or HTTPS end points and not
   HKP or HKPS end points must still be identified as as HKP or HKPS
   within the Python Code. The ``hkp4py`` module will rewrite these
   appropriately when the connection is made to the server.