From d9387650ffb159bbd04db74c4a1dba6f94e83116 Mon Sep 17 00:00:00 2001 From: NIIBE Yutaka Date: Fri, 12 Jul 2019 13:53:50 +0900 Subject: python: doc: Add suffix for org files. Signed-off-by: NIIBE Yutaka --- lang/python/doc/src/gpgme-python-howto | 3373 ---------------------------- lang/python/doc/src/gpgme-python-howto.org | 3373 ++++++++++++++++++++++++++++ lang/python/doc/src/index | 29 - lang/python/doc/src/index.org | 29 + lang/python/doc/src/maintenance-mode | 135 -- lang/python/doc/src/maintenance-mode.org | 135 ++ lang/python/doc/src/short-history | 187 -- lang/python/doc/src/short-history.org | 187 ++ lang/python/doc/src/what-is-new | 54 - lang/python/doc/src/what-is-new.org | 54 + lang/python/doc/src/what-was-new | 106 - lang/python/doc/src/what-was-new.org | 106 + 12 files changed, 3884 insertions(+), 3884 deletions(-) delete mode 100644 lang/python/doc/src/gpgme-python-howto create mode 100644 lang/python/doc/src/gpgme-python-howto.org delete mode 100644 lang/python/doc/src/index create mode 100644 lang/python/doc/src/index.org delete mode 100644 lang/python/doc/src/maintenance-mode create mode 100644 lang/python/doc/src/maintenance-mode.org delete mode 100644 lang/python/doc/src/short-history create mode 100644 lang/python/doc/src/short-history.org delete mode 100644 lang/python/doc/src/what-is-new create mode 100644 lang/python/doc/src/what-is-new.org delete mode 100644 lang/python/doc/src/what-was-new create mode 100644 lang/python/doc/src/what-was-new.org (limited to 'lang/python/doc/src') diff --git a/lang/python/doc/src/gpgme-python-howto b/lang/python/doc/src/gpgme-python-howto deleted file mode 100644 index e2e7e719..00000000 --- a/lang/python/doc/src/gpgme-python-howto +++ /dev/null @@ -1,3373 +0,0 @@ -# -*- mode: org -*- -#+TITLE: GNU Privacy Guard (GnuPG) Made Easy Python Bindings HOWTO (English) -#+AUTHOR: Ben McGinnes -#+LATEX_COMPILER: xelatex -#+LATEX_CLASS: article -#+LATEX_CLASS_OPTIONS: [12pt] -#+LATEX_HEADER: \usepackage{xltxtra} -#+LATEX_HEADER: \usepackage[margin=1in]{geometry} -#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} -#+LATEX_HEADER: \author{Ben McGinnes } - - -* Introduction - :PROPERTIES: - :CUSTOM_ID: intro - :END: - -| Version: | 0.1.5 | -| GPGME Version: | 1.13.0 | -| Author: | Ben McGinnes | -| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | -| Language: | Australian English, British English | -| Language codes: | en-AU, en-GB, en | - -This document provides basic instruction in how to use the GPGME -Python bindings to programmatically leverage the GPGME library. - - -** Python 2 versus Python 3 - :PROPERTIES: - :CUSTOM_ID: py2-vs-py3 - :END: - -Though the GPGME Python bindings themselves provide support for both -Python 2 and 3, the focus is unequivocally on Python 3 and -specifically from Python 3.4 and above. As a consequence all the -examples and instructions in this guide use Python 3 code. - -Much of it will work with Python 2, but much of it also deals with -Python 3 byte literals, particularly when reading and writing data. -Developers concentrating on Python 2.7, and possibly even 2.6, will -need to make the appropriate modifications to support the older string -and unicode types as opposed to bytes. - -There are multiple reasons for concentrating on Python 3; some of -which relate to the immediate integration of these bindings, some of -which relate to longer term plans for both GPGME and the python -bindings and some of which relate to the impending EOL period for -Python 2.7. Essentially, though, there is little value in tying the -bindings to a version of the language which is a dead end and the -advantages offered by Python 3 over Python 2 make handling the data -types with which GPGME deals considerably easier. - - -** Examples - :PROPERTIES: - :CUSTOM_ID: howto-python3-examples - :END: - -All of the examples found in this document can be found as Python 3 -scripts in the =lang/python/examples/howto= directory. - - -** Unofficial Drafts - :PROPERTIES: - :CUSTOM_ID: unofficial-drafts - :END: - -In addition to shipping with each release of GPGME, there is a section -on locations to read or download [[#draft-editions][draft editions]] of this document from -at the end of it. These are unofficial versions produced in between -major releases. - - -** What's New - :PROPERTIES: - :CUSTOM_ID: new-stuff - :END: - -Full details of what is new are now available in the [[file:what-is-new.org][What's New]] file -and archives of the preceding /What's New/ sections are available in -the [[file:what-was-new][What Was New]] file. - - -*** New in GPGME 1·13·0 - :PROPERTIES: - :CUSTOM_ID: new-stuff-1-13-0 - :END: - -See the [[file:what-is-new#new-stuff-1-13-0][What's New]] document for what is new in version 1.13.0. - - -*** New in GPGME 1·12·0 - :PROPERTIES: - :CUSTOM_ID: new-stuff-1-12-0 - :END: - -See the [[file:what-was-new#new-stuff-1-12-0][What Was New]] document for what was new in version 1.12.0. - - -* GPGME Concepts - :PROPERTIES: - :CUSTOM_ID: gpgme-concepts - :END: - - -** A C API - :PROPERTIES: - :CUSTOM_ID: gpgme-c-api - :END: - -Unlike many modern APIs with which programmers will be more familiar -with these days, the GPGME API is a C API. The API is intended for -use by C coders who would be able to access its features by including -the =gpgme.h= header file with their own C source code and then access -its functions just as they would any other C headers. - -This is a very effective method of gaining complete access to the API -and in the most efficient manner possible. It does, however, have the -drawback that it cannot be directly used by other languages without -some means of providing an interface to those languages. This is -where the need for bindings in various languages stems. - - -** Python bindings - :PROPERTIES: - :CUSTOM_ID: gpgme-python-bindings - :END: - -The Python bindings for GPGME provide a higher level means of -accessing the complete feature set of GPGME itself. It also provides -a more pythonic means of calling these API functions. - -The bindings are generated dynamically with SWIG and the copy of -=gpgme.h= generated when GPGME is compiled. - -This means that a version of the Python bindings is fundamentally tied -to the exact same version of GPGME used to generate that copy of -=gpgme.h=. - - -** Difference between the Python bindings and other GnuPG Python packages - :PROPERTIES: - :CUSTOM_ID: gpgme-python-bindings-diffs - :END: - -There have been numerous attempts to add GnuPG support to Python over -the years. Some of the most well known are listed here, along with -what differentiates them. - - -*** The python-gnupg package maintained by Vinay Sajip - :PROPERTIES: - :CUSTOM_ID: diffs-python-gnupg - :END: - -This is arguably the most popular means of integrating GPG with -Python. The package utilises the =subprocess= module to implement -wrappers for the =gpg= and =gpg2= executables normally invoked on the -command line (=gpg.exe= and =gpg2.exe= on Windows). - -The popularity of this package stemmed from its ease of use and -capability in providing the most commonly required features. - -Unfortunately it has been beset by a number of security issues in the -past; most of which stemmed from using unsafe methods of accessing the -command line via the =subprocess= calls. While some effort has been -made over the last two to three years (as of 2018) to mitigate this, -particularly by no longer providing shell access through those -subprocess calls, the wrapper is still somewhat limited in the scope -of its GnuPG features coverage. - -The python-gnupg package is available under the MIT license. - - -*** The gnupg package created and maintained by Isis Lovecruft - :PROPERTIES: - :CUSTOM_ID: diffs-isis-gnupg - :END: - -In 2015 Isis Lovecruft from the Tor Project forked and then -re-implemented the python-gnupg package as just gnupg. This new -package also relied on subprocess to call the =gpg= or =gpg2= -binaries, but did so somewhat more securely. - -The naming and version numbering selected for this package, however, -resulted in conflicts with the original python-gnupg and since its -functions were called in a different manner to python-gnupg, the -release of this package also resulted in a great deal of consternation -when people installed what they thought was an upgrade that -subsequently broke the code relying on it. - -The gnupg package is available under the GNU General Public License -version 3.0 (or any later version). - - -*** The PyME package maintained by Martin Albrecht - :PROPERTIES: - :CUSTOM_ID: diffs-pyme - :END: - -This package is the origin of these bindings, though they are somewhat -different now. For details of when and how the PyME package was -folded back into GPGME itself see the [[file:short-history.org][Short History]] document.[fn:1] - -The PyME package was first released in 2002 and was also the first -attempt to implement a low level binding to GPGME. In doing so it -provided access to considerably more functionality than either the -=python-gnupg= or =gnupg= packages. - -The PyME package is only available for Python 2.6 and 2.7. - -Porting the PyME package to Python 3.4 in 2015 is what resulted in it -being folded into the GPGME project and the current bindings are the -end result of that effort. - -The PyME package is available under the same dual licensing as GPGME -itself: the GNU General Public License version 2.0 (or any later -version) and the GNU Lesser General Public License version 2.1 (or any -later version). - - -* GPGME Python bindings installation - :PROPERTIES: - :CUSTOM_ID: gpgme-python-install - :END: - - -** No PyPI - :PROPERTIES: - :CUSTOM_ID: do-not-use-pypi - :END: - -Most third-party Python packages and modules are available and -distributed through the Python Package Installer, known as PyPI. - -Due to the nature of what these bindings are and how they work, it is -infeasible to install the GPGME Python bindings in the same way. - -This is because the bindings use SWIG to dynamically generate C -bindings against =gpgme.h= and =gpgme.h= is generated from -=gpgme.h.in= at compile time when GPGME is built from source. Thus to -include a package in PyPI which actually built correctly would require -either statically built libraries for every architecture bundled with -it or a full implementation of C for each architecture. - -See the additional notes regarding [[#snafu-cffi][CFFI and SWIG]] at the end of this -section for further details. - - -** Requirements - :PROPERTIES: - :CUSTOM_ID: gpgme-python-requirements - :END: - -The GPGME Python bindings only have three requirements: - -1. A suitable version of Python 2 or Python 3. With Python 2 that - means CPython 2.7 and with Python 3 that means CPython 3.4 or - higher. -2. [[https://www.swig.org][SWIG]]. -3. GPGME itself. Which also means that all of GPGME's dependencies - must be installed too. - - -*** Recommended Additions - :PROPERTIES: - :CUSTOM_ID: gpgme-python-recommendations - :END: - -Though none of the following are absolute requirements, they are all -recommended for use with the Python bindings. In some cases these -recommendations refer to which version(s) of CPython to use the -bindings with, while others refer to third party modules which provide -a significant advantage in some way. - -1. If possible, use Python 3 instead of 2. -2. Favour a more recent version of Python since even 3.4 is due to - reach EOL soon. In production systems and services, Python 3.6 - should be robust enough to be relied on. -3. If possible add the following Python modules which are not part of - the standard library: [[http://docs.python-requests.org/en/latest/index.html][Requests]], [[https://cython.org/][Cython]], [[https://pendulum.eustace.io/][Pendulum]] and [[https://github.com/Selfnet/hkp4py][hkp4py]]. - -Chances are quite high that at least the first one and maybe two of -those will already be installed. - -Note that, as with Cython, some of advanced use case scenarios will -bring with them additional requirements. Most of these will be fairly -well known and commonly installed ones, however, which are in many -cases likely to have already been installed on many systems or be -familiar to Python programmers. - - -** Installation - :PROPERTIES: - :CUSTOM_ID: installation - :END: - -Installing the Python bindings is effectively achieved by compiling -and installing GPGME itself. - -Once SWIG is installed with Python and all the dependencies for GPGME -are installed you only need to confirm that the version(s) of Python -you want the bindings installed for are in your =$PATH=. - -By default GPGME will attempt to install the bindings for the most -recent or highest version number of Python 2 and Python 3 it detects -in =$PATH=. It specifically checks for the =python= and =python3= -executables first and then checks for specific version numbers. - -For Python 2 it checks for these executables in this order: =python=, -=python2= and =python2.7=. - -For Python 3 it checks for these executables in this order: =python3=, - =python3.7=, =python3.6=, =python3.5= and =python3.4=.[fn:2] - -On systems where =python= is actually =python3= and not =python2= it -may be possible that =python2= may be overlooked, but there have been -no reports of that actually occurring as yet. - -In the three months or so since the release of Python 3.7.0 there has -been extensive testing and work with these bindings with no issues -specifically relating to the new version of Python or any of the new -features of either the language or the bindings. This has also been -the case with Python 3.7.1rc1. With that in mind and given the -release of Python 3.7.1 is scheduled for around the same time as GPGME -1.12.0, the order of preferred Python versions has been changed to -move Python 3.7 ahead of Python 3.6. - - -*** Installing GPGME - :PROPERTIES: - :CUSTOM_ID: install-gpgme - :END: - -See the GPGME =README= file for details of how to install GPGME from -source. - - -** Known Issues - :PROPERTIES: - :CUSTOM_ID: snafu - :END: - -There are a few known issues with the current build process and the -Python bindings. For the most part these are easily addressed should -they be encountered. - - -*** Breaking Builds - :PROPERTIES: - :CUSTOM_ID: snafu-a-swig-of-this-builds-character - :END: - -Occasionally when installing GPGME with the Python bindings included -it may be observed that the =make= portion of that process induces a -large very number of warnings and, eventually errors which end that -part of the build process. Yet following that with =make check= and -=make install= appears to work seamlessly. - -The cause of this is related to the way SWIG needs to be called to -dynamically generate the C bindings for GPGME in the first place. So -the entire process will always produce =lang/python/python2-gpg/= and -=lang/python/python3-gpg/= directories. These should contain the -build output generated during compilation, including the complete -bindings and module installed into =site-packages=. - -Occasionally the errors in the early part or some other conflict -(e.g. not installing as */root/* or */su/*) may result in nothing -being installed to the relevant =site-packages= directory and the -build directory missing a lot of expected files. Even when this -occurs, the solution is actually quite simple and will always work. - -That solution is simply to run the following commands as either the -*root* user or prepended with =sudo -H=[fn:3] in the =lang/python/= -directory: - -#+BEGIN_SRC shell - /path/to/pythonX.Y setup.py build - /path/to/pythonX.Y setup.py build - /path/to/pythonX.Y setup.py install -#+END_SRC - -Yes, the build command does need to be run twice. Yes, you still need -to run the potentially failing or incomplete steps during the -=configure=, =make= and =make install= steps with installing GPGME. -This is because those steps generate a lot of essential files needed, -both by and in order to create, the bindings (including both the -=setup.py= and =gpgme.h= files). - - -**** IMPORTANT Note - :PROPERTIES: - :CUSTOM_ID: snafu-swig-build-note - :END: - -If specifying a selected number of languages to create bindings for, -try to leave Python last. Currently the majority of the other -language bindings are also preceding Python of either version when -listed alphabetically (not counting the Qt bindings). - -If Python is set to precede one of the other languages then it is -possible that the errors described here may interrupt the build -process before generating bindings for those other languages. In -these cases it may be preferable to configure all preferred language -bindings separately with alternative =configure= steps for GPGME using -the =--enable-languages=$LANGUAGE= option. - -Alternatively =make= (or =gmake=, depending on your platform) may be -run with the the =-k= option, which tells make to keep going even if -errors are encountered. In that case the failure of one language's -set of bindings to build should not hamper another language's bindings -to build. - - -*** Reinstalling Responsibly - :PROPERTIES: - :CUSTOM_ID: snafu-lessons-for-the-lazy - :END: - -Regardless of whether you're installing for one version of Python or -several, there will come a point where reinstallation is required. -With most Python module installations, the installed files go into the -relevant site-packages directory and are then forgotten about. Then -the module is upgraded, the new files are copied over the old and -that's the end of the matter. - -While the same is true of these bindings, there have been intermittent -issues observed on some platforms which have benefited significantly -from removing all the previous installations of the bindings before -installing the updated versions. - -Removing the previous version(s) is simply a matter of changing to the -relevant =site-packages= directory for the version of Python in -question and removing the =gpg/= directory and any accompanying -egg-info files for that module. - -In most cases this will require root or administration privileges on -the system, but the same is true of installing the module in the first -place. - - -*** Multiple installations - :PROPERTIES: - :CUSTOM_ID: snafu-the-full-monty - :END: - -For a variety of reasons it may be either necessary or just preferable -to install the bindings to alternative installed Python versions which -meet the requirements of these bindings. - -On POSIX systems this will generally be most simply achieved by -running the manual installation commands (build, build, install) as -described in the previous section for each Python installation the -bindings need to be installed to. - -As per the SWIG documentation: the compilers, libraries and runtime -used to build GPGME and the Python Bindings *must* match those used to -compile Python itself, including the version number(s) (at least going -by major version numbers and probably minor numbers too). - -On most POSIX systems, including OS X, this will very likely be the -case in most, if not all, cases. - -Note that from GPGME [[https://dev.gnupg.org/rMff6ff616aea6f59b7f2ce1176492850ecdf3851e][1.12.1]] the default installation installs to each -version of Python it can find first. That is that it will currently -install for the first copies of Python versions 2.7, 3.4, 3.5, 3.6, -3.7 and 3.8 (dev branch) that it finds. Usually this will be in the -same prefix as GPGME itself, but is dictated by the =$PATH= when the -installation is performed. The above instructions can still be -performed on other python installations which the installer does not -find, including alternative prefixes. - - - -*** Won't Work With Windows - :PROPERTIES: - :CUSTOM_ID: snafu-runtime-not-funtime - :END: - -There are semi-regular reports of Windows users having considerable -difficulty in installing and using the Python bindings at all. Very -often, possibly even always, these reports come from Cygwin users -and/or MinGW users and/or Msys2 users. Though not all of them have -been confirmed, it appears that these reports have also come from -people who installed Python using the Windows installer files from the -[[https://python.org][Python website]] (i.e. mostly MSI installers, sometimes self-extracting -=.exe= files). - -The Windows versions of Python are not built using Cygwin, MinGW or -Msys2; they're built using Microsoft Visual Studio. Furthermore the -version used is /considerably/ more advanced than the version which -MinGW obtained a small number of files from many years ago in order to -be able to compile anything at all. Not only that, but there are -changes to the version of Visual Studio between some micro releases, -though that is is particularly the case with Python 2.7, since it has -been kept around far longer than it should have been. - -There are two theoretical solutions to this issue: - - 1. Compile and install the GnuPG stack, including GPGME and the - Python bindings using the same version of Microsoft Visual Studio - used by the Python Foundation to compile the version of Python - installed. - - If there are multiple versions of Python then this will need to be - done with each different version of Visual Studio used for those - versions of Python. - - 2. Compile and install Python using the same tools used by choice, - such as MinGW or Msys2. - -Do *not* use the official Windows installer for Python unless -following the first method. - -In this type of situation it may even be for the best to accept that -there are less limitations on permissive software than free software -and simply opt to use a recent version of the Community Edition of -Microsoft Visual Studio to compile and build all of it, no matter -what. - -Investigations into the extent or the limitations of this issue are -ongoing. - -The following table lists the version of Microsoft Visual Studio which -needs to be used when compiling GPGME and the Python bindings with -each version of the CPython binary released [[https://www.python.org/downloads/windows/][for Windows]]: - -| CPython | Microsoft product name | runtime filename | -| 2.7.6 | Visual Studio 2008 | MSVCR90.DLL | -| 3.4.0 | Visual Studio 2010 | MSVCR100.DLL | -| 3.5.0 | Visual Studio 2015 | *see below* | -| 3.6.0 | Visual Studio 2015 | *see below* | -| 3.7.0 | Visual Studio 2017* | *see below* | - -It is important to note that MingW and Msys2 ship with the Visual C -runtime from Microsoft Visual Studio 2005 and are thus *incompatible* -with all the versions of CPython which can be used with the GPGME -Python bindings. - -It is also important to note that from CPython 3.5 onwards, the Python -Foundation has adopted the reworking of the Visual C runtime which was -performed for Visual Studio 2015 and aimed at resolving many of these -kinds of issues. Much greater detail on these issues and the correct -file(s) to link to are available from Matthew Brett's invaluable page, -[[https://matthew-brett.github.io/pydagogue/python_msvc.html][Using Microsoft Visual C with Python]]. It is also worth reading the -Microsoft Developer Network blog post on [[http://blogs.msdn.com/b/vcblog/archive/2015/03/03/introducing-the-universal-crt.aspx][the universal CRT]] and Steve -Dower's blog posts on Python extensions ([[http://stevedower.id.au/blog/building-for-python-3-5][part 1]] and [[http://stevedower.id.au/blog/building-for-python-3-5-part-two][part 2]]). - -The second of those two posts by Steve Dower contains the details of -specific configuration options required for compiling anything to be -used with official CPython releases. In addition to those -configuration and compiler settings to use, the versions of Visual -Studio prior to Visual Studio 2015 did not support 64-bit systems by -default. So compiling a 64-bit version of these bindings for a 64-bit -version of CPython 2.7 or 3.4 requires additional work. - -In addition to the blog posts, the [[https://wiki.python.org/moin/WindowsCompilers][Windows compilers]] wiki page on the -CPython wiki is another essential reference on the relevant versions -of Visual Studio to use and the degree of compatibility with CPython -releases. - -Eventually someone will ask why there isn't an installable binary for -Windows, which the GPGME of the licenses do not preclude as long as -the source code is available in conjunction with such a release. - -The sheer number of versions of Visual Studio in conjunction with -differing configuration options depending on the target Windows -version and whether the architecture is 64-bit or 32-bit makes it -difficult to provide a correct binary installer for Windows users. At -the bare minimum doing so would require the GnuPG project compile ten -different versions of the bindings with each release; both 32-bit and -64-bit versions for CPython 2.7 and 3.4, with 64-bit versions for both -x86-64 (i.e. Intel and AMD) and ARM architectures for CPython 3.5, -3.6, 3.7 and later releases. That's the bare *minimum*, it'd probably -be higher. - -Additionally, with only a binary installation used in conjunction with -the CPython installer from =python.org= the advanced options available -which utilise [[#cython][Cython]] will not be able to be used at all. Cython -depends on being able to compile the C code it generates and that too -would need to utilise a matching runtime to both the installed version -of CPython and these bindings in order to work with the bindings. - -Considering all of that, what do we recommend? - - 1. Use a recent version of CPython; at least 3.5, but ideally 3.6 or - later. - - 2. Use Visual Studio 2015 or the standalone build tools for Visual - Studio 2017 (or later). - - 3. Compile both CPython and GPGME with these bindings using the tools - selected in step 2. - - 4. Ignore MingW, Msys2 and the official CPython binary installers. - - 5. Be thankful the answer to this question wasn't simply to say - something like, “install Linux” or “install FreeBSD” (or even - Apple's OS X). - - -*** CFFI is the Best™ and GPGME should use it instead of SWIG - :PROPERTIES: - :CUSTOM_ID: snafu-cffi - :END: - -There are many reasons for favouring [[https://cffi.readthedocs.io/en/latest/overview.html][CFFI]] and proponents of it are -quite happy to repeat these things as if all it would take to switch -from SWIG to CFFI is repeating that list as if it were a new concept. - -The fact is that there are things which Python's CFFI implementation -cannot handle in the GPGME C code. Beyond that there are features of -SWIG which are simply not available with CFFI at all. SWIG generates -the bindings to Python using the =gpgme.h= file, but that file is not -a single version shipped with each release, it too is generated when -GPGME is compiled. - -CFFI is currently unable to adapt to such a potentially mutable -codebase. If there were some means of applying SWIG's dynamic code -generation to produce the Python/CFFI API modes of accessing the GPGME -libraries (or the source source code directly), but such a thing does -not exist yet either and it currently appears that work is needed in -at least one of CFFI's dependencies before any of this can be -addressed. - -So if you're a massive fan of CFFI; that's great, but if you want this -project to switch to CFFI then rather than just insisting that it -should, I'd suggest you volunteer to bring CFFI up to the level this -project needs. - -If you're actually seriously considering doing so, then I'd suggest -taking the =gpgme-tool.c= file in the GPGME =src/= directory and -getting that to work with any of the CFFI API methods (not the ABI -methods, they'll work with pretty much anything). When you start -running into trouble with "ifdefs" then you'll know what sort of -things are lacking. That doesn't even take into account the amount of -work saved via SWIG's code generation techniques either. - - -*** Virtualised Environments - :PROPERTIES: - :CUSTOM_ID: snafu-venv - :END: - -It is fairly common practice amongst Python developers to, as much as -possible, use packages like virtualenv to keep various things that are -to be installed from interfering with each other. Given how much of -the GPGME bindings is often at odds with the usual pythonic way of -doing things, it stands to reason that this would be called into -question too. - -As it happens the answer as to whether or not the bindings can be used -with virtualenv, the answer is both yes and no. - -In general we recommend installing to the relevant path and matching -prefix of GPGME itself. Which means that when GPGME, and ideally the -rest of the GnuPG stack, is installed to a prefix like =/usr/local= or -=/opt/local= then the bindings would need to be installed to the main -Python installation and not a virtualised abstraction. Attempts to -separate the two in the past have been known to cause weird and -intermittent errors ranging from minor annoyances to complete failures -in the build process. - -As a consequence we only recommend building with and installing to the -main Python installations within the same prefix as GPGME is installed -to or which are found by GPGME's configuration stage immediately prior -to running the make commands. Which is exactly what the compiling and -installing process of GPGME does by default. - -Once that is done, however, it appears that a copy of the compiled -module may be installed into a virtualenv of the same major and minor -version matching the build. Alternatively it is possible to utilise a -=sites.pth= file in the =site-packages/= directory of a virtualenv -installation, which links back to the system installations -corresponding directory in order to import anything installed system -wide. This may or may not be appropriate on a case by case basis. - -Though extensive testing of either of these options is not yet -complete, preliminary testing of them indicates that both are viable -as long as the main installation is complete. Which means that -certain other options normally restricted to virtual environments are -also available, including integration with pythonic test suites -(e.g. [[https://docs.pytest.org/en/latest/index.html][pytest]]) and other large projects. - -That said, it is worth reiterating the warning regarding non-standard -installations. If one were to attempt to install the bindings only to -a virtual environment without somehow also including the full GnuPG -stack (or enough of it as to include GPGME) then it is highly likely -that errors would be encountered at some point and more than a little -likely that the build process itself would break. - -If a degree of separation from the main operating system is still -required in spite of these warnings, then consider other forms of -virtualisation. Either a virtual machine (e.g. [[https://www.virtualbox.org/][VirtualBox]]), a -hardware emulation layer (e.g. [[https://www.qemu.org/][QEMU]]) or an application container -(e.g. [[https://www.docker.com/why-docker][Docker]]). - -Finally it should be noted that the limited tests conducted thus far -have been using the =virtualenv= command in a new directory to create -the virtual python environment. As opposed to the standard =python3 --m venv= and it is possible that this will make a difference depending -on the system and version of Python in use. Another option is to run -the command =python3 -m virtualenv /path/to/install/virtual/thingy= -instead. - - -*** Post installation - :PROPERTIES: - :CUSTOM_ID: snafu-docs - :END: - -Following installation it is recommended to move the -=post_installer.py= script from the =lang/python/examples/howto/= -directory to the =lang/python/= directory and run it. This will fix -or restore files needed by Sphinx which may be removed during a -distribution build for release. It will also generate reST files from -Org mode files with Pandoc and generate Texinfo files from Org mode -files with GNU Emacs and Org mode (in batch mode). Additionally it -will fix the UTF-8 declaration line in the Texinfo files (Emacs -expects "UTF-8" to be "utf-8"). - - -* Fundamentals - :PROPERTIES: - :CUSTOM_ID: howto-fund-a-mental - :END: - -Before we can get to the fun stuff, there are a few matters regarding -GPGME's design which hold true whether you're dealing with the C code -directly or these Python bindings. - - -** No REST - :PROPERTIES: - :CUSTOM_ID: no-rest-for-the-wicked - :END: - -The first part of which is or will be fairly blatantly obvious upon -viewing the first example, but it's worth reiterating anyway. That -being that this API is /*not*/ a REST API. Nor indeed could it ever -be one. - -Most, if not all, Python programmers (and not just Python programmers) -know how easy it is to work with a RESTful API. In fact they've -become so popular that many other APIs attempt to emulate REST-like -behaviour as much as they are able. Right down to the use of JSON -formatted output to facilitate the use of their API without having to -retrain developers. - -This API does not do that. It would not be able to do that and also -provide access to the entire C API on which it's built. It does, -however, provide a very pythonic interface on top of the direct -bindings and it's this pythonic layer that this HOWTO deals with. - - -** Context - :PROPERTIES: - :CUSTOM_ID: howto-get-context - :END: - -One of the reasons which prevents this API from being RESTful is that -most operations require more than one instruction to the API to -perform the task. Sure, there are certain functions which can be -performed simultaneously, particularly if the result known or strongly -anticipated (e.g. selecting and encrypting to a key known to be in the -public keybox). - -There are many more, however, which cannot be manipulated so readily: -they must be performed in a specific sequence and the result of one -operation has a direct bearing on the outcome of subsequent -operations. Not merely by generating an error either. - -When dealing with this type of persistent state on the web, full of -both the RESTful and REST-like, it's most commonly referred to as a -session. In GPGME, however, it is called a context and every -operation type has one. - - -* Working with keys - :PROPERTIES: - :CUSTOM_ID: howto-keys - :END: - - -** Key selection - :PROPERTIES: - :CUSTOM_ID: howto-keys-selection - :END: - -Selecting keys to encrypt to or to sign with will be a common -occurrence when working with GPGMe and the means available for doing -so are quite simple. - -They do depend on utilising a Context; however once the data is -recorded in another variable, that Context does not need to be the -same one which subsequent operations are performed. - -The easiest way to select a specific key is by searching for that -key's key ID or fingerprint, preferably the full fingerprint without -any spaces in it. A long key ID will probably be okay, but is not -advised and short key IDs are already a problem with some being -generated to match specific patterns. It does not matter whether the -pattern is upper or lower case. - -So this is the best method: - -#+BEGIN_SRC python -i -import gpg - -k = gpg.Context().keylist(pattern="258E88DCBD3CD44D8E7AB43F6ECB6AF0DEADBEEF") -keys = list(k) -#+END_SRC - -This is passable and very likely to be common: - -#+BEGIN_SRC python -i -import gpg - -k = gpg.Context().keylist(pattern="0x6ECB6AF0DEADBEEF") -keys = list(k) -#+END_SRC - -And this is a really bad idea: - -#+BEGIN_SRC python -i -import gpg - -k = gpg.Context().keylist(pattern="0xDEADBEEF") -keys = list(k) -#+END_SRC - -Alternatively it may be that the intention is to create a list of keys -which all match a particular search string. For instance all the -addresses at a particular domain, like this: - -#+BEGIN_SRC python -i -import gpg - -ncsc = gpg.Context().keylist(pattern="ncsc.mil") -nsa = list(ncsc) -#+END_SRC - - -*** Counting keys - :PROPERTIES: - :CUSTOM_ID: howto-keys-counting - :END: - -Counting the number of keys in your public keybox (=pubring.kbx=), the -format which has superseded the old keyring format (=pubring.gpg= and -=secring.gpg=), or the number of secret keys is a very simple task. - -#+BEGIN_SRC python -i -import gpg - -c = gpg.Context() -seckeys = c.keylist(pattern=None, secret=True) -pubkeys = c.keylist(pattern=None, secret=False) - -seclist = list(seckeys) -secnum = len(seclist) - -publist = list(pubkeys) -pubnum = len(publist) - -print(""" - Number of secret keys: {0} - Number of public keys: {1} -""".format(secnum, pubnum)) -#+END_SRC - -NOTE: The [[#cython][Cython]] introduction in the [[#advanced-use][Advanced and Experimental]] -section uses this same key counting code with Cython to demonstrate -some areas where Cython can improve performance even with the -bindings. Users with large public keyrings or keyboxes, for instance, -should consider these options if they are comfortable with using -Cython. - - -** Get key - :PROPERTIES: - :CUSTOM_ID: howto-get-key - :END: - -An alternative method of getting a single key via its fingerprint is -available directly within a Context with =Context().get_key=. This is -the preferred method of selecting a key in order to modify it, sign or -certify it and for obtaining relevant data about a single key as a -part of other functions; when verifying a signature made by that key, -for instance. - -By default this method will select public keys, but it can select -secret keys as well. - -This first example demonstrates selecting the current key of Werner -Koch, which is due to expire at the end of 2018: - -#+BEGIN_SRC python -i -import gpg - -fingerprint = "80615870F5BAD690333686D0F2AD85AC1E42B367" -key = gpg.Context().get_key(fingerprint) -#+END_SRC - -Whereas this example demonstrates selecting the author's current key -with the =secret= key word argument set to =True=: - -#+BEGIN_SRC python -i -import gpg - -fingerprint = "DB4724E6FA4286C92B4E55C4321E4E2373590E5D" -key = gpg.Context().get_key(fingerprint, secret=True) -#+END_SRC - -It is, of course, quite possible to select expired, disabled and -revoked keys with this function, but only to effectively display -information about those keys. - -It is also possible to use both unicode or string literals and byte -literals with the fingerprint when getting a key in this way. - - -** Importing keys - :PROPERTIES: - :CUSTOM_ID: howto-import-key - :END: - -Importing keys is possible with the =key_import()= method and takes -one argument which is a bytes literal object containing either the -binary or ASCII armoured key data for one or more keys. - -The following example retrieves one or more keys from the SKS -keyservers via the web using the requests module. Since requests -returns the content as a bytes literal object, we can then use that -directly to import the resulting data into our keybox. - -#+BEGIN_SRC python -i -import gpg -import os.path -import requests - -c = gpg.Context() -url = "https://sks-keyservers.net/pks/lookup" -pattern = input("Enter the pattern to search for key or user IDs: ") -payload = {"op": "get", "search": pattern} - -r = requests.get(url, verify=True, params=payload) -result = c.key_import(r.content) - -if result is not None and hasattr(result, "considered") is False: - print(result) -elif result is not None and hasattr(result, "considered") is True: - num_keys = len(result.imports) - new_revs = result.new_revocations - new_sigs = result.new_signatures - new_subs = result.new_sub_keys - new_uids = result.new_user_ids - new_scrt = result.secret_imported - nochange = result.unchanged - print(""" - The total number of keys considered for import was: {0} - - Number of keys revoked: {1} - Number of new signatures: {2} - Number of new subkeys: {3} - Number of new user IDs: {4} - Number of new secret keys: {5} - Number of unchanged keys: {6} - - The key IDs for all considered keys were: -""".format(num_keys, new_revs, new_sigs, new_subs, new_uids, new_scrt, - nochange)) - for i in range(num_keys): - print("{0}\n".format(result.imports[i].fpr)) -else: - pass -#+END_SRC - -NOTE: When searching for a key ID of any length or a fingerprint -(without spaces), the SKS servers require the the leading =0x= -indicative of hexadecimal be included. Also note that the old short -key IDs (e.g. =0xDEADBEEF=) should no longer be used due to the -relative ease by which such key IDs can be reproduced, as demonstrated -by the Evil32 Project in 2014 (which was subsequently exploited in -2016). - -Testing for whether a string in any given search is or may be a -hexadecimal value which may be missing the leading =0x= is a simple -matter of using a try/except statement which attempts to convert the -string as hex to an integer and then back to hex; then using that to -search with. Raising a ValueError simply results in treating the -string as a string. This is the method and logic utilised in the -=import-keys-hkp.py= script (see below). - - -*** Working with ProtonMail - :PROPERTIES: - :CUSTOM_ID: import-protonmail - :END: - -Here is a variation on the example above which checks the constrained -ProtonMail keyserver for ProtonMail public keys. - -#+BEGIN_SRC python -i -import gpg -import requests -import sys - -print(""" -This script searches the ProtonMail key server for the specified key and -imports it. -""") - -c = gpg.Context(armor=True) -url = "https://api.protonmail.ch/pks/lookup" -ksearch = [] - -if len(sys.argv) >= 2: - keyterm = sys.argv[1] -else: - keyterm = input("Enter the key ID, UID or search string: ") - -if keyterm.count("@") == 2 and keyterm.startswith("@") is True: - ksearch.append(keyterm[1:]) - ksearch.append(keyterm[1:]) - ksearch.append(keyterm[1:]) -elif keyterm.count("@") == 1 and keyterm.startswith("@") is True: - ksearch.append("{0}@protonmail.com".format(keyterm[1:])) - ksearch.append("{0}@protonmail.ch".format(keyterm[1:])) - ksearch.append("{0}@pm.me".format(keyterm[1:])) -elif keyterm.count("@") == 0: - ksearch.append("{0}@protonmail.com".format(keyterm)) - ksearch.append("{0}@protonmail.ch".format(keyterm)) - ksearch.append("{0}@pm.me".format(keyterm)) -elif keyterm.count("@") == 2 and keyterm.startswith("@") is False: - uidlist = keyterm.split("@") - for uid in uidlist: - ksearch.append("{0}@protonmail.com".format(uid)) - ksearch.append("{0}@protonmail.ch".format(uid)) - ksearch.append("{0}@pm.me".format(uid)) -elif keyterm.count("@") > 2: - uidlist = keyterm.split("@") - for uid in uidlist: - ksearch.append("{0}@protonmail.com".format(uid)) - ksearch.append("{0}@protonmail.ch".format(uid)) - ksearch.append("{0}@pm.me".format(uid)) -else: - ksearch.append(keyterm) - -for k in ksearch: - payload = {"op": "get", "search": k} - try: - r = requests.get(url, verify=True, params=payload) - if r.ok is True: - result = c.key_import(r.content) - elif r.ok is False: - result = r.content - except Exception as e: - result = None - - if result is not None and hasattr(result, "considered") is False: - print("{0} for {1}".format(result.decode(), k)) - elif result is not None and hasattr(result, "considered") is True: - num_keys = len(result.imports) - new_revs = result.new_revocations - new_sigs = result.new_signatures - new_subs = result.new_sub_keys - new_uids = result.new_user_ids - new_scrt = result.secret_imported - nochange = result.unchanged - print(""" -The total number of keys considered for import was: {0} - -With UIDs wholely or partially matching the following string: - - {1} - - Number of keys revoked: {2} - Number of new signatures: {3} - Number of new subkeys: {4} - Number of new user IDs: {5} -Number of new secret keys: {6} - Number of unchanged keys: {7} - -The key IDs for all considered keys were: -""".format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt, - nochange)) - for i in range(num_keys): - print(result.imports[i].fpr) - print("") - elif result is None: - print(e) -#+END_SRC - -Both the above example, [[../examples/howto/pmkey-import.py][pmkey-import.py]], and a version which prompts -for an alternative GnuPG home directory, [[../examples/howto/pmkey-import-alt.py][pmkey-import-alt.py]], are -available with the other examples and are executable scripts. - -Note that while the ProtonMail servers are based on the SKS servers, -their server is related more to their API and is not feature complete -by comparison to the servers in the SKS pool. One notable difference -being that the ProtonMail server does not permit non ProtonMail users -to update their own keys, which could be a vector for attacking -ProtonMail users who may not receive a key's revocation if it had been -compromised. - - -*** Importing with HKP for Python - :PROPERTIES: - :CUSTOM_ID: import-hkp4py - :END: - -Performing the same tasks with the [[https://github.com/Selfnet/hkp4py][hkp4py module]] (available via PyPI) -is not too much different, but does provide a number of options of -benefit to end users. Not least of which being the ability to perform -some checks on a key before importing it or not. For instance it may -be the policy of a site or project to only import keys which have not -been revoked. The hkp4py module permits such checks prior to the -importing of the keys found. - -#+BEGIN_SRC python -i -import gpg -import hkp4py -import sys - -c = gpg.Context() -server = hkp4py.KeyServer("hkps://hkps.pool.sks-keyservers.net") -results = [] -keys = [] - -if len(sys.argv) > 2: - pattern = " ".join(sys.argv[1:]) -elif len(sys.argv) == 2: - pattern = sys.argv[1] -else: - pattern = input("Enter the pattern to search for keys or user IDs: ") - - -if pattern is not None: - try: - key = server.search(hex(int(pattern, 16))) - keyed = True - except ValueError as ve: - key = server.search(pattern) - keyed = False - - if key is not None: - keys.append(key[0]) - if keyed is True: - try: - fob = server.search(pattern) - except: - fob = None - if fob is not None: - keys.append(fob[0]) - else: - pass - else: - pass - - for logrus in pattern.split(): - try: - key = server.search(hex(int(logrus, 16))) - hexed = True - except ValueError as ve: - key = server.search(logrus) - hexed = False - - if key is not None: - keys.append(key[0]) - if hexed is True: - try: - fob = server.search(logrus) - except: - fob = None - if fob is not None: - keys.append(fob[0]) - else: - pass - else: - pass - - -if len(keys) > 0: - for key in keys: - import_result = c.key_import(key.key_blob) - results.append(import_result) - -for result in results: - if result is not None and hasattr(result, "considered") is False: - print(result) - elif result is not None and hasattr(result, "considered") is True: - num_keys = len(result.imports) - new_revs = result.new_revocations - new_sigs = result.new_signatures - new_subs = result.new_sub_keys - new_uids = result.new_user_ids - new_scrt = result.secret_imported - nochange = result.unchanged - print(""" -The total number of keys considered for import was: {0} - - Number of keys revoked: {1} - Number of new signatures: {2} - Number of new subkeys: {3} - Number of new user IDs: {4} -Number of new secret keys: {5} - Number of unchanged keys: {6} - -The key IDs for all considered keys were: -""".format(num_keys, new_revs, new_sigs, new_subs, new_uids, new_scrt, - nochange)) - for i in range(num_keys): - print(result.imports[i].fpr) - print("") - else: - pass -#+END_SRC - -Since the hkp4py module handles multiple keys just as effectively as -one (=keys= is a list of responses per matching key), the example -above is able to do a little bit more with the returned data before -anything is actually imported. - - -*** Importing from ProtonMail with HKP for Python - :PROPERTIES: - :CUSTOM_ID: import-protonmail-hkp4py - :END: - -Though this can provide certain benefits even when working with -ProtonMail, the scope is somewhat constrained there due to the -limitations of the ProtonMail keyserver. - -For instance, searching the SKS keyserver pool for the term "gnupg" -produces hundreds of results from any time the word appears in any -part of a user ID. Performing the same search on the ProtonMail -keyserver returns zero results, even though there are at least two -test accounts which include it as part of the username. - -The cause of this discrepancy is the deliberate configuration of that -server by ProtonMail to require an exact match of the full email -address of the ProtonMail user whose key is being requested. -Presumably this is intended to reduce breaches of privacy of their -users as an email address must already be known before a key for that -address can be obtained. - - -**** Import from ProtonMail via HKP for Python Example no. 1 - :PROPERTIES: - :CUSTOM_ID: import-hkp4py-pm1 - :END: - -The following script is available with the rest of the examples under -the somewhat less than original name, =pmkey-import-hkp.py=. - -#+BEGIN_SRC python -i -import gpg -import hkp4py -import os.path -import sys - -print(""" -This script searches the ProtonMail key server for the specified key and -imports it. - -Usage: pmkey-import-hkp.py [search strings] -""") - -c = gpg.Context(armor=True) -server = hkp4py.KeyServer("hkps://api.protonmail.ch") -keyterms = [] -ksearch = [] -allkeys = [] -results = [] -paradox = [] -homeless = None - -if len(sys.argv) > 2: - keyterms = sys.argv[1:] -elif len(sys.argv) == 2: - keyterm = sys.argv[1] - keyterms.append(keyterm) -else: - key_term = input("Enter the key ID, UID or search string: ") - keyterms = key_term.split() - -for keyterm in keyterms: - if keyterm.count("@") == 2 and keyterm.startswith("@") is True: - ksearch.append(keyterm[1:]) - ksearch.append(keyterm[1:]) - ksearch.append(keyterm[1:]) - elif keyterm.count("@") == 1 and keyterm.startswith("@") is True: - ksearch.append("{0}@protonmail.com".format(keyterm[1:])) - ksearch.append("{0}@protonmail.ch".format(keyterm[1:])) - ksearch.append("{0}@pm.me".format(keyterm[1:])) - elif keyterm.count("@") == 0: - ksearch.append("{0}@protonmail.com".format(keyterm)) - ksearch.append("{0}@protonmail.ch".format(keyterm)) - ksearch.append("{0}@pm.me".format(keyterm)) - elif keyterm.count("@") == 2 and keyterm.startswith("@") is False: - uidlist = keyterm.split("@") - for uid in uidlist: - ksearch.append("{0}@protonmail.com".format(uid)) - ksearch.append("{0}@protonmail.ch".format(uid)) - ksearch.append("{0}@pm.me".format(uid)) - elif keyterm.count("@") > 2: - uidlist = keyterm.split("@") - for uid in uidlist: - ksearch.append("{0}@protonmail.com".format(uid)) - ksearch.append("{0}@protonmail.ch".format(uid)) - ksearch.append("{0}@pm.me".format(uid)) - else: - ksearch.append(keyterm) - -for k in ksearch: - print("Checking for key for: {0}".format(k)) - try: - keys = server.search(k) - if isinstance(keys, list) is True: - for key in keys: - allkeys.append(key) - try: - import_result = c.key_import(key.key_blob) - except Exception as e: - import_result = c.key_import(key.key) - else: - paradox.append(keys) - import_result = None - except Exception as e: - import_result = None - results.append(import_result) - -for result in results: - if result is not None and hasattr(result, "considered") is False: - print("{0} for {1}".format(result.decode(), k)) - elif result is not None and hasattr(result, "considered") is True: - num_keys = len(result.imports) - new_revs = result.new_revocations - new_sigs = result.new_signatures - new_subs = result.new_sub_keys - new_uids = result.new_user_ids - new_scrt = result.secret_imported - nochange = result.unchanged - print(""" -The total number of keys considered for import was: {0} - -With UIDs wholely or partially matching the following string: - - {1} - - Number of keys revoked: {2} - Number of new signatures: {3} - Number of new subkeys: {4} - Number of new user IDs: {5} -Number of new secret keys: {6} - Number of unchanged keys: {7} - -The key IDs for all considered keys were: -""".format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt, - nochange)) - for i in range(num_keys): - print(result.imports[i].fpr) - print("") - elif result is None: - pass -#+END_SRC - - -**** Import from ProtonMail via HKP for Python Example no. 2 - :PROPERTIES: - :CUSTOM_ID: import-hkp4py-pm2 - :END: - -Like its counterpart above, this script can also be found with the -rest of the examples, by the name pmkey-import-hkp-alt.py. - -With this script a modicum of effort has been made to treat anything -passed as a =homedir= which either does not exist or which is not a -directory, as also being a pssible user ID to check for. It's not -guaranteed to pick up on all such cases, but it should cover most of -them. - -#+BEGIN_SRC python -i -import gpg -import hkp4py -import os.path -import sys - -print(""" -This script searches the ProtonMail key server for the specified key and -imports it. Optionally enables specifying a different GnuPG home directory. - -Usage: pmkey-import-hkp.py [homedir] [search string] - or: pmkey-import-hkp.py [search string] -""") - -c = gpg.Context(armor=True) -server = hkp4py.KeyServer("hkps://api.protonmail.ch") -keyterms = [] -ksearch = [] -allkeys = [] -results = [] -paradox = [] -homeless = None - -if len(sys.argv) > 3: - homedir = sys.argv[1] - keyterms = sys.argv[2:] -elif len(sys.argv) == 3: - homedir = sys.argv[1] - keyterm = sys.argv[2] - keyterms.append(keyterm) -elif len(sys.argv) == 2: - homedir = "" - keyterm = sys.argv[1] - keyterms.append(keyterm) -else: - keyterm = input("Enter the key ID, UID or search string: ") - homedir = input("Enter the GPG configuration directory path (optional): ") - keyterms.append(keyterm) - -if len(homedir) == 0: - homedir = None - homeless = False - -if homedir is not None: - if homedir.startswith("~"): - if os.path.exists(os.path.expanduser(homedir)) is True: - if os.path.isdir(os.path.expanduser(homedir)) is True: - c.home_dir = os.path.realpath(os.path.expanduser(homedir)) - else: - homeless = True - else: - homeless = True - elif os.path.exists(os.path.realpath(homedir)) is True: - if os.path.isdir(os.path.realpath(homedir)) is True: - c.home_dir = os.path.realpath(homedir) - else: - homeless = True - else: - homeless = True - -# First check to see if the homedir really is a homedir and if not, treat it as -# a search string. -if homeless is True: - keyterms.append(homedir) - c.home_dir = None -else: - pass - -for keyterm in keyterms: - if keyterm.count("@") == 2 and keyterm.startswith("@") is True: - ksearch.append(keyterm[1:]) - ksearch.append(keyterm[1:]) - ksearch.append(keyterm[1:]) - elif keyterm.count("@") == 1 and keyterm.startswith("@") is True: - ksearch.append("{0}@protonmail.com".format(keyterm[1:])) - ksearch.append("{0}@protonmail.ch".format(keyterm[1:])) - ksearch.append("{0}@pm.me".format(keyterm[1:])) - elif keyterm.count("@") == 0: - ksearch.append("{0}@protonmail.com".format(keyterm)) - ksearch.append("{0}@protonmail.ch".format(keyterm)) - ksearch.append("{0}@pm.me".format(keyterm)) - elif keyterm.count("@") == 2 and keyterm.startswith("@") is False: - uidlist = keyterm.split("@") - for uid in uidlist: - ksearch.append("{0}@protonmail.com".format(uid)) - ksearch.append("{0}@protonmail.ch".format(uid)) - ksearch.append("{0}@pm.me".format(uid)) - elif keyterm.count("@") > 2: - uidlist = keyterm.split("@") - for uid in uidlist: - ksearch.append("{0}@protonmail.com".format(uid)) - ksearch.append("{0}@protonmail.ch".format(uid)) - ksearch.append("{0}@pm.me".format(uid)) - else: - ksearch.append(keyterm) - -for k in ksearch: - print("Checking for key for: {0}".format(k)) - try: - keys = server.search(k) - if isinstance(keys, list) is True: - for key in keys: - allkeys.append(key) - try: - import_result = c.key_import(key.key_blob) - except Exception as e: - import_result = c.key_import(key.key) - else: - paradox.append(keys) - import_result = None - except Exception as e: - import_result = None - results.append(import_result) - -for result in results: - if result is not None and hasattr(result, "considered") is False: - print("{0} for {1}".format(result.decode(), k)) - elif result is not None and hasattr(result, "considered") is True: - num_keys = len(result.imports) - new_revs = result.new_revocations - new_sigs = result.new_signatures - new_subs = result.new_sub_keys - new_uids = result.new_user_ids - new_scrt = result.secret_imported - nochange = result.unchanged - print(""" -The total number of keys considered for import was: {0} - -With UIDs wholely or partially matching the following string: - - {1} - - Number of keys revoked: {2} - Number of new signatures: {3} - Number of new subkeys: {4} - Number of new user IDs: {5} -Number of new secret keys: {6} - Number of unchanged keys: {7} - -The key IDs for all considered keys were: -""".format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt, - nochange)) - for i in range(num_keys): - print(result.imports[i].fpr) - print("") - elif result is None: - pass -#+END_SRC - - -** Exporting keys - :PROPERTIES: - :CUSTOM_ID: howto-export-key - :END: - -Exporting keys remains a reasonably simple task, but has been -separated into three different functions for the OpenPGP cryptographic -engine. Two of those functions are for exporting public keys and the -third is for exporting secret keys. - - -*** Exporting public keys - :PROPERTIES: - :CUSTOM_ID: howto-export-public-key - :END: - -There are two methods of exporting public keys, both of which are very -similar to the other. The default method, =key_export()=, will export -a public key or keys matching a specified pattern as normal. The -alternative, the =key_export_minimal()= method, will do the same thing -except producing a minimised output with extra signatures and third -party signatures or certifications removed. - -#+BEGIN_SRC python -i -import gpg -import os.path -import sys - -print(""" -This script exports one or more public keys. -""") - -c = gpg.Context(armor=True) - -if len(sys.argv) >= 4: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = sys.argv[3] -elif len(sys.argv) == 3: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = input("Enter the GPG configuration directory path (optional): ") -elif len(sys.argv) == 2: - keyfile = sys.argv[1] - logrus = input("Enter the UID matching the key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") -else: - keyfile = input("Enter the path and filename to save the secret key to: ") - logrus = input("Enter the UID matching the key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") - -if homedir.startswith("~"): - if os.path.exists(os.path.expanduser(homedir)) is True: - c.home_dir = os.path.expanduser(homedir) - else: - pass -elif os.path.exists(homedir) is True: - c.home_dir = homedir -else: - pass - -try: - result = c.key_export(pattern=logrus) -except: - result = c.key_export(pattern=None) - -if result is not None: - with open(keyfile, "wb") as f: - f.write(result) -else: - pass -#+END_SRC - -It should be noted that the result will only return =None= when a -search pattern has been entered, but has not matched any keys. When -the search pattern itself is set to =None= this triggers the exporting -of the entire public keybox. - -#+BEGIN_SRC python -i -import gpg -import os.path -import sys - -print(""" -This script exports one or more public keys in minimised form. -""") - -c = gpg.Context(armor=True) - -if len(sys.argv) >= 4: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = sys.argv[3] -elif len(sys.argv) == 3: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = input("Enter the GPG configuration directory path (optional): ") -elif len(sys.argv) == 2: - keyfile = sys.argv[1] - logrus = input("Enter the UID matching the key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") -else: - keyfile = input("Enter the path and filename to save the secret key to: ") - logrus = input("Enter the UID matching the key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") - -if homedir.startswith("~"): - if os.path.exists(os.path.expanduser(homedir)) is True: - c.home_dir = os.path.expanduser(homedir) - else: - pass -elif os.path.exists(homedir) is True: - c.home_dir = homedir -else: - pass - -try: - result = c.key_export_minimal(pattern=logrus) -except: - result = c.key_export_minimal(pattern=None) - -if result is not None: - with open(keyfile, "wb") as f: - f.write(result) -else: - pass -#+END_SRC - - -*** Exporting secret keys - :PROPERTIES: - :CUSTOM_ID: howto-export-secret-key - :END: - -Exporting secret keys is, functionally, very similar to exporting -public keys; save for the invocation of =pinentry= via =gpg-agent= in -order to securely enter the key's passphrase and authorise the export. - -The following example exports the secret key to a file which is then -set with the same permissions as the output files created by the -command line secret key export options. - -#+BEGIN_SRC python -i -import gpg -import os -import os.path -import sys - -print(""" -This script exports one or more secret keys. - -The gpg-agent and pinentry are invoked to authorise the export. -""") - -c = gpg.Context(armor=True) - -if len(sys.argv) >= 4: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = sys.argv[3] -elif len(sys.argv) == 3: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = input("Enter the GPG configuration directory path (optional): ") -elif len(sys.argv) == 2: - keyfile = sys.argv[1] - logrus = input("Enter the UID matching the secret key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") -else: - keyfile = input("Enter the path and filename to save the secret key to: ") - logrus = input("Enter the UID matching the secret key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") - -if len(homedir) == 0: - homedir = None -elif homedir.startswith("~"): - userdir = os.path.expanduser(homedir) - if os.path.exists(userdir) is True: - homedir = os.path.realpath(userdir) - else: - homedir = None -else: - homedir = os.path.realpath(homedir) - -if os.path.exists(homedir) is False: - homedir = None -else: - if os.path.isdir(homedir) is False: - homedir = None - else: - pass - -if homedir is not None: - c.home_dir = homedir -else: - pass - -try: - result = c.key_export_secret(pattern=logrus) -except: - result = c.key_export_secret(pattern=None) - -if result is not None: - with open(keyfile, "wb") as f: - f.write(result) - os.chmod(keyfile, 0o600) -else: - pass -#+END_SRC - -Alternatively the approach of the following script can be used. This -longer example saves the exported secret key(s) in files in the GnuPG -home directory, in addition to setting the file permissions as only -readable and writable by the user. It also exports the secret key(s) -twice in order to output both GPG binary (=.gpg=) and ASCII armoured -(=.asc=) files. - -#+BEGIN_SRC python -i -import gpg -import os -import os.path -import subprocess -import sys - -print(""" -This script exports one or more secret keys as both ASCII armored and binary -file formats, saved in files within the user's GPG home directory. - -The gpg-agent and pinentry are invoked to authorise the export. -""") - -if sys.platform == "win32": - gpgconfcmd = "gpgconf.exe --list-dirs homedir" -else: - gpgconfcmd = "gpgconf --list-dirs homedir" - -a = gpg.Context(armor=True) -b = gpg.Context() -c = gpg.Context() - -if len(sys.argv) >= 4: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = sys.argv[3] -elif len(sys.argv) == 3: - keyfile = sys.argv[1] - logrus = sys.argv[2] - homedir = input("Enter the GPG configuration directory path (optional): ") -elif len(sys.argv) == 2: - keyfile = sys.argv[1] - logrus = input("Enter the UID matching the secret key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") -else: - keyfile = input("Enter the filename to save the secret key to: ") - logrus = input("Enter the UID matching the secret key(s) to export: ") - homedir = input("Enter the GPG configuration directory path (optional): ") - -if len(homedir) == 0: - homedir = None -elif homedir.startswith("~"): - userdir = os.path.expanduser(homedir) - if os.path.exists(userdir) is True: - homedir = os.path.realpath(userdir) - else: - homedir = None -else: - homedir = os.path.realpath(homedir) - -if os.path.exists(homedir) is False: - homedir = None -else: - if os.path.isdir(homedir) is False: - homedir = None - else: - pass - -if homedir is not None: - c.home_dir = homedir -else: - pass - -if c.home_dir is not None: - if c.home_dir.endswith("/"): - gpgfile = "{0}{1}.gpg".format(c.home_dir, keyfile) - ascfile = "{0}{1}.asc".format(c.home_dir, keyfile) - else: - gpgfile = "{0}/{1}.gpg".format(c.home_dir, keyfile) - ascfile = "{0}/{1}.asc".format(c.home_dir, keyfile) -else: - if os.path.exists(os.environ["GNUPGHOME"]) is True: - hd = os.environ["GNUPGHOME"] - else: - try: - hd = subprocess.getoutput(gpgconfcmd) - except: - process = subprocess.Popen(gpgconfcmd.split(), - stdout=subprocess.PIPE) - procom = process.communicate() - if sys.version_info[0] == 2: - hd = procom[0].strip() - else: - hd = procom[0].decode().strip() - gpgfile = "{0}/{1}.gpg".format(hd, keyfile) - ascfile = "{0}/{1}.asc".format(hd, keyfile) - -try: - a_result = a.key_export_secret(pattern=logrus) - b_result = b.key_export_secret(pattern=logrus) -except: - a_result = a.key_export_secret(pattern=None) - b_result = b.key_export_secret(pattern=None) - -if a_result is not None: - with open(ascfile, "wb") as f: - f.write(a_result) - os.chmod(ascfile, 0o600) -else: - pass - -if b_result is not None: - with open(gpgfile, "wb") as f: - f.write(b_result) - os.chmod(gpgfile, 0o600) -else: - pass -#+END_SRC - - -*** Sending public keys to the SKS Keyservers - :PROPERTIES: - :CUSTOM_ID: howto-send-public-key - :END: - -As with the previous section on importing keys, the =hkp4py= module -adds another option with exporting keys in order to send them to the -public keyservers. - -The following example demonstrates how this may be done. - -#+BEGIN_SRC python -i -import gpg -import hkp4py -import os.path -import sys - -print(""" -This script sends one or more public keys to the SKS keyservers and is -essentially a slight variation on the export-key.py script. -""") - -c = gpg.Context(armor=True) -server = hkp4py.KeyServer("hkps://hkps.pool.sks-keyservers.net") - -if len(sys.argv) > 2: - logrus = " ".join(sys.argv[1:]) -elif len(sys.argv) == 2: - logrus = sys.argv[1] -else: - logrus = input("Enter the UID matching the key(s) to send: ") - -if len(logrus) > 0: - try: - export_result = c.key_export(pattern=logrus) - except Exception as e: - print(e) - export_result = None -else: - export_result = c.key_export(pattern=None) - -if export_result is not None: - try: - try: - send_result = server.add(export_result) - except: - send_result = server.add(export_result.decode()) - if send_result is not None: - print(send_result) - else: - pass - except Exception as e: - print(e) -else: - pass -#+END_SRC - -An expanded version of this script with additional functions for -specifying an alternative homedir location is in the examples -directory as =send-key-to-keyserver.py=. - -The =hkp4py= module appears to handle both string and byte literal text -data equally well, but the GPGME bindings deal primarily with byte -literal data only and so this script sends in that format first, then -tries the string literal form. - - -* Basic Functions - :PROPERTIES: - :CUSTOM_ID: howto-the-basics - :END: - -The most frequently called features of any cryptographic library will -be the most fundamental tasks for encryption software. In this -section we will look at how to programmatically encrypt data, decrypt -it, sign it and verify signatures. - - -** Encryption - :PROPERTIES: - :CUSTOM_ID: howto-basic-encryption - :END: - -Encrypting is very straight forward. In the first example below the -message, =text=, is encrypted to a single recipient's key. In the -second example the message will be encrypted to multiple recipients. - - -*** Encrypting to one key - :PROPERTIES: - :CUSTOM_ID: howto-basic-encryption-single - :END: - -Once the the Context is set the main issues with encrypting data is -essentially reduced to key selection and the keyword arguments -specified in the =gpg.Context().encrypt()= method. - -Those keyword arguments are: =recipients=, a list of keys encrypted to -(covered in greater detail in the following section); =sign=, whether -or not to sign the plaintext data, see subsequent sections on signing -and verifying signatures below (defaults to =True=); =sink=, to write -results or partial results to a secure sink instead of returning it -(defaults to =None=); =passphrase=, only used when utilising symmetric -encryption (defaults to =None=); =always_trust=, used to override the -trust model settings for recipient keys (defaults to =False=); -=add_encrypt_to=, utilises any preconfigured =encrypt-to= or -=default-key= settings in the user's =gpg.conf= file (defaults to -=False=); =prepare=, prepare for encryption (defaults to =False=); -=expect_sign=, prepare for signing (defaults to =False=); =compress=, -compresses the plaintext prior to encryption (defaults to =True=). - -#+BEGIN_SRC python -i -import gpg - -a_key = "0x12345678DEADBEEF" -text = b"""Some text to test with. - -Since the text in this case must be bytes, it is most likely that -the input form will be a separate file which is opened with "rb" -as this is the simplest method of obtaining the correct data format. -""" - -c = gpg.Context(armor=True) -rkey = list(c.keylist(pattern=a_key, secret=False)) -ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, sign=False) - -with open("secret_plans.txt.asc", "wb") as afile: - afile.write(ciphertext) -#+END_SRC - -Though this is even more likely to be used like this; with the -plaintext input read from a file, the recipient keys used for -encryption regardless of key trust status and the encrypted output -also encrypted to any preconfigured keys set in the =gpg.conf= file: - -#+BEGIN_SRC python -i -import gpg - -a_key = "0x12345678DEADBEEF" - -with open("secret_plans.txt", "rb") as afile: - text = afile.read() - -c = gpg.Context(armor=True) -rkey = list(c.keylist(pattern=a_key, secret=False)) -ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, sign=True, - always_trust=True, - add_encrypt_to=True) - -with open("secret_plans.txt.asc", "wb") as afile: - afile.write(ciphertext) -#+END_SRC - -If the =recipients= parameter is empty then the plaintext is encrypted -symmetrically. If no =passphrase= is supplied as a parameter or via a -callback registered with the =Context()= then an out-of-band prompt -for the passphrase via pinentry will be invoked. - - -*** Encrypting to multiple keys - :PROPERTIES: - :CUSTOM_ID: howto-basic-encryption-multiple - :END: - -Encrypting to multiple keys essentially just expands upon the key -selection process and the recipients from the previous examples. - -The following example encrypts a message (=text=) to everyone with an -email address on the =gnupg.org= domain,[fn:4] but does /not/ encrypt -to a default key or other key which is configured to normally encrypt -to. - -#+BEGIN_SRC python -i -import gpg - -text = b"""Oh look, another test message. - -The same rules apply as with the previous example and more likely -than not, the message will actually be drawn from reading the -contents of a file or, maybe, from entering data at an input() -prompt. - -Since the text in this case must be bytes, it is most likely that -the input form will be a separate file which is opened with "rb" -as this is the simplest method of obtaining the correct data -format. -""" - -c = gpg.Context(armor=True) -rpattern = list(c.keylist(pattern="@gnupg.org", secret=False)) -logrus = [] - -for i in range(len(rpattern)): - if rpattern[i].can_encrypt == 1: - logrus.append(rpattern[i]) - -ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, - sign=False, always_trust=True) - -with open("secret_plans.txt.asc", "wb") as afile: - afile.write(ciphertext) -#+END_SRC - -All it would take to change the above example to sign the message -and also encrypt the message to any configured default keys would -be to change the =c.encrypt= line to this: - -#+BEGIN_SRC python -i -ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, - always_trust=True, - add_encrypt_to=True) -#+END_SRC - -The only keyword arguments requiring modification are those for which -the default values are changing. The default value of =sign= is -=True=, the default of =always_trust= is =False=, the default of -=add_encrypt_to= is =False=. - -If =always_trust= is not set to =True= and any of the recipient keys -are not trusted (e.g. not signed or locally signed) then the -encryption will raise an error. It is possible to mitigate this -somewhat with something more like this: - -#+BEGIN_SRC python -i -import gpg - -with open("secret_plans.txt.asc", "rb") as afile: - text = afile.read() - -c = gpg.Context(armor=True) -rpattern = list(c.keylist(pattern="@gnupg.org", secret=False)) -logrus = [] - -for i in range(len(rpattern)): - if rpattern[i].can_encrypt == 1: - logrus.append(rpattern[i]) - - try: - ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, - add_encrypt_to=True) - except gpg.errors.InvalidRecipients as e: - for i in range(len(e.recipients)): - for n in range(len(logrus)): - if logrus[n].fpr == e.recipients[i].fpr: - logrus.remove(logrus[n]) - else: - pass - try: - ciphertext, result, sign_result = c.encrypt(text, - recipients=logrus, - add_encrypt_to=True) - with open("secret_plans.txt.asc", "wb") as afile: - afile.write(ciphertext) - except: - pass -#+END_SRC - -This will attempt to encrypt to all the keys searched for, then remove -invalid recipients if it fails and try again. - - -** Decryption - :PROPERTIES: - :CUSTOM_ID: howto-basic-decryption - :END: - -Decrypting something encrypted to a key in one's secret keyring is -fairly straight forward. - -In this example code, however, preconfiguring either =gpg.Context()= -or =gpg.core.Context()= as =c= is unnecessary because there is no need -to modify the Context prior to conducting the decryption and since the -Context is only used once, setting it to =c= simply adds lines for no -gain. - -#+BEGIN_SRC python -i -import gpg - -ciphertext = input("Enter path and filename of encrypted file: ") -newfile = input("Enter path and filename of file to save decrypted data to: ") - -with open(ciphertext, "rb") as cfile: - try: - plaintext, result, verify_result = gpg.Context().decrypt(cfile) - except gpg.errors.GPGMEError as e: - plaintext = None - print(e) - -if plaintext is not None: - with open(newfile, "wb") as nfile: - nfile.write(plaintext) - else: - pass -#+END_SRC - -The data available in =plaintext= in this example is the decrypted -content as a byte object, the recipient key IDs and algorithms in -=result= and the results of verifying any signatures of the data in -=verify_result=. - -If =gpg.Context().decrypt(cfile, verify=False)= is called instead, -then =verify_result= will be returned as =None= and the rest remains -as described here. - - -** Signing text and files - :PROPERTIES: - :CUSTOM_ID: howto-basic-signing - :END: - -The following sections demonstrate how to specify keys to sign with. - - -*** Signing key selection - :PROPERTIES: - :CUSTOM_ID: howto-basic-signing-signers - :END: - -By default GPGME and the Python bindings will use the default key -configured for the user invoking the GPGME API. If there is no -default key specified and there is more than one secret key available -it may be necessary to specify the key or keys with which to sign -messages and files. - -#+BEGIN_SRC python -i -import gpg - -logrus = input("Enter the email address or string to match signing keys to: ") -hancock = gpg.Context().keylist(pattern=logrus, secret=True) -sig_src = list(hancock) -#+END_SRC - -The signing examples in the following sections include the explicitly -designated =signers= parameter in two of the five examples; once where -the resulting signature would be ASCII armoured and once where it -would not be armoured. - -While it would be possible to enter a key ID or fingerprint here to -match a specific key, it is not possible to enter two fingerprints and -match two keys since the patten expects a string, bytes or None and -not a list. A string with two fingerprints won't match any single -key. - - -*** Normal or default signing messages or files - :PROPERTIES: - :CUSTOM_ID: howto-basic-signing-normal - :END: - -The normal or default signing process is essentially the same as is -most often invoked when also encrypting a message or file. So when -the encryption component is not utilised, the result is to produce an -encoded and signed output which may or may not be ASCII armoured and -which may or may not also be compressed. - -By default compression will be used unless GnuPG detects that the -plaintext is already compressed. ASCII armouring will be determined -according to the value of =gpg.Context().armor=. - -The compression algorithm is selected in much the same way as the -symmetric encryption algorithm or the hash digest algorithm is when -multiple keys are involved; from the preferences saved into the key -itself or by comparison with the preferences with all other keys -involved. - -#+BEGIN_SRC python -i -import gpg - -text0 = """Declaration of ... something. - -""" -text = text0.encode() - -c = gpg.Context(armor=True, signers=sig_src) -signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL) - -with open("/path/to/statement.txt.asc", "w") as afile: - afile.write(signed_data.decode()) -#+END_SRC - -Though everything in this example is accurate, it is more likely that -reading the input data from another file and writing the result to a -new file will be performed more like the way it is done in the next -example. Even if the output format is ASCII armoured. - -#+BEGIN_SRC python -i -import gpg - -with open("/path/to/statement.txt", "rb") as tfile: - text = tfile.read() - -c = gpg.Context() -signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL) - -with open("/path/to/statement.txt.sig", "wb") as afile: - afile.write(signed_data) -#+END_SRC - - -*** Detached signing messages and files - :PROPERTIES: - :CUSTOM_ID: howto-basic-signing-detached - :END: - -Detached signatures will often be needed in programmatic uses of -GPGME, either for signing files (e.g. tarballs of code releases) or as -a component of message signing (e.g. PGP/MIME encoded email). - -#+BEGIN_SRC python -i -import gpg - -text0 = """Declaration of ... something. - -""" -text = text0.encode() - -c = gpg.Context(armor=True) -signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH) - -with open("/path/to/statement.txt.asc", "w") as afile: - afile.write(signed_data.decode()) -#+END_SRC - -As with normal signatures, detached signatures are best handled as -byte literals, even when the output is ASCII armoured. - -#+BEGIN_SRC python -i -import gpg - -with open("/path/to/statement.txt", "rb") as tfile: - text = tfile.read() - -c = gpg.Context(signers=sig_src) -signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH) - -with open("/path/to/statement.txt.sig", "wb") as afile: - afile.write(signed_data) -#+END_SRC - - -*** Clearsigning messages or text - :PROPERTIES: - :CUSTOM_ID: howto-basic-signing-clear - :END: - -Though PGP/in-line messages are no longer encouraged in favour of -PGP/MIME, there is still sometimes value in utilising in-line -signatures. This is where clear-signed messages or text is of value. - -#+BEGIN_SRC python -i -import gpg - -text0 = """Declaration of ... something. - -""" -text = text0.encode() - -c = gpg.Context() -signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR) - -with open("/path/to/statement.txt.asc", "w") as afile: - afile.write(signed_data.decode()) -#+END_SRC - -In spite of the appearance of a clear-signed message, the data handled -by GPGME in signing it must still be byte literals. - -#+BEGIN_SRC python -i -import gpg - -with open("/path/to/statement.txt", "rb") as tfile: - text = tfile.read() - -c = gpg.Context() -signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR) - -with open("/path/to/statement.txt.asc", "wb") as afile: - afile.write(signed_data) -#+END_SRC - - -** Signature verification - :PROPERTIES: - :CUSTOM_ID: howto-basic-verification - :END: - -Essentially there are two principal methods of verification of a -signature. The first of these is for use with the normal or default -signing method and for clear-signed messages. The second is for use -with files and data with detached signatures. - -The following example is intended for use with the default signing -method where the file was not ASCII armoured: - -#+BEGIN_SRC python -i -import gpg -import time - -filename = "statement.txt" -gpg_file = "statement.txt.gpg" - -c = gpg.Context() - -try: - data, result = c.verify(open(gpg_file)) - verified = True -except gpg.errors.BadSignatures as e: - verified = False - print(e) - -if verified is True: - for i in range(len(result.signatures)): - sign = result.signatures[i] - print("""Good signature from: -{0} -with key {1} -made at {2} -""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, - time.ctime(sign.timestamp))) -else: - pass -#+END_SRC - -Whereas this next example, which is almost identical would work with -normal ASCII armoured files and with clear-signed files: - -#+BEGIN_SRC python -i -import gpg -import time - -filename = "statement.txt" -asc_file = "statement.txt.asc" - -c = gpg.Context() - -try: - data, result = c.verify(open(asc_file)) - verified = True -except gpg.errors.BadSignatures as e: - verified = False - print(e) - -if verified is True: - for i in range(len(result.signatures)): - sign = result.signatures[i] - print("""Good signature from: -{0} -with key {1} -made at {2} -""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, - time.ctime(sign.timestamp))) -else: - pass -#+END_SRC - -In both of the previous examples it is also possible to compare the -original data that was signed against the signed data in =data= to see -if it matches with something like this: - -#+BEGIN_SRC python -i -with open(filename, "rb") as afile: - text = afile.read() - -if text == data: - print("Good signature.") -else: - pass -#+END_SRC - -The following two examples, however, deal with detached signatures. -With his method of verification the data that was signed does not get -returned since it is already being explicitly referenced in the first -argument of =c.verify=. So =data= is =None= and only the information -in =result= is available. - -#+BEGIN_SRC python -i -import gpg -import time - -filename = "statement.txt" -sig_file = "statement.txt.sig" - -c = gpg.Context() - -try: - data, result = c.verify(open(filename), open(sig_file)) - verified = True -except gpg.errors.BadSignatures as e: - verified = False - print(e) - -if verified is True: - for i in range(len(result.signatures)): - sign = result.signatures[i] - print("""Good signature from: -{0} -with key {1} -made at {2} -""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, - time.ctime(sign.timestamp))) -else: - pass -#+END_SRC - -#+BEGIN_SRC python -i -import gpg -import time - -filename = "statement.txt" -asc_file = "statement.txt.asc" - -c = gpg.Context() - -try: - data, result = c.verify(open(filename), open(asc_file)) - verified = True -except gpg.errors.BadSignatures as e: - verified = False - print(e) - -if verified is True: - for i in range(len(result.signatures)): - sign = result.signatures[i] - print("""Good signature from: -{0} -with key {1} -made at {2} -""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, - time.ctime(sign.timestamp))) -else: - pass -#+END_SRC - - -* Creating keys and subkeys - :PROPERTIES: - :CUSTOM_ID: key-generation - :END: - -The one thing, aside from GnuPG itself, that GPGME depends on, of -course, is the keys themselves. So it is necessary to be able to -generate them and modify them by adding subkeys, revoking or disabling -them, sometimes deleting them and doing the same for user IDs. - -In the following examples a key will be created for the world's -greatest secret agent, Danger Mouse. Since Danger Mouse is a secret -agent he needs to be able to protect information to =SECRET= level -clearance, so his keys will be 3072-bit keys. - -The pre-configured =gpg.conf= file which sets cipher, digest and other -preferences contains the following configuration parameters: - -#+BEGIN_SRC conf - expert - allow-freeform-uid - allow-secret-key-import - trust-model tofu+pgp - tofu-default-policy unknown - enable-large-rsa - enable-dsa2 - cert-digest-algo SHA512 - default-preference-list TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1 ZLIB BZIP2 ZIP Uncompressed - personal-cipher-preferences TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES - personal-digest-preferences SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1 - personal-compress-preferences ZLIB BZIP2 ZIP Uncompressed -#+END_SRC - - -** Primary key - :PROPERTIES: - :CUSTOM_ID: keygen-primary - :END: - -Generating a primary key uses the =create_key= method in a Context. -It contains multiple arguments and keyword arguments, including: -=userid=, =algorithm=, =expires_in=, =expires=, =sign=, =encrypt=, -=certify=, =authenticate=, =passphrase= and =force=. The defaults for -all of those except =userid=, =algorithm=, =expires_in=, =expires= and -=passphrase= is =False=. The defaults for =algorithm= and -=passphrase= is =None=. The default for =expires_in= is =0=. The -default for =expires= is =True=. There is no default for =userid=. - -If =passphrase= is left as =None= then the key will not be generated -with a passphrase, if =passphrase= is set to a string then that will -be the passphrase and if =passphrase= is set to =True= then gpg-agent -will launch pinentry to prompt for a passphrase. For the sake of -convenience, these examples will keep =passphrase= set to =None=. - -#+BEGIN_SRC python -i -import gpg - -c = gpg.Context() - -c.home_dir = "~/.gnupg-dm" -userid = "Danger Mouse " - -dmkey = c.create_key(userid, algorithm="rsa3072", expires_in=31536000, - sign=True, certify=True) -#+END_SRC - -One thing to note here is the use of setting the =c.home_dir= -parameter. This enables generating the key or keys in a different -location. In this case to keep the new key data created for this -example in a separate location rather than adding it to existing and -active key store data. As with the default directory, =~/.gnupg=, any -temporary or separate directory needs the permissions set to only -permit access by the directory owner. On posix systems this means -setting the directory permissions to 700. - -The =temp-homedir-config.py= script in the HOWTO examples directory -will create an alternative homedir with these configuration options -already set and the correct directory and file permissions. - -The successful generation of the key can be confirmed via the returned -=GenkeyResult= object, which includes the following data: - -#+BEGIN_SRC python -i -print(""" - Fingerprint: {0} - Primary Key: {1} - Public Key: {2} - Secret Key: {3} - Sub Key: {4} -User IDs: {5} -""".format(dmkey.fpr, dmkey.primary, dmkey.pubkey, dmkey.seckey, dmkey.sub, - dmkey.uid)) -#+END_SRC - -Alternatively the information can be confirmed using the command line -program: - -#+BEGIN_SRC shell - bash-4.4$ gpg --homedir ~/.gnupg-dm -K - ~/.gnupg-dm/pubring.kbx - ---------------------- - sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] - 177B7C25DB99745EE2EE13ED026D2F19E99E63AA - uid [ultimate] Danger Mouse - - bash-4.4$ -#+END_SRC - -As with generating keys manually, to preconfigure expanded preferences -for the cipher, digest and compression algorithms, the =gpg.conf= file -must contain those details in the home directory in which the new key -is being generated. I used a cut down version of my own =gpg.conf= -file in order to be able to generate this: - -#+BEGIN_SRC shell - bash-4.4$ gpg --homedir ~/.gnupg-dm --edit-key 177B7C25DB99745EE2EE13ED026D2F19E99E63AA showpref quit - Secret key is available. - - sec rsa3072/026D2F19E99E63AA - created: 2018-03-15 expires: 2019-03-15 usage: SC - trust: ultimate validity: ultimate - [ultimate] (1). Danger Mouse - - [ultimate] (1). Danger Mouse - Cipher: TWOFISH, CAMELLIA256, AES256, CAMELLIA192, AES192, CAMELLIA128, AES, BLOWFISH, IDEA, CAST5, 3DES - Digest: SHA512, SHA384, SHA256, SHA224, RIPEMD160, SHA1 - Compression: ZLIB, BZIP2, ZIP, Uncompressed - Features: MDC, Keyserver no-modify - - bash-4.4$ -#+END_SRC - - -** Subkeys - :PROPERTIES: - :CUSTOM_ID: keygen-subkeys - :END: - -Adding subkeys to a primary key is fairly similar to creating the -primary key with the =create_subkey= method. Most of the arguments -are the same, but not quite all. Instead of the =userid= argument -there is now a =key= argument for selecting which primary key to add -the subkey to. - -In the following example an encryption subkey will be added to the -primary key. Since Danger Mouse is a security conscious secret agent, -this subkey will only be valid for about six months, half the length -of the primary key. - -#+BEGIN_SRC python -i -import gpg - -c = gpg.Context() -c.home_dir = "~/.gnupg-dm" - -key = c.get_key(dmkey.fpr, secret=True) -dmsub = c.create_subkey(key, algorithm="rsa3072", expires_in=15768000, - encrypt=True) -#+END_SRC - -As with the primary key, the results here can be checked with: - -#+BEGIN_SRC python -i -print(""" - Fingerprint: {0} - Primary Key: {1} - Public Key: {2} - Secret Key: {3} - Sub Key: {4} -User IDs: {5} -""".format(dmsub.fpr, dmsub.primary, dmsub.pubkey, dmsub.seckey, dmsub.sub, - dmsub.uid)) -#+END_SRC - -As well as on the command line with: - -#+BEGIN_SRC shell - bash-4.4$ gpg --homedir ~/.gnupg-dm -K - ~/.gnupg-dm/pubring.kbx - ---------------------- - sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] - 177B7C25DB99745EE2EE13ED026D2F19E99E63AA - uid [ultimate] Danger Mouse - ssb rsa3072 2018-03-15 [E] [expires: 2018-09-13] - - bash-4.4$ -#+END_SRC - - -** User IDs - :PROPERTIES: - :CUSTOM_ID: keygen-uids - :END: - - -*** Adding User IDs - :PROPERTIES: - :CUSTOM_ID: keygen-uids-add - :END: - -By comparison to creating primary keys and subkeys, adding a new user -ID to an existing key is much simpler. The method used to do this is -=key_add_uid= and the only arguments it takes are for the =key= and -the new =uid=. - -#+BEGIN_SRC python -i -import gpg - -c = gpg.Context() -c.home_dir = "~/.gnupg-dm" - -dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" -key = c.get_key(dmfpr, secret=True) -uid = "Danger Mouse " - -c.key_add_uid(key, uid) -#+END_SRC - -Unsurprisingly the result of this is: - -#+BEGIN_SRC shell - bash-4.4$ gpg --homedir ~/.gnupg-dm -K - ~/.gnupg-dm/pubring.kbx - ---------------------- - sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] - 177B7C25DB99745EE2EE13ED026D2F19E99E63AA - uid [ultimate] Danger Mouse - uid [ultimate] Danger Mouse - ssb rsa3072 2018-03-15 [E] [expires: 2018-09-13] - - bash-4.4$ -#+END_SRC - - -*** Revoking User IDs - :PROPERTIES: - :CUSTOM_ID: keygen-uids-revoke - :END: - -Revoking a user ID is a fairly similar process, except that it uses -the =key_revoke_uid= method. - -#+BEGIN_SRC python -i -import gpg - -c = gpg.Context() -c.home_dir = "~/.gnupg-dm" - -dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" -key = c.get_key(dmfpr, secret=True) -uid = "Danger Mouse " - -c.key_revoke_uid(key, uid) -#+END_SRC - - -** Key certification - :PROPERTIES: - :CUSTOM_ID: key-sign - :END: - -Since key certification is more frequently referred to as key signing, -the method used to perform this function is =key_sign=. - -The =key_sign= method takes four arguments: =key=, =uids=, -=expires_in= and =local=. The default value of =uids= is =None= and -which results in all user IDs being selected. The default value of -both =expires_in= and =local= is =False=; which results in the -signature never expiring and being able to be exported. - -The =key= is the key being signed rather than the key doing the -signing. To change the key doing the signing refer to the signing key -selection above for signing messages and files. - -If the =uids= value is not =None= then it must either be a string to -match a single user ID or a list of strings to match multiple user -IDs. In this case the matching of those strings must be precise and -it is case sensitive. - -To sign Danger Mouse's key for just the initial user ID with a -signature which will last a little over a month, do this: - -#+BEGIN_SRC python -i -import gpg - -c = gpg.Context() -uid = "Danger Mouse " - -dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" -key = c.get_key(dmfpr, secret=True) -c.key_sign(key, uids=uid, expires_in=2764800) -#+END_SRC - - -*** Verifying key certifications - :PROPERTIES: - :CUSTOM_ID: key-sign-verify - :END: - -#+BEGIN_SRC python -i -import gpg -import time - -c = gpg.Context() -dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" -keys = list(c.keylist(pattern=dmuid, mode=gpg.constants.keylist.mode.SIGS)) -key = keys[0] - -for user in key.uids: - for sig in user.signatures: - print("0x{0}".format(sig.keyid), "", time.ctime(sig.timestamp), "", - sig.uid) -#+END_SRC - -Which for Danger Mouse displays the following: - -#+BEGIN_EXAMPLE - 0x92E3F6115435C65A Thu Mar 15 13:17:44 2018 Danger Mouse - 0x321E4E2373590E5D Mon Nov 26 12:46:05 2018 Ben McGinnes -#+END_EXAMPLE - -The two key signatures listed are for the self-certification of Danger -Mouse's key made when the key was created in March, 2018; and the -second is a signature made by the author and set to expire at the end -of the year. Note that the second signature was made with the -following code (including the preceding code to display the output of -the certifications or key signatures): - -#+BEGIN_SRC python -i -import gpg -import math -import pendulum -import time - -hd = "/home/dm/.gnupg" -c = gpg.Context() -d = gpg.Context(home_dir=hd) -dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" -dmuid = "Danger Mouse " -dkeys = list(c.keylist(pattern=dmuid)) -dmkey = dkeys[0] - -c.key_import(d.key_export(pattern=None)) - -tp = pendulum.period(pendulum.now(tz="local"), pendulum.datetime(2019, 1, 1)) -ts = tp.total_seconds() -total_secs = math.ceil(ts) -c.key_sign(dmkey, uids=dmuid, expires_in=total_secs) - -d.key_import(c.key_export(pattern=dmuid)) -keys = list(c.keylist(pattern=dmuid, mode=gpg.constants.keylist.mode.SIGS)) -key = keys[0] - -for user in key.uids: - for sig in user.signatures: - print("0x{0}".format(sig.keyid), "", time.ctime(sig.timestamp), "", - sig.uid) -#+END_SRC - -Note that this final code block includes the use of a module which is -/not/ part of Python's standard library, the [[https://pendulum.eustace.io/][pendulum module]]. Unlike -the standard datetime module, pendulum makes working with dates and -times significantly easier in Python; just as the requests module -makes working with HTTP and HTTPS easier than the builtin modules do. - -Though neither requests nor pendulum are required modules for using -the GPGME Python bindings, they are both highly recommended more -generally. - - -* Advanced or Experimental Use Cases - :PROPERTIES: - :CUSTOM_ID: advanced-use - :END: - - -** C plus Python plus SWIG plus Cython - :PROPERTIES: - :CUSTOM_ID: cython - :END: - -In spite of the apparent incongruence of using Python bindings to a C -interface only to generate more C from the Python; it is in fact quite -possible to use the GPGME bindings with [[http://docs.cython.org/en/latest/index.html][Cython]]. Though in many cases -the benefits may not be obvious since the most computationally -intensive work never leaves the level of the C code with which GPGME -itself is interacting with. - -Nevertheless, there are some situations where the benefits are -demonstrable. One of the better and easier examples being the one of -the early examples in this HOWTO, the [[#howto-keys-counting][key counting]] code. Running that -example as an executable Python script, =keycount.py= (available in -the =examples/howto/= directory), will take a noticeable amount of time -to run on most systems where the public keybox or keyring contains a -few thousand public keys. - -Earlier in the evening, prior to starting this section, I ran that -script on my laptop; as I tend to do periodically and timed it using -=time= utility, with the following results: - -#+BEGIN_SRC shell - bash-4.4$ time keycount.py - - Number of secret keys: 23 - Number of public keys: 12112 - - - real 11m52.945s - user 0m0.913s - sys 0m0.752s - - bash-4.4$ -#+END_SRC - -Sometime after that I imported another key and followed it with a -little test of Cython. This test was kept fairly basic, essentially -lifting the material from the [[http://docs.cython.org/en/latest/src/tutorial/cython_tutorial.html][Cython Basic Tutorial]] to demonstrate -compiling Python code to C. The first step was to take the example -key counting code quoted previously, essentially from the importing of -the =gpg= module to the end of the script: - -#+BEGIN_SRC python -i -import gpg - -c = gpg.Context() -seckeys = c.keylist(pattern=None, secret=True) -pubkeys = c.keylist(pattern=None, secret=False) - -seclist = list(seckeys) -secnum = len(seclist) - -publist = list(pubkeys) -pubnum = len(publist) - -print(""" - Number of secret keys: {0} - Number of public keys: {1} - -""".format(secnum, pubnum)) -#+END_SRC - -Save that into a file called =keycount.pyx= and then create a -=setup.py= file which contains this: - -#+BEGIN_SRC python -i -from distutils.core import setup -from Cython.Build import cythonize - -setup( - ext_modules = cythonize("keycount.pyx") -) -#+END_SRC - -Compile it: - -#+BEGIN_SRC shell - bash-4.4$ python setup.py build_ext --inplace - bash-4.4$ -#+END_SRC - -Then run it in a similar manner to =keycount.py=: - -#+BEGIN_SRC shell - bash-4.4$ time python3.7 -c "import keycount" - - Number of secret keys: 23 - Number of public keys: 12113 - - - real 6m47.905s - user 0m0.785s - sys 0m0.331s - - bash-4.4$ -#+END_SRC - -Cython turned =keycount.pyx= into an 81KB =keycount.o= file in the -=build/= directory, a 24KB =keycount.cpython-37m-darwin.so= file to be -imported into Python 3.7 and a 113KB =keycount.c= generated C source -code file of nearly three thousand lines. Quite a bit bigger than the -314 bytes of the =keycount.pyx= file or the full 1,452 bytes of the -full executable =keycount.py= example script. - -On the other hand it ran in nearly half the time; taking 6 minutes and -47.905 seconds to run. As opposed to the 11 minutes and 52.945 seconds -which the CPython script alone took. - -The =keycount.pyx= and =setup.py= files used to generate this example -have been added to the =examples/howto/advanced/cython/= directory -The example versions include some additional options to annotate the -existing code and to detect Cython's use. The latter comes from the -[[http://docs.cython.org/en/latest/src/tutorial/pure.html#magic-attributes-within-the-pxd][Magic Attributes]] section of the Cython documentation. - - -* Miscellaneous extras and work-arounds - :PROPERTIES: - :CUSTOM_ID: cheats-and-hacks - :END: - -Most of the things in the following sections are here simply because -there was no better place to put them, even though some are only -peripherally related to the GPGME Python bindings. Some are also -workarounds for functions not integrated with GPGME as yet. This is -especially true of the first of these, dealing with [[#group-lines][group lines]]. - - -** Group lines - :PROPERTIES: - :CUSTOM_ID: group-lines - :END: - -There is not yet an easy way to access groups configured in the -gpg.conf file from within GPGME. As a consequence these central -groupings of keys cannot be shared amongst multiple programs, such as -MUAs readily. - -The following code, however, provides a work-around for obtaining this -information in Python. - -#+BEGIN_SRC python -i -import subprocess -import sys - -if sys.platform == "win32": - gpgconfcmd = "gpgconf.exe --list-options gpg" -else: - gpgconfcmd = "gpgconf --list-options gpg" - -process = subprocess.Popen(gpgconfcmd.split(), stdout=subprocess.PIPE) -procom = process.communicate() - -if sys.version_info[0] == 2: - lines = procom[0].splitlines() -else: - lines = procom[0].decode().splitlines() - -for line in lines: - if line.startswith("group") is True: - break - -groups = line.split(":")[-1].replace('"', '').split(',') - -group_lines = [] -group_lists = [] - -for group in groups: - group_lines.append(group.split("=")) - group_lists.append(group.split("=")) - -for glist in group_lists: - glist[1] = glist[1].split() -#+END_SRC - -The result of that code is that =group_lines= is a list of lists where -=group_lines[i][0]= is the name of the group and =group_lines[i][1]= -is the key IDs of the group as a string. - -The =group_lists= result is very similar in that it is a list of -lists. The first part, =group_lists[i][0]= matches -=group_lines[i][0]= as the name of the group, but =group_lists[i][1]= -is the key IDs of the group as a list. - -A demonstration of using the =groups.py= module is also available in -the form of the executable =mutt-groups.py= script. This second -script reads all the group entries in a user's =gpg.conf= file and -converts them into crypt-hooks suitable for use with the Mutt and -Neomutt mail clients. - - -** Keyserver access for Python - :PROPERTIES: - :CUSTOM_ID: hkp4py - :END: - -The [[https://github.com/Selfnet/hkp4py][hkp4py]] module by Marcel Fest was originally a port of the old -[[https://github.com/dgladkov/python-hkp][python-hkp]] module from Python 2 to Python 3 and updated to use the -[[http://docs.python-requests.org/en/latest/index.html][requests]] module instead. It has since been modified to provide -support for Python 2.7 as well and is available via PyPI. - -Since it rewrites the =hkp= protocol prefix as =http= and =hkps= as -=https=, the module is able to be used even with servers which do not -support the full scope of keyserver functions.[fn:5] It also works quite -readily when incorporated into a [[#cython][Cython]] generated and compiled version -of any code. - - -*** Key import format - :PROPERTIES: - :CUSTOM_ID: hkp4py-strings - :END: - -The hkp4py module returns key data via requests as string literals -(=r.text=) instead of byte literals (=r.content=). This means that -the retrurned key data must be encoded to UTF-8 when importing that -key material using a =gpg.Context().key_import()= method. - -For this reason an alternative method has been added to the =search= -function of =hkp4py.KeyServer()= which returns the key in the correct -format as expected by =key_import=. When importing using this module, -it is now possible to import with this: - -#+BEGIN_SRC python -i -for key in keys: - if key.revoked is False: - gpg.Context().key_import(key.key_blob) - else: - pass -#+END_SRC - -Without that recent addition it would have been necessary to encode -the contents of each =hkp4py.KeyServer().search()[i].key= in -=hkp4py.KeyServer().search()= before trying to import it. - -An example of this is included in the [[#howto-import-key][Importing Keys]] section of this -HOWTO and the corresponding executable version of that example is -available in the =lang/python/examples/howto= directory as normal; the -executable version is the =import-keys-hkp.py= file. - - -** GPGME version checking - :PROPERTIES: - :CUSTOM_ID: gpgme-version-check - :END: - -For various reasons it may be necessary to check which version of -GPGME the bindings have been built against; including whether a -minimum required version of GPGME is in use. - -For the most part the =gpg.version.versionstr= and -=gpg.version.versionlist= methods have been quite sufficient. The -former returns the same string as =gpgme-config --version=, while the -latter returns the major, minor and patch values in a list. - -To check if the installed bindings have actually been built against -the current installed libgpgme version, this check can be performed: - -#+BEGIN_SRC python -i -import gpg -import subprocess -import sys - -gpgme_version_call = subprocess.Popen(["gpgme-config", "--version"], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE) -gpgme_version_str = gpgme_version_call.communicate() - -if sys.version_info[0] == 2: - gpgme_version = gpgme_version_str[0].strip() -elif sys.version_info[0] >= 3: - gpgme_version = gpgme_version_str[0].decode().strip() -else: - gpgme_version = None - -if gpgme_version is not None: - if gpgme_version == gpg.version.versionstr: - print("The GPGME Python bindings match libgpgme.") - else: - print("The GPGME Python bindings do NOT match libgpgme.") -else: - print("Upgrade Python and reinstall the GPGME Python bindings.") -#+END_SRC - -For many developers, however, the preferred checking means checking -for a minimum version or point release. This is now readily available -via the =gpg.version.versionintlist= method (added in version -=1.12.1-beta79=). It is also now possible to easily check whether the -installed GPGME Python bindings were built from a development or beta -branch of the GPGME source code. - -The following code demonstrates how both of those methods may be used: - -#+BEGIN_SRC python -i -import gpg - -try: - if gpg.version.is_beta is True: - print("The installed GPGME Python bindings were built from beta code.") - else: - print("The installed GPGME Python bindings are a released version.") -except Exception as e: - print(e) - -try: - if gpg.version.versionintlist[0] == 1: - if gpg.version.versionintlist[1] == 12: - if gpg.version.versionintlist[2] == 1: - print("This is the minimum version for using versionintlist.") - elif gpg.version.versionintlist[2] > 1: - print("The versionintlist method is available.") - else: - pass - elif gpg.version.versionintlist[1] > 12: - print("The versionintlist method is available.") - else: - pass - elif gpg.version.versionintlist[0] > 1: - print("The versionintlist method is available.") - else: - pass -except Exception as e: - print(e) -#+END_SRC - -The points where =pass= is used in the above example will most likely -also produce an =Exception= error since those results should only -occur in versions which do not have the =gpgme.version.is_beta= and -=gpgme.version.versionintlist= methods available. - - -* Copyright and Licensing - :PROPERTIES: - :CUSTOM_ID: copyright-and-license - :END: - - -** Copyright - :PROPERTIES: - :CUSTOM_ID: copyright - :END: - -Copyright © The GnuPG Project, 2018. - - -** Draft Editions of this HOWTO - :PROPERTIES: - :CUSTOM_ID: draft-editions - :END: - -Draft editions of this HOWTO may be periodically available directly -from the author at any of the following URLs: - -- [[https://files.au.adversary.org/crypto/gpgme-python-howto.html][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 SSL)]] -- [[http://files.au.adversary.org/crypto/gpgme-python-howto.html][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 no SSL)]] -- [[https://files.au.adversary.org/crypto/gpgme-python-howto-split/index.html][GPGME Python Bindings HOWTO draft (HTML multiple files, AWS S3 SSL)]] -- [[http://files.au.adversary.org/crypto/gpgme-python-howto/index.html][GPGME Python Bindings HOWTO draft (HTML multiple files, AWS S3 no SSL)]] - -These draft versions have been generated from this document via GNU -Emacs [[https://orgmode.org/][Org mode]] to =.texi= and [[https://www.gnu.org/software/texinfo/][GNU Texinfo]] to HTML. Though it is -likely that the specific [[https://files.au.adversary.org/crypto/gpgme-python-howto][file]] [[http://files.au.adversary.org/crypto/gpgme-python-howto.org][version]] used will be on the same server -with the generated output formats. Occasionally I may include the Org -mode generated XHTML versions: - -- [[https://files.au.adversary.org/crypto/gpgme-python-howto.xhtml][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 SSL)]] -- [[http://files.au.adversary.org/crypto/gpgme-python-howto.xhtml][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 no SSL)]] - -That XHTML version, however, is exported in a way which inherits a -colour scheme from [[https://github.com/holomorph/emacs-zenburn][the author's Emacs theme]] (which is a higher contrast -version of [[http://kippura.org/zenburnpage/][Zenburn]] ported by [[https://github.com/holomorph][Holomorph]]). So it's fine for people who -prefer dark themed web pages, but not so great for everyone else. - -The GNU Texinfo and reStructured Text versions ship with the software, -while the GNU Emacs Info version is generated from the Texinfo -version using GNU Texinfo or GNU Makeinfo. The Texinfo format is -generated from the original Org mode source file in Org mode itself -either within GNU Emacs or via the command line by invoking Emacs in -batch mode: - -#+BEGIN_SRC shell - emacs gpgme-python-howto.org --batch -f org-texinfo-export-to-texinfo --kill - emacs gpgme-python-howto --batch -f org-texinfo-export-to-texinfo --kill -#+END_SRC - -The reStructuredText format is also generated from the Org mode source -file, except it is generated using [[https://pandoc.org][Pandoc]] with either of the following -commands (depending on the filename): - -#+BEGIN_SRC shell - pandoc -f org -t rst+smart -o gpgme-python-howto.rst gpgme-python-howto.org - pandoc -f org -t rst+smart -o gpgme-python-howto.rst gpgme-python-howto -#+END_SRC - -Note that the Org mode source files are identified as such via a mode -line at the top of each file and have had their =.org= file extensions -dropped in order to make scripted generation of output formats easier -and not require renaming files post-conversion. - -Due to a bug in Org mode's texinfo conversion method, the recommended -steps for generating the Texinfo files for all the files in the -=lang/python/doc/src/= directory are as follows: - -#+BEGIN_SRC shell - for x in * ; do - emacs $x --batch -f org-texinfo-export-to-texinfo --kill - cat $x.texi | sed -e 's/@documentencoding UTF-8/@documentencoding utf-8/g' > ../texinfo/$x.texi - pandoc -f org -t rst+smart -o ../rst/$x.rst $x - done ; - rm -fv *.texi - cd ../texinfo - mkdir info - mkdir html - for x in *.texi ; do - makeinfo -v $x - makeinfo --html --no-split $x - done ; - mv *.info info/ - mv *.html html/ -#+END_SRC - -This code snippet includes the generation of the reStructuredText -files and would be expected to be run from the =doc/src/= directory -containing the Org mode source files. It also assumes that the -commands are being run on POSIX compliant systems with basic tools -like sed, the Bourne shell and GNU Emacs[fn:6] available. The code -snippet also includes the steps for generating the Emacs Info files -and HTML files from the Texinfo files. Using reStructuredText files -with Sphinx is best left for the documentation of that project. - -In addition to these there is a significantly less frequently updated -version as a HTML [[https://files.au.adversary.org/crypto/gpgme-python/dita/webhelp/index.html][WebHelp site]] (AWS S3 SSL); generated from DITA XML -source files, which can be found in [[https://dev.gnupg.org/source/gpgme/browse/ben%252Fhowto-dita/][an alternative branch]] of the GPGME -git repository. - -Various generated output formats may occasionally be found in -subdirectories of the [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python][gpgme-python]] directory. In particular within -the [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python/dita][DITA]], [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python/rst][reStructuredText]] and [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python/texinfo][Texinfo]] subdirectories. The =rst= -directory contains output files generated with Sphinx and may include a -considerable number of its possible output formats, but there are no -guarantees as to how recent these are or even if they are present. - -These draft editions are not official documents and the version of -documentation in the master branch or which ships with released -versions is the only official documentation. Nevertheless, these -draft editions may occasionally be of use by providing more accessible -web versions which are updated between releases. They are provided on -the understanding that they may contain errors or may contain content -subject to change prior to an official release. - - -** License GPL compatible - :PROPERTIES: - :CUSTOM_ID: license - :END: - -This file is free software; as a special exception the author gives -unlimited permission to copy and/or distribute it, with or without -modifications, as long as this notice is preserved. - -This file is distributed in the hope that it will be useful, but -WITHOUT ANY WARRANTY, to the extent permitted by law; without even the -implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR -PURPOSE. - - -* Footnotes - -[fn:1] =short-history= and/or =short-history.html=. - -[fn:2] With no issues reported specific to Python 3.7, the release of -Python 3.7.1 at around the same time as GPGME 1.12.0 and the testing -with Python 3.7.1rc1, there is no reason to delay moving 3.7 ahead of -3.6 now. Production environments with more conservative requirements -will always enforce their own policies anyway and installation to each -supported minor release is quite possible too. - -[fn:3] Yes, even if you use virtualenv with everything you do in -Python. If you want to install this module as just your user account -then you will need to manually configure, compile and install the -/entire/ GnuPG stack as that user as well. This includes libraries -which are not often installed that way. It can be done and there are -circumstances under which it is worthwhile, but generally only on -POSIX systems which utilise single user mode (some even require it). - -[fn:4] You probably don't really want to do this. Searching the -keyservers for "gnupg.org" produces over 400 results, the majority of -which aren't actually at the gnupg.org domain, but just included a -comment regarding the project in their key somewhere. - -[fn:5] Such as with ProtonMail servers. This also means that -restricted servers which only advertise either HTTP or HTTPS end -points and not HKP or HKPS end points must still be identified as as -HKP or HKPS within the Python Code. The =hkp4py= module will rewrite -these appropriately when the connection is made to the server. - -[fn:6] Okay, Emacs might not necessarily qualify as a basic tool, but -it is common enough that having it installed on a system isn't too -great an expectation, nor is it difficult to add to most POSIX -systems, even if the users of those systems do not personally use it. diff --git a/lang/python/doc/src/gpgme-python-howto.org b/lang/python/doc/src/gpgme-python-howto.org new file mode 100644 index 00000000..e2e7e719 --- /dev/null +++ b/lang/python/doc/src/gpgme-python-howto.org @@ -0,0 +1,3373 @@ +# -*- mode: org -*- +#+TITLE: GNU Privacy Guard (GnuPG) Made Easy Python Bindings HOWTO (English) +#+AUTHOR: Ben McGinnes +#+LATEX_COMPILER: xelatex +#+LATEX_CLASS: article +#+LATEX_CLASS_OPTIONS: [12pt] +#+LATEX_HEADER: \usepackage{xltxtra} +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} +#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} +#+LATEX_HEADER: \author{Ben McGinnes } + + +* Introduction + :PROPERTIES: + :CUSTOM_ID: intro + :END: + +| Version: | 0.1.5 | +| GPGME Version: | 1.13.0 | +| Author: | Ben McGinnes | +| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | +| Language: | Australian English, British English | +| Language codes: | en-AU, en-GB, en | + +This document provides basic instruction in how to use the GPGME +Python bindings to programmatically leverage the GPGME library. + + +** Python 2 versus Python 3 + :PROPERTIES: + :CUSTOM_ID: py2-vs-py3 + :END: + +Though the GPGME Python bindings themselves provide support for both +Python 2 and 3, the focus is unequivocally on Python 3 and +specifically from Python 3.4 and above. As a consequence all the +examples and instructions in this guide use Python 3 code. + +Much of it will work with Python 2, but much of it also deals with +Python 3 byte literals, particularly when reading and writing data. +Developers concentrating on Python 2.7, and possibly even 2.6, will +need to make the appropriate modifications to support the older string +and unicode types as opposed to bytes. + +There are multiple reasons for concentrating on Python 3; some of +which relate to the immediate integration of these bindings, some of +which relate to longer term plans for both GPGME and the python +bindings and some of which relate to the impending EOL period for +Python 2.7. Essentially, though, there is little value in tying the +bindings to a version of the language which is a dead end and the +advantages offered by Python 3 over Python 2 make handling the data +types with which GPGME deals considerably easier. + + +** Examples + :PROPERTIES: + :CUSTOM_ID: howto-python3-examples + :END: + +All of the examples found in this document can be found as Python 3 +scripts in the =lang/python/examples/howto= directory. + + +** Unofficial Drafts + :PROPERTIES: + :CUSTOM_ID: unofficial-drafts + :END: + +In addition to shipping with each release of GPGME, there is a section +on locations to read or download [[#draft-editions][draft editions]] of this document from +at the end of it. These are unofficial versions produced in between +major releases. + + +** What's New + :PROPERTIES: + :CUSTOM_ID: new-stuff + :END: + +Full details of what is new are now available in the [[file:what-is-new.org][What's New]] file +and archives of the preceding /What's New/ sections are available in +the [[file:what-was-new][What Was New]] file. + + +*** New in GPGME 1·13·0 + :PROPERTIES: + :CUSTOM_ID: new-stuff-1-13-0 + :END: + +See the [[file:what-is-new#new-stuff-1-13-0][What's New]] document for what is new in version 1.13.0. + + +*** New in GPGME 1·12·0 + :PROPERTIES: + :CUSTOM_ID: new-stuff-1-12-0 + :END: + +See the [[file:what-was-new#new-stuff-1-12-0][What Was New]] document for what was new in version 1.12.0. + + +* GPGME Concepts + :PROPERTIES: + :CUSTOM_ID: gpgme-concepts + :END: + + +** A C API + :PROPERTIES: + :CUSTOM_ID: gpgme-c-api + :END: + +Unlike many modern APIs with which programmers will be more familiar +with these days, the GPGME API is a C API. The API is intended for +use by C coders who would be able to access its features by including +the =gpgme.h= header file with their own C source code and then access +its functions just as they would any other C headers. + +This is a very effective method of gaining complete access to the API +and in the most efficient manner possible. It does, however, have the +drawback that it cannot be directly used by other languages without +some means of providing an interface to those languages. This is +where the need for bindings in various languages stems. + + +** Python bindings + :PROPERTIES: + :CUSTOM_ID: gpgme-python-bindings + :END: + +The Python bindings for GPGME provide a higher level means of +accessing the complete feature set of GPGME itself. It also provides +a more pythonic means of calling these API functions. + +The bindings are generated dynamically with SWIG and the copy of +=gpgme.h= generated when GPGME is compiled. + +This means that a version of the Python bindings is fundamentally tied +to the exact same version of GPGME used to generate that copy of +=gpgme.h=. + + +** Difference between the Python bindings and other GnuPG Python packages + :PROPERTIES: + :CUSTOM_ID: gpgme-python-bindings-diffs + :END: + +There have been numerous attempts to add GnuPG support to Python over +the years. Some of the most well known are listed here, along with +what differentiates them. + + +*** The python-gnupg package maintained by Vinay Sajip + :PROPERTIES: + :CUSTOM_ID: diffs-python-gnupg + :END: + +This is arguably the most popular means of integrating GPG with +Python. The package utilises the =subprocess= module to implement +wrappers for the =gpg= and =gpg2= executables normally invoked on the +command line (=gpg.exe= and =gpg2.exe= on Windows). + +The popularity of this package stemmed from its ease of use and +capability in providing the most commonly required features. + +Unfortunately it has been beset by a number of security issues in the +past; most of which stemmed from using unsafe methods of accessing the +command line via the =subprocess= calls. While some effort has been +made over the last two to three years (as of 2018) to mitigate this, +particularly by no longer providing shell access through those +subprocess calls, the wrapper is still somewhat limited in the scope +of its GnuPG features coverage. + +The python-gnupg package is available under the MIT license. + + +*** The gnupg package created and maintained by Isis Lovecruft + :PROPERTIES: + :CUSTOM_ID: diffs-isis-gnupg + :END: + +In 2015 Isis Lovecruft from the Tor Project forked and then +re-implemented the python-gnupg package as just gnupg. This new +package also relied on subprocess to call the =gpg= or =gpg2= +binaries, but did so somewhat more securely. + +The naming and version numbering selected for this package, however, +resulted in conflicts with the original python-gnupg and since its +functions were called in a different manner to python-gnupg, the +release of this package also resulted in a great deal of consternation +when people installed what they thought was an upgrade that +subsequently broke the code relying on it. + +The gnupg package is available under the GNU General Public License +version 3.0 (or any later version). + + +*** The PyME package maintained by Martin Albrecht + :PROPERTIES: + :CUSTOM_ID: diffs-pyme + :END: + +This package is the origin of these bindings, though they are somewhat +different now. For details of when and how the PyME package was +folded back into GPGME itself see the [[file:short-history.org][Short History]] document.[fn:1] + +The PyME package was first released in 2002 and was also the first +attempt to implement a low level binding to GPGME. In doing so it +provided access to considerably more functionality than either the +=python-gnupg= or =gnupg= packages. + +The PyME package is only available for Python 2.6 and 2.7. + +Porting the PyME package to Python 3.4 in 2015 is what resulted in it +being folded into the GPGME project and the current bindings are the +end result of that effort. + +The PyME package is available under the same dual licensing as GPGME +itself: the GNU General Public License version 2.0 (or any later +version) and the GNU Lesser General Public License version 2.1 (or any +later version). + + +* GPGME Python bindings installation + :PROPERTIES: + :CUSTOM_ID: gpgme-python-install + :END: + + +** No PyPI + :PROPERTIES: + :CUSTOM_ID: do-not-use-pypi + :END: + +Most third-party Python packages and modules are available and +distributed through the Python Package Installer, known as PyPI. + +Due to the nature of what these bindings are and how they work, it is +infeasible to install the GPGME Python bindings in the same way. + +This is because the bindings use SWIG to dynamically generate C +bindings against =gpgme.h= and =gpgme.h= is generated from +=gpgme.h.in= at compile time when GPGME is built from source. Thus to +include a package in PyPI which actually built correctly would require +either statically built libraries for every architecture bundled with +it or a full implementation of C for each architecture. + +See the additional notes regarding [[#snafu-cffi][CFFI and SWIG]] at the end of this +section for further details. + + +** Requirements + :PROPERTIES: + :CUSTOM_ID: gpgme-python-requirements + :END: + +The GPGME Python bindings only have three requirements: + +1. A suitable version of Python 2 or Python 3. With Python 2 that + means CPython 2.7 and with Python 3 that means CPython 3.4 or + higher. +2. [[https://www.swig.org][SWIG]]. +3. GPGME itself. Which also means that all of GPGME's dependencies + must be installed too. + + +*** Recommended Additions + :PROPERTIES: + :CUSTOM_ID: gpgme-python-recommendations + :END: + +Though none of the following are absolute requirements, they are all +recommended for use with the Python bindings. In some cases these +recommendations refer to which version(s) of CPython to use the +bindings with, while others refer to third party modules which provide +a significant advantage in some way. + +1. If possible, use Python 3 instead of 2. +2. Favour a more recent version of Python since even 3.4 is due to + reach EOL soon. In production systems and services, Python 3.6 + should be robust enough to be relied on. +3. If possible add the following Python modules which are not part of + the standard library: [[http://docs.python-requests.org/en/latest/index.html][Requests]], [[https://cython.org/][Cython]], [[https://pendulum.eustace.io/][Pendulum]] and [[https://github.com/Selfnet/hkp4py][hkp4py]]. + +Chances are quite high that at least the first one and maybe two of +those will already be installed. + +Note that, as with Cython, some of advanced use case scenarios will +bring with them additional requirements. Most of these will be fairly +well known and commonly installed ones, however, which are in many +cases likely to have already been installed on many systems or be +familiar to Python programmers. + + +** Installation + :PROPERTIES: + :CUSTOM_ID: installation + :END: + +Installing the Python bindings is effectively achieved by compiling +and installing GPGME itself. + +Once SWIG is installed with Python and all the dependencies for GPGME +are installed you only need to confirm that the version(s) of Python +you want the bindings installed for are in your =$PATH=. + +By default GPGME will attempt to install the bindings for the most +recent or highest version number of Python 2 and Python 3 it detects +in =$PATH=. It specifically checks for the =python= and =python3= +executables first and then checks for specific version numbers. + +For Python 2 it checks for these executables in this order: =python=, +=python2= and =python2.7=. + +For Python 3 it checks for these executables in this order: =python3=, + =python3.7=, =python3.6=, =python3.5= and =python3.4=.[fn:2] + +On systems where =python= is actually =python3= and not =python2= it +may be possible that =python2= may be overlooked, but there have been +no reports of that actually occurring as yet. + +In the three months or so since the release of Python 3.7.0 there has +been extensive testing and work with these bindings with no issues +specifically relating to the new version of Python or any of the new +features of either the language or the bindings. This has also been +the case with Python 3.7.1rc1. With that in mind and given the +release of Python 3.7.1 is scheduled for around the same time as GPGME +1.12.0, the order of preferred Python versions has been changed to +move Python 3.7 ahead of Python 3.6. + + +*** Installing GPGME + :PROPERTIES: + :CUSTOM_ID: install-gpgme + :END: + +See the GPGME =README= file for details of how to install GPGME from +source. + + +** Known Issues + :PROPERTIES: + :CUSTOM_ID: snafu + :END: + +There are a few known issues with the current build process and the +Python bindings. For the most part these are easily addressed should +they be encountered. + + +*** Breaking Builds + :PROPERTIES: + :CUSTOM_ID: snafu-a-swig-of-this-builds-character + :END: + +Occasionally when installing GPGME with the Python bindings included +it may be observed that the =make= portion of that process induces a +large very number of warnings and, eventually errors which end that +part of the build process. Yet following that with =make check= and +=make install= appears to work seamlessly. + +The cause of this is related to the way SWIG needs to be called to +dynamically generate the C bindings for GPGME in the first place. So +the entire process will always produce =lang/python/python2-gpg/= and +=lang/python/python3-gpg/= directories. These should contain the +build output generated during compilation, including the complete +bindings and module installed into =site-packages=. + +Occasionally the errors in the early part or some other conflict +(e.g. not installing as */root/* or */su/*) may result in nothing +being installed to the relevant =site-packages= directory and the +build directory missing a lot of expected files. Even when this +occurs, the solution is actually quite simple and will always work. + +That solution is simply to run the following commands as either the +*root* user or prepended with =sudo -H=[fn:3] in the =lang/python/= +directory: + +#+BEGIN_SRC shell + /path/to/pythonX.Y setup.py build + /path/to/pythonX.Y setup.py build + /path/to/pythonX.Y setup.py install +#+END_SRC + +Yes, the build command does need to be run twice. Yes, you still need +to run the potentially failing or incomplete steps during the +=configure=, =make= and =make install= steps with installing GPGME. +This is because those steps generate a lot of essential files needed, +both by and in order to create, the bindings (including both the +=setup.py= and =gpgme.h= files). + + +**** IMPORTANT Note + :PROPERTIES: + :CUSTOM_ID: snafu-swig-build-note + :END: + +If specifying a selected number of languages to create bindings for, +try to leave Python last. Currently the majority of the other +language bindings are also preceding Python of either version when +listed alphabetically (not counting the Qt bindings). + +If Python is set to precede one of the other languages then it is +possible that the errors described here may interrupt the build +process before generating bindings for those other languages. In +these cases it may be preferable to configure all preferred language +bindings separately with alternative =configure= steps for GPGME using +the =--enable-languages=$LANGUAGE= option. + +Alternatively =make= (or =gmake=, depending on your platform) may be +run with the the =-k= option, which tells make to keep going even if +errors are encountered. In that case the failure of one language's +set of bindings to build should not hamper another language's bindings +to build. + + +*** Reinstalling Responsibly + :PROPERTIES: + :CUSTOM_ID: snafu-lessons-for-the-lazy + :END: + +Regardless of whether you're installing for one version of Python or +several, there will come a point where reinstallation is required. +With most Python module installations, the installed files go into the +relevant site-packages directory and are then forgotten about. Then +the module is upgraded, the new files are copied over the old and +that's the end of the matter. + +While the same is true of these bindings, there have been intermittent +issues observed on some platforms which have benefited significantly +from removing all the previous installations of the bindings before +installing the updated versions. + +Removing the previous version(s) is simply a matter of changing to the +relevant =site-packages= directory for the version of Python in +question and removing the =gpg/= directory and any accompanying +egg-info files for that module. + +In most cases this will require root or administration privileges on +the system, but the same is true of installing the module in the first +place. + + +*** Multiple installations + :PROPERTIES: + :CUSTOM_ID: snafu-the-full-monty + :END: + +For a variety of reasons it may be either necessary or just preferable +to install the bindings to alternative installed Python versions which +meet the requirements of these bindings. + +On POSIX systems this will generally be most simply achieved by +running the manual installation commands (build, build, install) as +described in the previous section for each Python installation the +bindings need to be installed to. + +As per the SWIG documentation: the compilers, libraries and runtime +used to build GPGME and the Python Bindings *must* match those used to +compile Python itself, including the version number(s) (at least going +by major version numbers and probably minor numbers too). + +On most POSIX systems, including OS X, this will very likely be the +case in most, if not all, cases. + +Note that from GPGME [[https://dev.gnupg.org/rMff6ff616aea6f59b7f2ce1176492850ecdf3851e][1.12.1]] the default installation installs to each +version of Python it can find first. That is that it will currently +install for the first copies of Python versions 2.7, 3.4, 3.5, 3.6, +3.7 and 3.8 (dev branch) that it finds. Usually this will be in the +same prefix as GPGME itself, but is dictated by the =$PATH= when the +installation is performed. The above instructions can still be +performed on other python installations which the installer does not +find, including alternative prefixes. + + + +*** Won't Work With Windows + :PROPERTIES: + :CUSTOM_ID: snafu-runtime-not-funtime + :END: + +There are semi-regular reports of Windows users having considerable +difficulty in installing and using the Python bindings at all. Very +often, possibly even always, these reports come from Cygwin users +and/or MinGW users and/or Msys2 users. Though not all of them have +been confirmed, it appears that these reports have also come from +people who installed Python using the Windows installer files from the +[[https://python.org][Python website]] (i.e. mostly MSI installers, sometimes self-extracting +=.exe= files). + +The Windows versions of Python are not built using Cygwin, MinGW or +Msys2; they're built using Microsoft Visual Studio. Furthermore the +version used is /considerably/ more advanced than the version which +MinGW obtained a small number of files from many years ago in order to +be able to compile anything at all. Not only that, but there are +changes to the version of Visual Studio between some micro releases, +though that is is particularly the case with Python 2.7, since it has +been kept around far longer than it should have been. + +There are two theoretical solutions to this issue: + + 1. Compile and install the GnuPG stack, including GPGME and the + Python bindings using the same version of Microsoft Visual Studio + used by the Python Foundation to compile the version of Python + installed. + + If there are multiple versions of Python then this will need to be + done with each different version of Visual Studio used for those + versions of Python. + + 2. Compile and install Python using the same tools used by choice, + such as MinGW or Msys2. + +Do *not* use the official Windows installer for Python unless +following the first method. + +In this type of situation it may even be for the best to accept that +there are less limitations on permissive software than free software +and simply opt to use a recent version of the Community Edition of +Microsoft Visual Studio to compile and build all of it, no matter +what. + +Investigations into the extent or the limitations of this issue are +ongoing. + +The following table lists the version of Microsoft Visual Studio which +needs to be used when compiling GPGME and the Python bindings with +each version of the CPython binary released [[https://www.python.org/downloads/windows/][for Windows]]: + +| CPython | Microsoft product name | runtime filename | +| 2.7.6 | Visual Studio 2008 | MSVCR90.DLL | +| 3.4.0 | Visual Studio 2010 | MSVCR100.DLL | +| 3.5.0 | Visual Studio 2015 | *see below* | +| 3.6.0 | Visual Studio 2015 | *see below* | +| 3.7.0 | Visual Studio 2017* | *see below* | + +It is important to note that MingW and Msys2 ship with the Visual C +runtime from Microsoft Visual Studio 2005 and are thus *incompatible* +with all the versions of CPython which can be used with the GPGME +Python bindings. + +It is also important to note that from CPython 3.5 onwards, the Python +Foundation has adopted the reworking of the Visual C runtime which was +performed for Visual Studio 2015 and aimed at resolving many of these +kinds of issues. Much greater detail on these issues and the correct +file(s) to link to are available from Matthew Brett's invaluable page, +[[https://matthew-brett.github.io/pydagogue/python_msvc.html][Using Microsoft Visual C with Python]]. It is also worth reading the +Microsoft Developer Network blog post on [[http://blogs.msdn.com/b/vcblog/archive/2015/03/03/introducing-the-universal-crt.aspx][the universal CRT]] and Steve +Dower's blog posts on Python extensions ([[http://stevedower.id.au/blog/building-for-python-3-5][part 1]] and [[http://stevedower.id.au/blog/building-for-python-3-5-part-two][part 2]]). + +The second of those two posts by Steve Dower contains the details of +specific configuration options required for compiling anything to be +used with official CPython releases. In addition to those +configuration and compiler settings to use, the versions of Visual +Studio prior to Visual Studio 2015 did not support 64-bit systems by +default. So compiling a 64-bit version of these bindings for a 64-bit +version of CPython 2.7 or 3.4 requires additional work. + +In addition to the blog posts, the [[https://wiki.python.org/moin/WindowsCompilers][Windows compilers]] wiki page on the +CPython wiki is another essential reference on the relevant versions +of Visual Studio to use and the degree of compatibility with CPython +releases. + +Eventually someone will ask why there isn't an installable binary for +Windows, which the GPGME of the licenses do not preclude as long as +the source code is available in conjunction with such a release. + +The sheer number of versions of Visual Studio in conjunction with +differing configuration options depending on the target Windows +version and whether the architecture is 64-bit or 32-bit makes it +difficult to provide a correct binary installer for Windows users. At +the bare minimum doing so would require the GnuPG project compile ten +different versions of the bindings with each release; both 32-bit and +64-bit versions for CPython 2.7 and 3.4, with 64-bit versions for both +x86-64 (i.e. Intel and AMD) and ARM architectures for CPython 3.5, +3.6, 3.7 and later releases. That's the bare *minimum*, it'd probably +be higher. + +Additionally, with only a binary installation used in conjunction with +the CPython installer from =python.org= the advanced options available +which utilise [[#cython][Cython]] will not be able to be used at all. Cython +depends on being able to compile the C code it generates and that too +would need to utilise a matching runtime to both the installed version +of CPython and these bindings in order to work with the bindings. + +Considering all of that, what do we recommend? + + 1. Use a recent version of CPython; at least 3.5, but ideally 3.6 or + later. + + 2. Use Visual Studio 2015 or the standalone build tools for Visual + Studio 2017 (or later). + + 3. Compile both CPython and GPGME with these bindings using the tools + selected in step 2. + + 4. Ignore MingW, Msys2 and the official CPython binary installers. + + 5. Be thankful the answer to this question wasn't simply to say + something like, “install Linux” or “install FreeBSD” (or even + Apple's OS X). + + +*** CFFI is the Best™ and GPGME should use it instead of SWIG + :PROPERTIES: + :CUSTOM_ID: snafu-cffi + :END: + +There are many reasons for favouring [[https://cffi.readthedocs.io/en/latest/overview.html][CFFI]] and proponents of it are +quite happy to repeat these things as if all it would take to switch +from SWIG to CFFI is repeating that list as if it were a new concept. + +The fact is that there are things which Python's CFFI implementation +cannot handle in the GPGME C code. Beyond that there are features of +SWIG which are simply not available with CFFI at all. SWIG generates +the bindings to Python using the =gpgme.h= file, but that file is not +a single version shipped with each release, it too is generated when +GPGME is compiled. + +CFFI is currently unable to adapt to such a potentially mutable +codebase. If there were some means of applying SWIG's dynamic code +generation to produce the Python/CFFI API modes of accessing the GPGME +libraries (or the source source code directly), but such a thing does +not exist yet either and it currently appears that work is needed in +at least one of CFFI's dependencies before any of this can be +addressed. + +So if you're a massive fan of CFFI; that's great, but if you want this +project to switch to CFFI then rather than just insisting that it +should, I'd suggest you volunteer to bring CFFI up to the level this +project needs. + +If you're actually seriously considering doing so, then I'd suggest +taking the =gpgme-tool.c= file in the GPGME =src/= directory and +getting that to work with any of the CFFI API methods (not the ABI +methods, they'll work with pretty much anything). When you start +running into trouble with "ifdefs" then you'll know what sort of +things are lacking. That doesn't even take into account the amount of +work saved via SWIG's code generation techniques either. + + +*** Virtualised Environments + :PROPERTIES: + :CUSTOM_ID: snafu-venv + :END: + +It is fairly common practice amongst Python developers to, as much as +possible, use packages like virtualenv to keep various things that are +to be installed from interfering with each other. Given how much of +the GPGME bindings is often at odds with the usual pythonic way of +doing things, it stands to reason that this would be called into +question too. + +As it happens the answer as to whether or not the bindings can be used +with virtualenv, the answer is both yes and no. + +In general we recommend installing to the relevant path and matching +prefix of GPGME itself. Which means that when GPGME, and ideally the +rest of the GnuPG stack, is installed to a prefix like =/usr/local= or +=/opt/local= then the bindings would need to be installed to the main +Python installation and not a virtualised abstraction. Attempts to +separate the two in the past have been known to cause weird and +intermittent errors ranging from minor annoyances to complete failures +in the build process. + +As a consequence we only recommend building with and installing to the +main Python installations within the same prefix as GPGME is installed +to or which are found by GPGME's configuration stage immediately prior +to running the make commands. Which is exactly what the compiling and +installing process of GPGME does by default. + +Once that is done, however, it appears that a copy of the compiled +module may be installed into a virtualenv of the same major and minor +version matching the build. Alternatively it is possible to utilise a +=sites.pth= file in the =site-packages/= directory of a virtualenv +installation, which links back to the system installations +corresponding directory in order to import anything installed system +wide. This may or may not be appropriate on a case by case basis. + +Though extensive testing of either of these options is not yet +complete, preliminary testing of them indicates that both are viable +as long as the main installation is complete. Which means that +certain other options normally restricted to virtual environments are +also available, including integration with pythonic test suites +(e.g. [[https://docs.pytest.org/en/latest/index.html][pytest]]) and other large projects. + +That said, it is worth reiterating the warning regarding non-standard +installations. If one were to attempt to install the bindings only to +a virtual environment without somehow also including the full GnuPG +stack (or enough of it as to include GPGME) then it is highly likely +that errors would be encountered at some point and more than a little +likely that the build process itself would break. + +If a degree of separation from the main operating system is still +required in spite of these warnings, then consider other forms of +virtualisation. Either a virtual machine (e.g. [[https://www.virtualbox.org/][VirtualBox]]), a +hardware emulation layer (e.g. [[https://www.qemu.org/][QEMU]]) or an application container +(e.g. [[https://www.docker.com/why-docker][Docker]]). + +Finally it should be noted that the limited tests conducted thus far +have been using the =virtualenv= command in a new directory to create +the virtual python environment. As opposed to the standard =python3 +-m venv= and it is possible that this will make a difference depending +on the system and version of Python in use. Another option is to run +the command =python3 -m virtualenv /path/to/install/virtual/thingy= +instead. + + +*** Post installation + :PROPERTIES: + :CUSTOM_ID: snafu-docs + :END: + +Following installation it is recommended to move the +=post_installer.py= script from the =lang/python/examples/howto/= +directory to the =lang/python/= directory and run it. This will fix +or restore files needed by Sphinx which may be removed during a +distribution build for release. It will also generate reST files from +Org mode files with Pandoc and generate Texinfo files from Org mode +files with GNU Emacs and Org mode (in batch mode). Additionally it +will fix the UTF-8 declaration line in the Texinfo files (Emacs +expects "UTF-8" to be "utf-8"). + + +* Fundamentals + :PROPERTIES: + :CUSTOM_ID: howto-fund-a-mental + :END: + +Before we can get to the fun stuff, there are a few matters regarding +GPGME's design which hold true whether you're dealing with the C code +directly or these Python bindings. + + +** No REST + :PROPERTIES: + :CUSTOM_ID: no-rest-for-the-wicked + :END: + +The first part of which is or will be fairly blatantly obvious upon +viewing the first example, but it's worth reiterating anyway. That +being that this API is /*not*/ a REST API. Nor indeed could it ever +be one. + +Most, if not all, Python programmers (and not just Python programmers) +know how easy it is to work with a RESTful API. In fact they've +become so popular that many other APIs attempt to emulate REST-like +behaviour as much as they are able. Right down to the use of JSON +formatted output to facilitate the use of their API without having to +retrain developers. + +This API does not do that. It would not be able to do that and also +provide access to the entire C API on which it's built. It does, +however, provide a very pythonic interface on top of the direct +bindings and it's this pythonic layer that this HOWTO deals with. + + +** Context + :PROPERTIES: + :CUSTOM_ID: howto-get-context + :END: + +One of the reasons which prevents this API from being RESTful is that +most operations require more than one instruction to the API to +perform the task. Sure, there are certain functions which can be +performed simultaneously, particularly if the result known or strongly +anticipated (e.g. selecting and encrypting to a key known to be in the +public keybox). + +There are many more, however, which cannot be manipulated so readily: +they must be performed in a specific sequence and the result of one +operation has a direct bearing on the outcome of subsequent +operations. Not merely by generating an error either. + +When dealing with this type of persistent state on the web, full of +both the RESTful and REST-like, it's most commonly referred to as a +session. In GPGME, however, it is called a context and every +operation type has one. + + +* Working with keys + :PROPERTIES: + :CUSTOM_ID: howto-keys + :END: + + +** Key selection + :PROPERTIES: + :CUSTOM_ID: howto-keys-selection + :END: + +Selecting keys to encrypt to or to sign with will be a common +occurrence when working with GPGMe and the means available for doing +so are quite simple. + +They do depend on utilising a Context; however once the data is +recorded in another variable, that Context does not need to be the +same one which subsequent operations are performed. + +The easiest way to select a specific key is by searching for that +key's key ID or fingerprint, preferably the full fingerprint without +any spaces in it. A long key ID will probably be okay, but is not +advised and short key IDs are already a problem with some being +generated to match specific patterns. It does not matter whether the +pattern is upper or lower case. + +So this is the best method: + +#+BEGIN_SRC python -i +import gpg + +k = gpg.Context().keylist(pattern="258E88DCBD3CD44D8E7AB43F6ECB6AF0DEADBEEF") +keys = list(k) +#+END_SRC + +This is passable and very likely to be common: + +#+BEGIN_SRC python -i +import gpg + +k = gpg.Context().keylist(pattern="0x6ECB6AF0DEADBEEF") +keys = list(k) +#+END_SRC + +And this is a really bad idea: + +#+BEGIN_SRC python -i +import gpg + +k = gpg.Context().keylist(pattern="0xDEADBEEF") +keys = list(k) +#+END_SRC + +Alternatively it may be that the intention is to create a list of keys +which all match a particular search string. For instance all the +addresses at a particular domain, like this: + +#+BEGIN_SRC python -i +import gpg + +ncsc = gpg.Context().keylist(pattern="ncsc.mil") +nsa = list(ncsc) +#+END_SRC + + +*** Counting keys + :PROPERTIES: + :CUSTOM_ID: howto-keys-counting + :END: + +Counting the number of keys in your public keybox (=pubring.kbx=), the +format which has superseded the old keyring format (=pubring.gpg= and +=secring.gpg=), or the number of secret keys is a very simple task. + +#+BEGIN_SRC python -i +import gpg + +c = gpg.Context() +seckeys = c.keylist(pattern=None, secret=True) +pubkeys = c.keylist(pattern=None, secret=False) + +seclist = list(seckeys) +secnum = len(seclist) + +publist = list(pubkeys) +pubnum = len(publist) + +print(""" + Number of secret keys: {0} + Number of public keys: {1} +""".format(secnum, pubnum)) +#+END_SRC + +NOTE: The [[#cython][Cython]] introduction in the [[#advanced-use][Advanced and Experimental]] +section uses this same key counting code with Cython to demonstrate +some areas where Cython can improve performance even with the +bindings. Users with large public keyrings or keyboxes, for instance, +should consider these options if they are comfortable with using +Cython. + + +** Get key + :PROPERTIES: + :CUSTOM_ID: howto-get-key + :END: + +An alternative method of getting a single key via its fingerprint is +available directly within a Context with =Context().get_key=. This is +the preferred method of selecting a key in order to modify it, sign or +certify it and for obtaining relevant data about a single key as a +part of other functions; when verifying a signature made by that key, +for instance. + +By default this method will select public keys, but it can select +secret keys as well. + +This first example demonstrates selecting the current key of Werner +Koch, which is due to expire at the end of 2018: + +#+BEGIN_SRC python -i +import gpg + +fingerprint = "80615870F5BAD690333686D0F2AD85AC1E42B367" +key = gpg.Context().get_key(fingerprint) +#+END_SRC + +Whereas this example demonstrates selecting the author's current key +with the =secret= key word argument set to =True=: + +#+BEGIN_SRC python -i +import gpg + +fingerprint = "DB4724E6FA4286C92B4E55C4321E4E2373590E5D" +key = gpg.Context().get_key(fingerprint, secret=True) +#+END_SRC + +It is, of course, quite possible to select expired, disabled and +revoked keys with this function, but only to effectively display +information about those keys. + +It is also possible to use both unicode or string literals and byte +literals with the fingerprint when getting a key in this way. + + +** Importing keys + :PROPERTIES: + :CUSTOM_ID: howto-import-key + :END: + +Importing keys is possible with the =key_import()= method and takes +one argument which is a bytes literal object containing either the +binary or ASCII armoured key data for one or more keys. + +The following example retrieves one or more keys from the SKS +keyservers via the web using the requests module. Since requests +returns the content as a bytes literal object, we can then use that +directly to import the resulting data into our keybox. + +#+BEGIN_SRC python -i +import gpg +import os.path +import requests + +c = gpg.Context() +url = "https://sks-keyservers.net/pks/lookup" +pattern = input("Enter the pattern to search for key or user IDs: ") +payload = {"op": "get", "search": pattern} + +r = requests.get(url, verify=True, params=payload) +result = c.key_import(r.content) + +if result is not None and hasattr(result, "considered") is False: + print(result) +elif result is not None and hasattr(result, "considered") is True: + num_keys = len(result.imports) + new_revs = result.new_revocations + new_sigs = result.new_signatures + new_subs = result.new_sub_keys + new_uids = result.new_user_ids + new_scrt = result.secret_imported + nochange = result.unchanged + print(""" + The total number of keys considered for import was: {0} + + Number of keys revoked: {1} + Number of new signatures: {2} + Number of new subkeys: {3} + Number of new user IDs: {4} + Number of new secret keys: {5} + Number of unchanged keys: {6} + + The key IDs for all considered keys were: +""".format(num_keys, new_revs, new_sigs, new_subs, new_uids, new_scrt, + nochange)) + for i in range(num_keys): + print("{0}\n".format(result.imports[i].fpr)) +else: + pass +#+END_SRC + +NOTE: When searching for a key ID of any length or a fingerprint +(without spaces), the SKS servers require the the leading =0x= +indicative of hexadecimal be included. Also note that the old short +key IDs (e.g. =0xDEADBEEF=) should no longer be used due to the +relative ease by which such key IDs can be reproduced, as demonstrated +by the Evil32 Project in 2014 (which was subsequently exploited in +2016). + +Testing for whether a string in any given search is or may be a +hexadecimal value which may be missing the leading =0x= is a simple +matter of using a try/except statement which attempts to convert the +string as hex to an integer and then back to hex; then using that to +search with. Raising a ValueError simply results in treating the +string as a string. This is the method and logic utilised in the +=import-keys-hkp.py= script (see below). + + +*** Working with ProtonMail + :PROPERTIES: + :CUSTOM_ID: import-protonmail + :END: + +Here is a variation on the example above which checks the constrained +ProtonMail keyserver for ProtonMail public keys. + +#+BEGIN_SRC python -i +import gpg +import requests +import sys + +print(""" +This script searches the ProtonMail key server for the specified key and +imports it. +""") + +c = gpg.Context(armor=True) +url = "https://api.protonmail.ch/pks/lookup" +ksearch = [] + +if len(sys.argv) >= 2: + keyterm = sys.argv[1] +else: + keyterm = input("Enter the key ID, UID or search string: ") + +if keyterm.count("@") == 2 and keyterm.startswith("@") is True: + ksearch.append(keyterm[1:]) + ksearch.append(keyterm[1:]) + ksearch.append(keyterm[1:]) +elif keyterm.count("@") == 1 and keyterm.startswith("@") is True: + ksearch.append("{0}@protonmail.com".format(keyterm[1:])) + ksearch.append("{0}@protonmail.ch".format(keyterm[1:])) + ksearch.append("{0}@pm.me".format(keyterm[1:])) +elif keyterm.count("@") == 0: + ksearch.append("{0}@protonmail.com".format(keyterm)) + ksearch.append("{0}@protonmail.ch".format(keyterm)) + ksearch.append("{0}@pm.me".format(keyterm)) +elif keyterm.count("@") == 2 and keyterm.startswith("@") is False: + uidlist = keyterm.split("@") + for uid in uidlist: + ksearch.append("{0}@protonmail.com".format(uid)) + ksearch.append("{0}@protonmail.ch".format(uid)) + ksearch.append("{0}@pm.me".format(uid)) +elif keyterm.count("@") > 2: + uidlist = keyterm.split("@") + for uid in uidlist: + ksearch.append("{0}@protonmail.com".format(uid)) + ksearch.append("{0}@protonmail.ch".format(uid)) + ksearch.append("{0}@pm.me".format(uid)) +else: + ksearch.append(keyterm) + +for k in ksearch: + payload = {"op": "get", "search": k} + try: + r = requests.get(url, verify=True, params=payload) + if r.ok is True: + result = c.key_import(r.content) + elif r.ok is False: + result = r.content + except Exception as e: + result = None + + if result is not None and hasattr(result, "considered") is False: + print("{0} for {1}".format(result.decode(), k)) + elif result is not None and hasattr(result, "considered") is True: + num_keys = len(result.imports) + new_revs = result.new_revocations + new_sigs = result.new_signatures + new_subs = result.new_sub_keys + new_uids = result.new_user_ids + new_scrt = result.secret_imported + nochange = result.unchanged + print(""" +The total number of keys considered for import was: {0} + +With UIDs wholely or partially matching the following string: + + {1} + + Number of keys revoked: {2} + Number of new signatures: {3} + Number of new subkeys: {4} + Number of new user IDs: {5} +Number of new secret keys: {6} + Number of unchanged keys: {7} + +The key IDs for all considered keys were: +""".format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt, + nochange)) + for i in range(num_keys): + print(result.imports[i].fpr) + print("") + elif result is None: + print(e) +#+END_SRC + +Both the above example, [[../examples/howto/pmkey-import.py][pmkey-import.py]], and a version which prompts +for an alternative GnuPG home directory, [[../examples/howto/pmkey-import-alt.py][pmkey-import-alt.py]], are +available with the other examples and are executable scripts. + +Note that while the ProtonMail servers are based on the SKS servers, +their server is related more to their API and is not feature complete +by comparison to the servers in the SKS pool. One notable difference +being that the ProtonMail server does not permit non ProtonMail users +to update their own keys, which could be a vector for attacking +ProtonMail users who may not receive a key's revocation if it had been +compromised. + + +*** Importing with HKP for Python + :PROPERTIES: + :CUSTOM_ID: import-hkp4py + :END: + +Performing the same tasks with the [[https://github.com/Selfnet/hkp4py][hkp4py module]] (available via PyPI) +is not too much different, but does provide a number of options of +benefit to end users. Not least of which being the ability to perform +some checks on a key before importing it or not. For instance it may +be the policy of a site or project to only import keys which have not +been revoked. The hkp4py module permits such checks prior to the +importing of the keys found. + +#+BEGIN_SRC python -i +import gpg +import hkp4py +import sys + +c = gpg.Context() +server = hkp4py.KeyServer("hkps://hkps.pool.sks-keyservers.net") +results = [] +keys = [] + +if len(sys.argv) > 2: + pattern = " ".join(sys.argv[1:]) +elif len(sys.argv) == 2: + pattern = sys.argv[1] +else: + pattern = input("Enter the pattern to search for keys or user IDs: ") + + +if pattern is not None: + try: + key = server.search(hex(int(pattern, 16))) + keyed = True + except ValueError as ve: + key = server.search(pattern) + keyed = False + + if key is not None: + keys.append(key[0]) + if keyed is True: + try: + fob = server.search(pattern) + except: + fob = None + if fob is not None: + keys.append(fob[0]) + else: + pass + else: + pass + + for logrus in pattern.split(): + try: + key = server.search(hex(int(logrus, 16))) + hexed = True + except ValueError as ve: + key = server.search(logrus) + hexed = False + + if key is not None: + keys.append(key[0]) + if hexed is True: + try: + fob = server.search(logrus) + except: + fob = None + if fob is not None: + keys.append(fob[0]) + else: + pass + else: + pass + + +if len(keys) > 0: + for key in keys: + import_result = c.key_import(key.key_blob) + results.append(import_result) + +for result in results: + if result is not None and hasattr(result, "considered") is False: + print(result) + elif result is not None and hasattr(result, "considered") is True: + num_keys = len(result.imports) + new_revs = result.new_revocations + new_sigs = result.new_signatures + new_subs = result.new_sub_keys + new_uids = result.new_user_ids + new_scrt = result.secret_imported + nochange = result.unchanged + print(""" +The total number of keys considered for import was: {0} + + Number of keys revoked: {1} + Number of new signatures: {2} + Number of new subkeys: {3} + Number of new user IDs: {4} +Number of new secret keys: {5} + Number of unchanged keys: {6} + +The key IDs for all considered keys were: +""".format(num_keys, new_revs, new_sigs, new_subs, new_uids, new_scrt, + nochange)) + for i in range(num_keys): + print(result.imports[i].fpr) + print("") + else: + pass +#+END_SRC + +Since the hkp4py module handles multiple keys just as effectively as +one (=keys= is a list of responses per matching key), the example +above is able to do a little bit more with the returned data before +anything is actually imported. + + +*** Importing from ProtonMail with HKP for Python + :PROPERTIES: + :CUSTOM_ID: import-protonmail-hkp4py + :END: + +Though this can provide certain benefits even when working with +ProtonMail, the scope is somewhat constrained there due to the +limitations of the ProtonMail keyserver. + +For instance, searching the SKS keyserver pool for the term "gnupg" +produces hundreds of results from any time the word appears in any +part of a user ID. Performing the same search on the ProtonMail +keyserver returns zero results, even though there are at least two +test accounts which include it as part of the username. + +The cause of this discrepancy is the deliberate configuration of that +server by ProtonMail to require an exact match of the full email +address of the ProtonMail user whose key is being requested. +Presumably this is intended to reduce breaches of privacy of their +users as an email address must already be known before a key for that +address can be obtained. + + +**** Import from ProtonMail via HKP for Python Example no. 1 + :PROPERTIES: + :CUSTOM_ID: import-hkp4py-pm1 + :END: + +The following script is available with the rest of the examples under +the somewhat less than original name, =pmkey-import-hkp.py=. + +#+BEGIN_SRC python -i +import gpg +import hkp4py +import os.path +import sys + +print(""" +This script searches the ProtonMail key server for the specified key and +imports it. + +Usage: pmkey-import-hkp.py [search strings] +""") + +c = gpg.Context(armor=True) +server = hkp4py.KeyServer("hkps://api.protonmail.ch") +keyterms = [] +ksearch = [] +allkeys = [] +results = [] +paradox = [] +homeless = None + +if len(sys.argv) > 2: + keyterms = sys.argv[1:] +elif len(sys.argv) == 2: + keyterm = sys.argv[1] + keyterms.append(keyterm) +else: + key_term = input("Enter the key ID, UID or search string: ") + keyterms = key_term.split() + +for keyterm in keyterms: + if keyterm.count("@") == 2 and keyterm.startswith("@") is True: + ksearch.append(keyterm[1:]) + ksearch.append(keyterm[1:]) + ksearch.append(keyterm[1:]) + elif keyterm.count("@") == 1 and keyterm.startswith("@") is True: + ksearch.append("{0}@protonmail.com".format(keyterm[1:])) + ksearch.append("{0}@protonmail.ch".format(keyterm[1:])) + ksearch.append("{0}@pm.me".format(keyterm[1:])) + elif keyterm.count("@") == 0: + ksearch.append("{0}@protonmail.com".format(keyterm)) + ksearch.append("{0}@protonmail.ch".format(keyterm)) + ksearch.append("{0}@pm.me".format(keyterm)) + elif keyterm.count("@") == 2 and keyterm.startswith("@") is False: + uidlist = keyterm.split("@") + for uid in uidlist: + ksearch.append("{0}@protonmail.com".format(uid)) + ksearch.append("{0}@protonmail.ch".format(uid)) + ksearch.append("{0}@pm.me".format(uid)) + elif keyterm.count("@") > 2: + uidlist = keyterm.split("@") + for uid in uidlist: + ksearch.append("{0}@protonmail.com".format(uid)) + ksearch.append("{0}@protonmail.ch".format(uid)) + ksearch.append("{0}@pm.me".format(uid)) + else: + ksearch.append(keyterm) + +for k in ksearch: + print("Checking for key for: {0}".format(k)) + try: + keys = server.search(k) + if isinstance(keys, list) is True: + for key in keys: + allkeys.append(key) + try: + import_result = c.key_import(key.key_blob) + except Exception as e: + import_result = c.key_import(key.key) + else: + paradox.append(keys) + import_result = None + except Exception as e: + import_result = None + results.append(import_result) + +for result in results: + if result is not None and hasattr(result, "considered") is False: + print("{0} for {1}".format(result.decode(), k)) + elif result is not None and hasattr(result, "considered") is True: + num_keys = len(result.imports) + new_revs = result.new_revocations + new_sigs = result.new_signatures + new_subs = result.new_sub_keys + new_uids = result.new_user_ids + new_scrt = result.secret_imported + nochange = result.unchanged + print(""" +The total number of keys considered for import was: {0} + +With UIDs wholely or partially matching the following string: + + {1} + + Number of keys revoked: {2} + Number of new signatures: {3} + Number of new subkeys: {4} + Number of new user IDs: {5} +Number of new secret keys: {6} + Number of unchanged keys: {7} + +The key IDs for all considered keys were: +""".format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt, + nochange)) + for i in range(num_keys): + print(result.imports[i].fpr) + print("") + elif result is None: + pass +#+END_SRC + + +**** Import from ProtonMail via HKP for Python Example no. 2 + :PROPERTIES: + :CUSTOM_ID: import-hkp4py-pm2 + :END: + +Like its counterpart above, this script can also be found with the +rest of the examples, by the name pmkey-import-hkp-alt.py. + +With this script a modicum of effort has been made to treat anything +passed as a =homedir= which either does not exist or which is not a +directory, as also being a pssible user ID to check for. It's not +guaranteed to pick up on all such cases, but it should cover most of +them. + +#+BEGIN_SRC python -i +import gpg +import hkp4py +import os.path +import sys + +print(""" +This script searches the ProtonMail key server for the specified key and +imports it. Optionally enables specifying a different GnuPG home directory. + +Usage: pmkey-import-hkp.py [homedir] [search string] + or: pmkey-import-hkp.py [search string] +""") + +c = gpg.Context(armor=True) +server = hkp4py.KeyServer("hkps://api.protonmail.ch") +keyterms = [] +ksearch = [] +allkeys = [] +results = [] +paradox = [] +homeless = None + +if len(sys.argv) > 3: + homedir = sys.argv[1] + keyterms = sys.argv[2:] +elif len(sys.argv) == 3: + homedir = sys.argv[1] + keyterm = sys.argv[2] + keyterms.append(keyterm) +elif len(sys.argv) == 2: + homedir = "" + keyterm = sys.argv[1] + keyterms.append(keyterm) +else: + keyterm = input("Enter the key ID, UID or search string: ") + homedir = input("Enter the GPG configuration directory path (optional): ") + keyterms.append(keyterm) + +if len(homedir) == 0: + homedir = None + homeless = False + +if homedir is not None: + if homedir.startswith("~"): + if os.path.exists(os.path.expanduser(homedir)) is True: + if os.path.isdir(os.path.expanduser(homedir)) is True: + c.home_dir = os.path.realpath(os.path.expanduser(homedir)) + else: + homeless = True + else: + homeless = True + elif os.path.exists(os.path.realpath(homedir)) is True: + if os.path.isdir(os.path.realpath(homedir)) is True: + c.home_dir = os.path.realpath(homedir) + else: + homeless = True + else: + homeless = True + +# First check to see if the homedir really is a homedir and if not, treat it as +# a search string. +if homeless is True: + keyterms.append(homedir) + c.home_dir = None +else: + pass + +for keyterm in keyterms: + if keyterm.count("@") == 2 and keyterm.startswith("@") is True: + ksearch.append(keyterm[1:]) + ksearch.append(keyterm[1:]) + ksearch.append(keyterm[1:]) + elif keyterm.count("@") == 1 and keyterm.startswith("@") is True: + ksearch.append("{0}@protonmail.com".format(keyterm[1:])) + ksearch.append("{0}@protonmail.ch".format(keyterm[1:])) + ksearch.append("{0}@pm.me".format(keyterm[1:])) + elif keyterm.count("@") == 0: + ksearch.append("{0}@protonmail.com".format(keyterm)) + ksearch.append("{0}@protonmail.ch".format(keyterm)) + ksearch.append("{0}@pm.me".format(keyterm)) + elif keyterm.count("@") == 2 and keyterm.startswith("@") is False: + uidlist = keyterm.split("@") + for uid in uidlist: + ksearch.append("{0}@protonmail.com".format(uid)) + ksearch.append("{0}@protonmail.ch".format(uid)) + ksearch.append("{0}@pm.me".format(uid)) + elif keyterm.count("@") > 2: + uidlist = keyterm.split("@") + for uid in uidlist: + ksearch.append("{0}@protonmail.com".format(uid)) + ksearch.append("{0}@protonmail.ch".format(uid)) + ksearch.append("{0}@pm.me".format(uid)) + else: + ksearch.append(keyterm) + +for k in ksearch: + print("Checking for key for: {0}".format(k)) + try: + keys = server.search(k) + if isinstance(keys, list) is True: + for key in keys: + allkeys.append(key) + try: + import_result = c.key_import(key.key_blob) + except Exception as e: + import_result = c.key_import(key.key) + else: + paradox.append(keys) + import_result = None + except Exception as e: + import_result = None + results.append(import_result) + +for result in results: + if result is not None and hasattr(result, "considered") is False: + print("{0} for {1}".format(result.decode(), k)) + elif result is not None and hasattr(result, "considered") is True: + num_keys = len(result.imports) + new_revs = result.new_revocations + new_sigs = result.new_signatures + new_subs = result.new_sub_keys + new_uids = result.new_user_ids + new_scrt = result.secret_imported + nochange = result.unchanged + print(""" +The total number of keys considered for import was: {0} + +With UIDs wholely or partially matching the following string: + + {1} + + Number of keys revoked: {2} + Number of new signatures: {3} + Number of new subkeys: {4} + Number of new user IDs: {5} +Number of new secret keys: {6} + Number of unchanged keys: {7} + +The key IDs for all considered keys were: +""".format(num_keys, k, new_revs, new_sigs, new_subs, new_uids, new_scrt, + nochange)) + for i in range(num_keys): + print(result.imports[i].fpr) + print("") + elif result is None: + pass +#+END_SRC + + +** Exporting keys + :PROPERTIES: + :CUSTOM_ID: howto-export-key + :END: + +Exporting keys remains a reasonably simple task, but has been +separated into three different functions for the OpenPGP cryptographic +engine. Two of those functions are for exporting public keys and the +third is for exporting secret keys. + + +*** Exporting public keys + :PROPERTIES: + :CUSTOM_ID: howto-export-public-key + :END: + +There are two methods of exporting public keys, both of which are very +similar to the other. The default method, =key_export()=, will export +a public key or keys matching a specified pattern as normal. The +alternative, the =key_export_minimal()= method, will do the same thing +except producing a minimised output with extra signatures and third +party signatures or certifications removed. + +#+BEGIN_SRC python -i +import gpg +import os.path +import sys + +print(""" +This script exports one or more public keys. +""") + +c = gpg.Context(armor=True) + +if len(sys.argv) >= 4: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = sys.argv[3] +elif len(sys.argv) == 3: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = input("Enter the GPG configuration directory path (optional): ") +elif len(sys.argv) == 2: + keyfile = sys.argv[1] + logrus = input("Enter the UID matching the key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") +else: + keyfile = input("Enter the path and filename to save the secret key to: ") + logrus = input("Enter the UID matching the key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") + +if homedir.startswith("~"): + if os.path.exists(os.path.expanduser(homedir)) is True: + c.home_dir = os.path.expanduser(homedir) + else: + pass +elif os.path.exists(homedir) is True: + c.home_dir = homedir +else: + pass + +try: + result = c.key_export(pattern=logrus) +except: + result = c.key_export(pattern=None) + +if result is not None: + with open(keyfile, "wb") as f: + f.write(result) +else: + pass +#+END_SRC + +It should be noted that the result will only return =None= when a +search pattern has been entered, but has not matched any keys. When +the search pattern itself is set to =None= this triggers the exporting +of the entire public keybox. + +#+BEGIN_SRC python -i +import gpg +import os.path +import sys + +print(""" +This script exports one or more public keys in minimised form. +""") + +c = gpg.Context(armor=True) + +if len(sys.argv) >= 4: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = sys.argv[3] +elif len(sys.argv) == 3: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = input("Enter the GPG configuration directory path (optional): ") +elif len(sys.argv) == 2: + keyfile = sys.argv[1] + logrus = input("Enter the UID matching the key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") +else: + keyfile = input("Enter the path and filename to save the secret key to: ") + logrus = input("Enter the UID matching the key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") + +if homedir.startswith("~"): + if os.path.exists(os.path.expanduser(homedir)) is True: + c.home_dir = os.path.expanduser(homedir) + else: + pass +elif os.path.exists(homedir) is True: + c.home_dir = homedir +else: + pass + +try: + result = c.key_export_minimal(pattern=logrus) +except: + result = c.key_export_minimal(pattern=None) + +if result is not None: + with open(keyfile, "wb") as f: + f.write(result) +else: + pass +#+END_SRC + + +*** Exporting secret keys + :PROPERTIES: + :CUSTOM_ID: howto-export-secret-key + :END: + +Exporting secret keys is, functionally, very similar to exporting +public keys; save for the invocation of =pinentry= via =gpg-agent= in +order to securely enter the key's passphrase and authorise the export. + +The following example exports the secret key to a file which is then +set with the same permissions as the output files created by the +command line secret key export options. + +#+BEGIN_SRC python -i +import gpg +import os +import os.path +import sys + +print(""" +This script exports one or more secret keys. + +The gpg-agent and pinentry are invoked to authorise the export. +""") + +c = gpg.Context(armor=True) + +if len(sys.argv) >= 4: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = sys.argv[3] +elif len(sys.argv) == 3: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = input("Enter the GPG configuration directory path (optional): ") +elif len(sys.argv) == 2: + keyfile = sys.argv[1] + logrus = input("Enter the UID matching the secret key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") +else: + keyfile = input("Enter the path and filename to save the secret key to: ") + logrus = input("Enter the UID matching the secret key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") + +if len(homedir) == 0: + homedir = None +elif homedir.startswith("~"): + userdir = os.path.expanduser(homedir) + if os.path.exists(userdir) is True: + homedir = os.path.realpath(userdir) + else: + homedir = None +else: + homedir = os.path.realpath(homedir) + +if os.path.exists(homedir) is False: + homedir = None +else: + if os.path.isdir(homedir) is False: + homedir = None + else: + pass + +if homedir is not None: + c.home_dir = homedir +else: + pass + +try: + result = c.key_export_secret(pattern=logrus) +except: + result = c.key_export_secret(pattern=None) + +if result is not None: + with open(keyfile, "wb") as f: + f.write(result) + os.chmod(keyfile, 0o600) +else: + pass +#+END_SRC + +Alternatively the approach of the following script can be used. This +longer example saves the exported secret key(s) in files in the GnuPG +home directory, in addition to setting the file permissions as only +readable and writable by the user. It also exports the secret key(s) +twice in order to output both GPG binary (=.gpg=) and ASCII armoured +(=.asc=) files. + +#+BEGIN_SRC python -i +import gpg +import os +import os.path +import subprocess +import sys + +print(""" +This script exports one or more secret keys as both ASCII armored and binary +file formats, saved in files within the user's GPG home directory. + +The gpg-agent and pinentry are invoked to authorise the export. +""") + +if sys.platform == "win32": + gpgconfcmd = "gpgconf.exe --list-dirs homedir" +else: + gpgconfcmd = "gpgconf --list-dirs homedir" + +a = gpg.Context(armor=True) +b = gpg.Context() +c = gpg.Context() + +if len(sys.argv) >= 4: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = sys.argv[3] +elif len(sys.argv) == 3: + keyfile = sys.argv[1] + logrus = sys.argv[2] + homedir = input("Enter the GPG configuration directory path (optional): ") +elif len(sys.argv) == 2: + keyfile = sys.argv[1] + logrus = input("Enter the UID matching the secret key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") +else: + keyfile = input("Enter the filename to save the secret key to: ") + logrus = input("Enter the UID matching the secret key(s) to export: ") + homedir = input("Enter the GPG configuration directory path (optional): ") + +if len(homedir) == 0: + homedir = None +elif homedir.startswith("~"): + userdir = os.path.expanduser(homedir) + if os.path.exists(userdir) is True: + homedir = os.path.realpath(userdir) + else: + homedir = None +else: + homedir = os.path.realpath(homedir) + +if os.path.exists(homedir) is False: + homedir = None +else: + if os.path.isdir(homedir) is False: + homedir = None + else: + pass + +if homedir is not None: + c.home_dir = homedir +else: + pass + +if c.home_dir is not None: + if c.home_dir.endswith("/"): + gpgfile = "{0}{1}.gpg".format(c.home_dir, keyfile) + ascfile = "{0}{1}.asc".format(c.home_dir, keyfile) + else: + gpgfile = "{0}/{1}.gpg".format(c.home_dir, keyfile) + ascfile = "{0}/{1}.asc".format(c.home_dir, keyfile) +else: + if os.path.exists(os.environ["GNUPGHOME"]) is True: + hd = os.environ["GNUPGHOME"] + else: + try: + hd = subprocess.getoutput(gpgconfcmd) + except: + process = subprocess.Popen(gpgconfcmd.split(), + stdout=subprocess.PIPE) + procom = process.communicate() + if sys.version_info[0] == 2: + hd = procom[0].strip() + else: + hd = procom[0].decode().strip() + gpgfile = "{0}/{1}.gpg".format(hd, keyfile) + ascfile = "{0}/{1}.asc".format(hd, keyfile) + +try: + a_result = a.key_export_secret(pattern=logrus) + b_result = b.key_export_secret(pattern=logrus) +except: + a_result = a.key_export_secret(pattern=None) + b_result = b.key_export_secret(pattern=None) + +if a_result is not None: + with open(ascfile, "wb") as f: + f.write(a_result) + os.chmod(ascfile, 0o600) +else: + pass + +if b_result is not None: + with open(gpgfile, "wb") as f: + f.write(b_result) + os.chmod(gpgfile, 0o600) +else: + pass +#+END_SRC + + +*** Sending public keys to the SKS Keyservers + :PROPERTIES: + :CUSTOM_ID: howto-send-public-key + :END: + +As with the previous section on importing keys, the =hkp4py= module +adds another option with exporting keys in order to send them to the +public keyservers. + +The following example demonstrates how this may be done. + +#+BEGIN_SRC python -i +import gpg +import hkp4py +import os.path +import sys + +print(""" +This script sends one or more public keys to the SKS keyservers and is +essentially a slight variation on the export-key.py script. +""") + +c = gpg.Context(armor=True) +server = hkp4py.KeyServer("hkps://hkps.pool.sks-keyservers.net") + +if len(sys.argv) > 2: + logrus = " ".join(sys.argv[1:]) +elif len(sys.argv) == 2: + logrus = sys.argv[1] +else: + logrus = input("Enter the UID matching the key(s) to send: ") + +if len(logrus) > 0: + try: + export_result = c.key_export(pattern=logrus) + except Exception as e: + print(e) + export_result = None +else: + export_result = c.key_export(pattern=None) + +if export_result is not None: + try: + try: + send_result = server.add(export_result) + except: + send_result = server.add(export_result.decode()) + if send_result is not None: + print(send_result) + else: + pass + except Exception as e: + print(e) +else: + pass +#+END_SRC + +An expanded version of this script with additional functions for +specifying an alternative homedir location is in the examples +directory as =send-key-to-keyserver.py=. + +The =hkp4py= module appears to handle both string and byte literal text +data equally well, but the GPGME bindings deal primarily with byte +literal data only and so this script sends in that format first, then +tries the string literal form. + + +* Basic Functions + :PROPERTIES: + :CUSTOM_ID: howto-the-basics + :END: + +The most frequently called features of any cryptographic library will +be the most fundamental tasks for encryption software. In this +section we will look at how to programmatically encrypt data, decrypt +it, sign it and verify signatures. + + +** Encryption + :PROPERTIES: + :CUSTOM_ID: howto-basic-encryption + :END: + +Encrypting is very straight forward. In the first example below the +message, =text=, is encrypted to a single recipient's key. In the +second example the message will be encrypted to multiple recipients. + + +*** Encrypting to one key + :PROPERTIES: + :CUSTOM_ID: howto-basic-encryption-single + :END: + +Once the the Context is set the main issues with encrypting data is +essentially reduced to key selection and the keyword arguments +specified in the =gpg.Context().encrypt()= method. + +Those keyword arguments are: =recipients=, a list of keys encrypted to +(covered in greater detail in the following section); =sign=, whether +or not to sign the plaintext data, see subsequent sections on signing +and verifying signatures below (defaults to =True=); =sink=, to write +results or partial results to a secure sink instead of returning it +(defaults to =None=); =passphrase=, only used when utilising symmetric +encryption (defaults to =None=); =always_trust=, used to override the +trust model settings for recipient keys (defaults to =False=); +=add_encrypt_to=, utilises any preconfigured =encrypt-to= or +=default-key= settings in the user's =gpg.conf= file (defaults to +=False=); =prepare=, prepare for encryption (defaults to =False=); +=expect_sign=, prepare for signing (defaults to =False=); =compress=, +compresses the plaintext prior to encryption (defaults to =True=). + +#+BEGIN_SRC python -i +import gpg + +a_key = "0x12345678DEADBEEF" +text = b"""Some text to test with. + +Since the text in this case must be bytes, it is most likely that +the input form will be a separate file which is opened with "rb" +as this is the simplest method of obtaining the correct data format. +""" + +c = gpg.Context(armor=True) +rkey = list(c.keylist(pattern=a_key, secret=False)) +ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, sign=False) + +with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) +#+END_SRC + +Though this is even more likely to be used like this; with the +plaintext input read from a file, the recipient keys used for +encryption regardless of key trust status and the encrypted output +also encrypted to any preconfigured keys set in the =gpg.conf= file: + +#+BEGIN_SRC python -i +import gpg + +a_key = "0x12345678DEADBEEF" + +with open("secret_plans.txt", "rb") as afile: + text = afile.read() + +c = gpg.Context(armor=True) +rkey = list(c.keylist(pattern=a_key, secret=False)) +ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, sign=True, + always_trust=True, + add_encrypt_to=True) + +with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) +#+END_SRC + +If the =recipients= parameter is empty then the plaintext is encrypted +symmetrically. If no =passphrase= is supplied as a parameter or via a +callback registered with the =Context()= then an out-of-band prompt +for the passphrase via pinentry will be invoked. + + +*** Encrypting to multiple keys + :PROPERTIES: + :CUSTOM_ID: howto-basic-encryption-multiple + :END: + +Encrypting to multiple keys essentially just expands upon the key +selection process and the recipients from the previous examples. + +The following example encrypts a message (=text=) to everyone with an +email address on the =gnupg.org= domain,[fn:4] but does /not/ encrypt +to a default key or other key which is configured to normally encrypt +to. + +#+BEGIN_SRC python -i +import gpg + +text = b"""Oh look, another test message. + +The same rules apply as with the previous example and more likely +than not, the message will actually be drawn from reading the +contents of a file or, maybe, from entering data at an input() +prompt. + +Since the text in this case must be bytes, it is most likely that +the input form will be a separate file which is opened with "rb" +as this is the simplest method of obtaining the correct data +format. +""" + +c = gpg.Context(armor=True) +rpattern = list(c.keylist(pattern="@gnupg.org", secret=False)) +logrus = [] + +for i in range(len(rpattern)): + if rpattern[i].can_encrypt == 1: + logrus.append(rpattern[i]) + +ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, + sign=False, always_trust=True) + +with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) +#+END_SRC + +All it would take to change the above example to sign the message +and also encrypt the message to any configured default keys would +be to change the =c.encrypt= line to this: + +#+BEGIN_SRC python -i +ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, + always_trust=True, + add_encrypt_to=True) +#+END_SRC + +The only keyword arguments requiring modification are those for which +the default values are changing. The default value of =sign= is +=True=, the default of =always_trust= is =False=, the default of +=add_encrypt_to= is =False=. + +If =always_trust= is not set to =True= and any of the recipient keys +are not trusted (e.g. not signed or locally signed) then the +encryption will raise an error. It is possible to mitigate this +somewhat with something more like this: + +#+BEGIN_SRC python -i +import gpg + +with open("secret_plans.txt.asc", "rb") as afile: + text = afile.read() + +c = gpg.Context(armor=True) +rpattern = list(c.keylist(pattern="@gnupg.org", secret=False)) +logrus = [] + +for i in range(len(rpattern)): + if rpattern[i].can_encrypt == 1: + logrus.append(rpattern[i]) + + try: + ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, + add_encrypt_to=True) + except gpg.errors.InvalidRecipients as e: + for i in range(len(e.recipients)): + for n in range(len(logrus)): + if logrus[n].fpr == e.recipients[i].fpr: + logrus.remove(logrus[n]) + else: + pass + try: + ciphertext, result, sign_result = c.encrypt(text, + recipients=logrus, + add_encrypt_to=True) + with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) + except: + pass +#+END_SRC + +This will attempt to encrypt to all the keys searched for, then remove +invalid recipients if it fails and try again. + + +** Decryption + :PROPERTIES: + :CUSTOM_ID: howto-basic-decryption + :END: + +Decrypting something encrypted to a key in one's secret keyring is +fairly straight forward. + +In this example code, however, preconfiguring either =gpg.Context()= +or =gpg.core.Context()= as =c= is unnecessary because there is no need +to modify the Context prior to conducting the decryption and since the +Context is only used once, setting it to =c= simply adds lines for no +gain. + +#+BEGIN_SRC python -i +import gpg + +ciphertext = input("Enter path and filename of encrypted file: ") +newfile = input("Enter path and filename of file to save decrypted data to: ") + +with open(ciphertext, "rb") as cfile: + try: + plaintext, result, verify_result = gpg.Context().decrypt(cfile) + except gpg.errors.GPGMEError as e: + plaintext = None + print(e) + +if plaintext is not None: + with open(newfile, "wb") as nfile: + nfile.write(plaintext) + else: + pass +#+END_SRC + +The data available in =plaintext= in this example is the decrypted +content as a byte object, the recipient key IDs and algorithms in +=result= and the results of verifying any signatures of the data in +=verify_result=. + +If =gpg.Context().decrypt(cfile, verify=False)= is called instead, +then =verify_result= will be returned as =None= and the rest remains +as described here. + + +** Signing text and files + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing + :END: + +The following sections demonstrate how to specify keys to sign with. + + +*** Signing key selection + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-signers + :END: + +By default GPGME and the Python bindings will use the default key +configured for the user invoking the GPGME API. If there is no +default key specified and there is more than one secret key available +it may be necessary to specify the key or keys with which to sign +messages and files. + +#+BEGIN_SRC python -i +import gpg + +logrus = input("Enter the email address or string to match signing keys to: ") +hancock = gpg.Context().keylist(pattern=logrus, secret=True) +sig_src = list(hancock) +#+END_SRC + +The signing examples in the following sections include the explicitly +designated =signers= parameter in two of the five examples; once where +the resulting signature would be ASCII armoured and once where it +would not be armoured. + +While it would be possible to enter a key ID or fingerprint here to +match a specific key, it is not possible to enter two fingerprints and +match two keys since the patten expects a string, bytes or None and +not a list. A string with two fingerprints won't match any single +key. + + +*** Normal or default signing messages or files + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-normal + :END: + +The normal or default signing process is essentially the same as is +most often invoked when also encrypting a message or file. So when +the encryption component is not utilised, the result is to produce an +encoded and signed output which may or may not be ASCII armoured and +which may or may not also be compressed. + +By default compression will be used unless GnuPG detects that the +plaintext is already compressed. ASCII armouring will be determined +according to the value of =gpg.Context().armor=. + +The compression algorithm is selected in much the same way as the +symmetric encryption algorithm or the hash digest algorithm is when +multiple keys are involved; from the preferences saved into the key +itself or by comparison with the preferences with all other keys +involved. + +#+BEGIN_SRC python -i +import gpg + +text0 = """Declaration of ... something. + +""" +text = text0.encode() + +c = gpg.Context(armor=True, signers=sig_src) +signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL) + +with open("/path/to/statement.txt.asc", "w") as afile: + afile.write(signed_data.decode()) +#+END_SRC + +Though everything in this example is accurate, it is more likely that +reading the input data from another file and writing the result to a +new file will be performed more like the way it is done in the next +example. Even if the output format is ASCII armoured. + +#+BEGIN_SRC python -i +import gpg + +with open("/path/to/statement.txt", "rb") as tfile: + text = tfile.read() + +c = gpg.Context() +signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL) + +with open("/path/to/statement.txt.sig", "wb") as afile: + afile.write(signed_data) +#+END_SRC + + +*** Detached signing messages and files + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-detached + :END: + +Detached signatures will often be needed in programmatic uses of +GPGME, either for signing files (e.g. tarballs of code releases) or as +a component of message signing (e.g. PGP/MIME encoded email). + +#+BEGIN_SRC python -i +import gpg + +text0 = """Declaration of ... something. + +""" +text = text0.encode() + +c = gpg.Context(armor=True) +signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH) + +with open("/path/to/statement.txt.asc", "w") as afile: + afile.write(signed_data.decode()) +#+END_SRC + +As with normal signatures, detached signatures are best handled as +byte literals, even when the output is ASCII armoured. + +#+BEGIN_SRC python -i +import gpg + +with open("/path/to/statement.txt", "rb") as tfile: + text = tfile.read() + +c = gpg.Context(signers=sig_src) +signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH) + +with open("/path/to/statement.txt.sig", "wb") as afile: + afile.write(signed_data) +#+END_SRC + + +*** Clearsigning messages or text + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-clear + :END: + +Though PGP/in-line messages are no longer encouraged in favour of +PGP/MIME, there is still sometimes value in utilising in-line +signatures. This is where clear-signed messages or text is of value. + +#+BEGIN_SRC python -i +import gpg + +text0 = """Declaration of ... something. + +""" +text = text0.encode() + +c = gpg.Context() +signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR) + +with open("/path/to/statement.txt.asc", "w") as afile: + afile.write(signed_data.decode()) +#+END_SRC + +In spite of the appearance of a clear-signed message, the data handled +by GPGME in signing it must still be byte literals. + +#+BEGIN_SRC python -i +import gpg + +with open("/path/to/statement.txt", "rb") as tfile: + text = tfile.read() + +c = gpg.Context() +signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR) + +with open("/path/to/statement.txt.asc", "wb") as afile: + afile.write(signed_data) +#+END_SRC + + +** Signature verification + :PROPERTIES: + :CUSTOM_ID: howto-basic-verification + :END: + +Essentially there are two principal methods of verification of a +signature. The first of these is for use with the normal or default +signing method and for clear-signed messages. The second is for use +with files and data with detached signatures. + +The following example is intended for use with the default signing +method where the file was not ASCII armoured: + +#+BEGIN_SRC python -i +import gpg +import time + +filename = "statement.txt" +gpg_file = "statement.txt.gpg" + +c = gpg.Context() + +try: + data, result = c.verify(open(gpg_file)) + verified = True +except gpg.errors.BadSignatures as e: + verified = False + print(e) + +if verified is True: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: +{0} +with key {1} +made at {2} +""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, + time.ctime(sign.timestamp))) +else: + pass +#+END_SRC + +Whereas this next example, which is almost identical would work with +normal ASCII armoured files and with clear-signed files: + +#+BEGIN_SRC python -i +import gpg +import time + +filename = "statement.txt" +asc_file = "statement.txt.asc" + +c = gpg.Context() + +try: + data, result = c.verify(open(asc_file)) + verified = True +except gpg.errors.BadSignatures as e: + verified = False + print(e) + +if verified is True: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: +{0} +with key {1} +made at {2} +""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, + time.ctime(sign.timestamp))) +else: + pass +#+END_SRC + +In both of the previous examples it is also possible to compare the +original data that was signed against the signed data in =data= to see +if it matches with something like this: + +#+BEGIN_SRC python -i +with open(filename, "rb") as afile: + text = afile.read() + +if text == data: + print("Good signature.") +else: + pass +#+END_SRC + +The following two examples, however, deal with detached signatures. +With his method of verification the data that was signed does not get +returned since it is already being explicitly referenced in the first +argument of =c.verify=. So =data= is =None= and only the information +in =result= is available. + +#+BEGIN_SRC python -i +import gpg +import time + +filename = "statement.txt" +sig_file = "statement.txt.sig" + +c = gpg.Context() + +try: + data, result = c.verify(open(filename), open(sig_file)) + verified = True +except gpg.errors.BadSignatures as e: + verified = False + print(e) + +if verified is True: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: +{0} +with key {1} +made at {2} +""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, + time.ctime(sign.timestamp))) +else: + pass +#+END_SRC + +#+BEGIN_SRC python -i +import gpg +import time + +filename = "statement.txt" +asc_file = "statement.txt.asc" + +c = gpg.Context() + +try: + data, result = c.verify(open(filename), open(asc_file)) + verified = True +except gpg.errors.BadSignatures as e: + verified = False + print(e) + +if verified is True: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: +{0} +with key {1} +made at {2} +""".format(c.get_key(sign.fpr).uids[0].uid, sign.fpr, + time.ctime(sign.timestamp))) +else: + pass +#+END_SRC + + +* Creating keys and subkeys + :PROPERTIES: + :CUSTOM_ID: key-generation + :END: + +The one thing, aside from GnuPG itself, that GPGME depends on, of +course, is the keys themselves. So it is necessary to be able to +generate them and modify them by adding subkeys, revoking or disabling +them, sometimes deleting them and doing the same for user IDs. + +In the following examples a key will be created for the world's +greatest secret agent, Danger Mouse. Since Danger Mouse is a secret +agent he needs to be able to protect information to =SECRET= level +clearance, so his keys will be 3072-bit keys. + +The pre-configured =gpg.conf= file which sets cipher, digest and other +preferences contains the following configuration parameters: + +#+BEGIN_SRC conf + expert + allow-freeform-uid + allow-secret-key-import + trust-model tofu+pgp + tofu-default-policy unknown + enable-large-rsa + enable-dsa2 + cert-digest-algo SHA512 + default-preference-list TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1 ZLIB BZIP2 ZIP Uncompressed + personal-cipher-preferences TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES + personal-digest-preferences SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1 + personal-compress-preferences ZLIB BZIP2 ZIP Uncompressed +#+END_SRC + + +** Primary key + :PROPERTIES: + :CUSTOM_ID: keygen-primary + :END: + +Generating a primary key uses the =create_key= method in a Context. +It contains multiple arguments and keyword arguments, including: +=userid=, =algorithm=, =expires_in=, =expires=, =sign=, =encrypt=, +=certify=, =authenticate=, =passphrase= and =force=. The defaults for +all of those except =userid=, =algorithm=, =expires_in=, =expires= and +=passphrase= is =False=. The defaults for =algorithm= and +=passphrase= is =None=. The default for =expires_in= is =0=. The +default for =expires= is =True=. There is no default for =userid=. + +If =passphrase= is left as =None= then the key will not be generated +with a passphrase, if =passphrase= is set to a string then that will +be the passphrase and if =passphrase= is set to =True= then gpg-agent +will launch pinentry to prompt for a passphrase. For the sake of +convenience, these examples will keep =passphrase= set to =None=. + +#+BEGIN_SRC python -i +import gpg + +c = gpg.Context() + +c.home_dir = "~/.gnupg-dm" +userid = "Danger Mouse " + +dmkey = c.create_key(userid, algorithm="rsa3072", expires_in=31536000, + sign=True, certify=True) +#+END_SRC + +One thing to note here is the use of setting the =c.home_dir= +parameter. This enables generating the key or keys in a different +location. In this case to keep the new key data created for this +example in a separate location rather than adding it to existing and +active key store data. As with the default directory, =~/.gnupg=, any +temporary or separate directory needs the permissions set to only +permit access by the directory owner. On posix systems this means +setting the directory permissions to 700. + +The =temp-homedir-config.py= script in the HOWTO examples directory +will create an alternative homedir with these configuration options +already set and the correct directory and file permissions. + +The successful generation of the key can be confirmed via the returned +=GenkeyResult= object, which includes the following data: + +#+BEGIN_SRC python -i +print(""" + Fingerprint: {0} + Primary Key: {1} + Public Key: {2} + Secret Key: {3} + Sub Key: {4} +User IDs: {5} +""".format(dmkey.fpr, dmkey.primary, dmkey.pubkey, dmkey.seckey, dmkey.sub, + dmkey.uid)) +#+END_SRC + +Alternatively the information can be confirmed using the command line +program: + +#+BEGIN_SRC shell + bash-4.4$ gpg --homedir ~/.gnupg-dm -K + ~/.gnupg-dm/pubring.kbx + ---------------------- + sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] + 177B7C25DB99745EE2EE13ED026D2F19E99E63AA + uid [ultimate] Danger Mouse + + bash-4.4$ +#+END_SRC + +As with generating keys manually, to preconfigure expanded preferences +for the cipher, digest and compression algorithms, the =gpg.conf= file +must contain those details in the home directory in which the new key +is being generated. I used a cut down version of my own =gpg.conf= +file in order to be able to generate this: + +#+BEGIN_SRC shell + bash-4.4$ gpg --homedir ~/.gnupg-dm --edit-key 177B7C25DB99745EE2EE13ED026D2F19E99E63AA showpref quit + Secret key is available. + + sec rsa3072/026D2F19E99E63AA + created: 2018-03-15 expires: 2019-03-15 usage: SC + trust: ultimate validity: ultimate + [ultimate] (1). Danger Mouse + + [ultimate] (1). Danger Mouse + Cipher: TWOFISH, CAMELLIA256, AES256, CAMELLIA192, AES192, CAMELLIA128, AES, BLOWFISH, IDEA, CAST5, 3DES + Digest: SHA512, SHA384, SHA256, SHA224, RIPEMD160, SHA1 + Compression: ZLIB, BZIP2, ZIP, Uncompressed + Features: MDC, Keyserver no-modify + + bash-4.4$ +#+END_SRC + + +** Subkeys + :PROPERTIES: + :CUSTOM_ID: keygen-subkeys + :END: + +Adding subkeys to a primary key is fairly similar to creating the +primary key with the =create_subkey= method. Most of the arguments +are the same, but not quite all. Instead of the =userid= argument +there is now a =key= argument for selecting which primary key to add +the subkey to. + +In the following example an encryption subkey will be added to the +primary key. Since Danger Mouse is a security conscious secret agent, +this subkey will only be valid for about six months, half the length +of the primary key. + +#+BEGIN_SRC python -i +import gpg + +c = gpg.Context() +c.home_dir = "~/.gnupg-dm" + +key = c.get_key(dmkey.fpr, secret=True) +dmsub = c.create_subkey(key, algorithm="rsa3072", expires_in=15768000, + encrypt=True) +#+END_SRC + +As with the primary key, the results here can be checked with: + +#+BEGIN_SRC python -i +print(""" + Fingerprint: {0} + Primary Key: {1} + Public Key: {2} + Secret Key: {3} + Sub Key: {4} +User IDs: {5} +""".format(dmsub.fpr, dmsub.primary, dmsub.pubkey, dmsub.seckey, dmsub.sub, + dmsub.uid)) +#+END_SRC + +As well as on the command line with: + +#+BEGIN_SRC shell + bash-4.4$ gpg --homedir ~/.gnupg-dm -K + ~/.gnupg-dm/pubring.kbx + ---------------------- + sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] + 177B7C25DB99745EE2EE13ED026D2F19E99E63AA + uid [ultimate] Danger Mouse + ssb rsa3072 2018-03-15 [E] [expires: 2018-09-13] + + bash-4.4$ +#+END_SRC + + +** User IDs + :PROPERTIES: + :CUSTOM_ID: keygen-uids + :END: + + +*** Adding User IDs + :PROPERTIES: + :CUSTOM_ID: keygen-uids-add + :END: + +By comparison to creating primary keys and subkeys, adding a new user +ID to an existing key is much simpler. The method used to do this is +=key_add_uid= and the only arguments it takes are for the =key= and +the new =uid=. + +#+BEGIN_SRC python -i +import gpg + +c = gpg.Context() +c.home_dir = "~/.gnupg-dm" + +dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" +key = c.get_key(dmfpr, secret=True) +uid = "Danger Mouse " + +c.key_add_uid(key, uid) +#+END_SRC + +Unsurprisingly the result of this is: + +#+BEGIN_SRC shell + bash-4.4$ gpg --homedir ~/.gnupg-dm -K + ~/.gnupg-dm/pubring.kbx + ---------------------- + sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] + 177B7C25DB99745EE2EE13ED026D2F19E99E63AA + uid [ultimate] Danger Mouse + uid [ultimate] Danger Mouse + ssb rsa3072 2018-03-15 [E] [expires: 2018-09-13] + + bash-4.4$ +#+END_SRC + + +*** Revoking User IDs + :PROPERTIES: + :CUSTOM_ID: keygen-uids-revoke + :END: + +Revoking a user ID is a fairly similar process, except that it uses +the =key_revoke_uid= method. + +#+BEGIN_SRC python -i +import gpg + +c = gpg.Context() +c.home_dir = "~/.gnupg-dm" + +dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" +key = c.get_key(dmfpr, secret=True) +uid = "Danger Mouse " + +c.key_revoke_uid(key, uid) +#+END_SRC + + +** Key certification + :PROPERTIES: + :CUSTOM_ID: key-sign + :END: + +Since key certification is more frequently referred to as key signing, +the method used to perform this function is =key_sign=. + +The =key_sign= method takes four arguments: =key=, =uids=, +=expires_in= and =local=. The default value of =uids= is =None= and +which results in all user IDs being selected. The default value of +both =expires_in= and =local= is =False=; which results in the +signature never expiring and being able to be exported. + +The =key= is the key being signed rather than the key doing the +signing. To change the key doing the signing refer to the signing key +selection above for signing messages and files. + +If the =uids= value is not =None= then it must either be a string to +match a single user ID or a list of strings to match multiple user +IDs. In this case the matching of those strings must be precise and +it is case sensitive. + +To sign Danger Mouse's key for just the initial user ID with a +signature which will last a little over a month, do this: + +#+BEGIN_SRC python -i +import gpg + +c = gpg.Context() +uid = "Danger Mouse " + +dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" +key = c.get_key(dmfpr, secret=True) +c.key_sign(key, uids=uid, expires_in=2764800) +#+END_SRC + + +*** Verifying key certifications + :PROPERTIES: + :CUSTOM_ID: key-sign-verify + :END: + +#+BEGIN_SRC python -i +import gpg +import time + +c = gpg.Context() +dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" +keys = list(c.keylist(pattern=dmuid, mode=gpg.constants.keylist.mode.SIGS)) +key = keys[0] + +for user in key.uids: + for sig in user.signatures: + print("0x{0}".format(sig.keyid), "", time.ctime(sig.timestamp), "", + sig.uid) +#+END_SRC + +Which for Danger Mouse displays the following: + +#+BEGIN_EXAMPLE + 0x92E3F6115435C65A Thu Mar 15 13:17:44 2018 Danger Mouse + 0x321E4E2373590E5D Mon Nov 26 12:46:05 2018 Ben McGinnes +#+END_EXAMPLE + +The two key signatures listed are for the self-certification of Danger +Mouse's key made when the key was created in March, 2018; and the +second is a signature made by the author and set to expire at the end +of the year. Note that the second signature was made with the +following code (including the preceding code to display the output of +the certifications or key signatures): + +#+BEGIN_SRC python -i +import gpg +import math +import pendulum +import time + +hd = "/home/dm/.gnupg" +c = gpg.Context() +d = gpg.Context(home_dir=hd) +dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" +dmuid = "Danger Mouse " +dkeys = list(c.keylist(pattern=dmuid)) +dmkey = dkeys[0] + +c.key_import(d.key_export(pattern=None)) + +tp = pendulum.period(pendulum.now(tz="local"), pendulum.datetime(2019, 1, 1)) +ts = tp.total_seconds() +total_secs = math.ceil(ts) +c.key_sign(dmkey, uids=dmuid, expires_in=total_secs) + +d.key_import(c.key_export(pattern=dmuid)) +keys = list(c.keylist(pattern=dmuid, mode=gpg.constants.keylist.mode.SIGS)) +key = keys[0] + +for user in key.uids: + for sig in user.signatures: + print("0x{0}".format(sig.keyid), "", time.ctime(sig.timestamp), "", + sig.uid) +#+END_SRC + +Note that this final code block includes the use of a module which is +/not/ part of Python's standard library, the [[https://pendulum.eustace.io/][pendulum module]]. Unlike +the standard datetime module, pendulum makes working with dates and +times significantly easier in Python; just as the requests module +makes working with HTTP and HTTPS easier than the builtin modules do. + +Though neither requests nor pendulum are required modules for using +the GPGME Python bindings, they are both highly recommended more +generally. + + +* Advanced or Experimental Use Cases + :PROPERTIES: + :CUSTOM_ID: advanced-use + :END: + + +** C plus Python plus SWIG plus Cython + :PROPERTIES: + :CUSTOM_ID: cython + :END: + +In spite of the apparent incongruence of using Python bindings to a C +interface only to generate more C from the Python; it is in fact quite +possible to use the GPGME bindings with [[http://docs.cython.org/en/latest/index.html][Cython]]. Though in many cases +the benefits may not be obvious since the most computationally +intensive work never leaves the level of the C code with which GPGME +itself is interacting with. + +Nevertheless, there are some situations where the benefits are +demonstrable. One of the better and easier examples being the one of +the early examples in this HOWTO, the [[#howto-keys-counting][key counting]] code. Running that +example as an executable Python script, =keycount.py= (available in +the =examples/howto/= directory), will take a noticeable amount of time +to run on most systems where the public keybox or keyring contains a +few thousand public keys. + +Earlier in the evening, prior to starting this section, I ran that +script on my laptop; as I tend to do periodically and timed it using +=time= utility, with the following results: + +#+BEGIN_SRC shell + bash-4.4$ time keycount.py + + Number of secret keys: 23 + Number of public keys: 12112 + + + real 11m52.945s + user 0m0.913s + sys 0m0.752s + + bash-4.4$ +#+END_SRC + +Sometime after that I imported another key and followed it with a +little test of Cython. This test was kept fairly basic, essentially +lifting the material from the [[http://docs.cython.org/en/latest/src/tutorial/cython_tutorial.html][Cython Basic Tutorial]] to demonstrate +compiling Python code to C. The first step was to take the example +key counting code quoted previously, essentially from the importing of +the =gpg= module to the end of the script: + +#+BEGIN_SRC python -i +import gpg + +c = gpg.Context() +seckeys = c.keylist(pattern=None, secret=True) +pubkeys = c.keylist(pattern=None, secret=False) + +seclist = list(seckeys) +secnum = len(seclist) + +publist = list(pubkeys) +pubnum = len(publist) + +print(""" + Number of secret keys: {0} + Number of public keys: {1} + +""".format(secnum, pubnum)) +#+END_SRC + +Save that into a file called =keycount.pyx= and then create a +=setup.py= file which contains this: + +#+BEGIN_SRC python -i +from distutils.core import setup +from Cython.Build import cythonize + +setup( + ext_modules = cythonize("keycount.pyx") +) +#+END_SRC + +Compile it: + +#+BEGIN_SRC shell + bash-4.4$ python setup.py build_ext --inplace + bash-4.4$ +#+END_SRC + +Then run it in a similar manner to =keycount.py=: + +#+BEGIN_SRC shell + bash-4.4$ time python3.7 -c "import keycount" + + Number of secret keys: 23 + Number of public keys: 12113 + + + real 6m47.905s + user 0m0.785s + sys 0m0.331s + + bash-4.4$ +#+END_SRC + +Cython turned =keycount.pyx= into an 81KB =keycount.o= file in the +=build/= directory, a 24KB =keycount.cpython-37m-darwin.so= file to be +imported into Python 3.7 and a 113KB =keycount.c= generated C source +code file of nearly three thousand lines. Quite a bit bigger than the +314 bytes of the =keycount.pyx= file or the full 1,452 bytes of the +full executable =keycount.py= example script. + +On the other hand it ran in nearly half the time; taking 6 minutes and +47.905 seconds to run. As opposed to the 11 minutes and 52.945 seconds +which the CPython script alone took. + +The =keycount.pyx= and =setup.py= files used to generate this example +have been added to the =examples/howto/advanced/cython/= directory +The example versions include some additional options to annotate the +existing code and to detect Cython's use. The latter comes from the +[[http://docs.cython.org/en/latest/src/tutorial/pure.html#magic-attributes-within-the-pxd][Magic Attributes]] section of the Cython documentation. + + +* Miscellaneous extras and work-arounds + :PROPERTIES: + :CUSTOM_ID: cheats-and-hacks + :END: + +Most of the things in the following sections are here simply because +there was no better place to put them, even though some are only +peripherally related to the GPGME Python bindings. Some are also +workarounds for functions not integrated with GPGME as yet. This is +especially true of the first of these, dealing with [[#group-lines][group lines]]. + + +** Group lines + :PROPERTIES: + :CUSTOM_ID: group-lines + :END: + +There is not yet an easy way to access groups configured in the +gpg.conf file from within GPGME. As a consequence these central +groupings of keys cannot be shared amongst multiple programs, such as +MUAs readily. + +The following code, however, provides a work-around for obtaining this +information in Python. + +#+BEGIN_SRC python -i +import subprocess +import sys + +if sys.platform == "win32": + gpgconfcmd = "gpgconf.exe --list-options gpg" +else: + gpgconfcmd = "gpgconf --list-options gpg" + +process = subprocess.Popen(gpgconfcmd.split(), stdout=subprocess.PIPE) +procom = process.communicate() + +if sys.version_info[0] == 2: + lines = procom[0].splitlines() +else: + lines = procom[0].decode().splitlines() + +for line in lines: + if line.startswith("group") is True: + break + +groups = line.split(":")[-1].replace('"', '').split(',') + +group_lines = [] +group_lists = [] + +for group in groups: + group_lines.append(group.split("=")) + group_lists.append(group.split("=")) + +for glist in group_lists: + glist[1] = glist[1].split() +#+END_SRC + +The result of that code is that =group_lines= is a list of lists where +=group_lines[i][0]= is the name of the group and =group_lines[i][1]= +is the key IDs of the group as a string. + +The =group_lists= result is very similar in that it is a list of +lists. The first part, =group_lists[i][0]= matches +=group_lines[i][0]= as the name of the group, but =group_lists[i][1]= +is the key IDs of the group as a list. + +A demonstration of using the =groups.py= module is also available in +the form of the executable =mutt-groups.py= script. This second +script reads all the group entries in a user's =gpg.conf= file and +converts them into crypt-hooks suitable for use with the Mutt and +Neomutt mail clients. + + +** Keyserver access for Python + :PROPERTIES: + :CUSTOM_ID: hkp4py + :END: + +The [[https://github.com/Selfnet/hkp4py][hkp4py]] module by Marcel Fest was originally a port of the old +[[https://github.com/dgladkov/python-hkp][python-hkp]] module from Python 2 to Python 3 and updated to use the +[[http://docs.python-requests.org/en/latest/index.html][requests]] module instead. It has since been modified to provide +support for Python 2.7 as well and is available via PyPI. + +Since it rewrites the =hkp= protocol prefix as =http= and =hkps= as +=https=, the module is able to be used even with servers which do not +support the full scope of keyserver functions.[fn:5] It also works quite +readily when incorporated into a [[#cython][Cython]] generated and compiled version +of any code. + + +*** Key import format + :PROPERTIES: + :CUSTOM_ID: hkp4py-strings + :END: + +The hkp4py module returns key data via requests as string literals +(=r.text=) instead of byte literals (=r.content=). This means that +the retrurned key data must be encoded to UTF-8 when importing that +key material using a =gpg.Context().key_import()= method. + +For this reason an alternative method has been added to the =search= +function of =hkp4py.KeyServer()= which returns the key in the correct +format as expected by =key_import=. When importing using this module, +it is now possible to import with this: + +#+BEGIN_SRC python -i +for key in keys: + if key.revoked is False: + gpg.Context().key_import(key.key_blob) + else: + pass +#+END_SRC + +Without that recent addition it would have been necessary to encode +the contents of each =hkp4py.KeyServer().search()[i].key= in +=hkp4py.KeyServer().search()= before trying to import it. + +An example of this is included in the [[#howto-import-key][Importing Keys]] section of this +HOWTO and the corresponding executable version of that example is +available in the =lang/python/examples/howto= directory as normal; the +executable version is the =import-keys-hkp.py= file. + + +** GPGME version checking + :PROPERTIES: + :CUSTOM_ID: gpgme-version-check + :END: + +For various reasons it may be necessary to check which version of +GPGME the bindings have been built against; including whether a +minimum required version of GPGME is in use. + +For the most part the =gpg.version.versionstr= and +=gpg.version.versionlist= methods have been quite sufficient. The +former returns the same string as =gpgme-config --version=, while the +latter returns the major, minor and patch values in a list. + +To check if the installed bindings have actually been built against +the current installed libgpgme version, this check can be performed: + +#+BEGIN_SRC python -i +import gpg +import subprocess +import sys + +gpgme_version_call = subprocess.Popen(["gpgme-config", "--version"], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) +gpgme_version_str = gpgme_version_call.communicate() + +if sys.version_info[0] == 2: + gpgme_version = gpgme_version_str[0].strip() +elif sys.version_info[0] >= 3: + gpgme_version = gpgme_version_str[0].decode().strip() +else: + gpgme_version = None + +if gpgme_version is not None: + if gpgme_version == gpg.version.versionstr: + print("The GPGME Python bindings match libgpgme.") + else: + print("The GPGME Python bindings do NOT match libgpgme.") +else: + print("Upgrade Python and reinstall the GPGME Python bindings.") +#+END_SRC + +For many developers, however, the preferred checking means checking +for a minimum version or point release. This is now readily available +via the =gpg.version.versionintlist= method (added in version +=1.12.1-beta79=). It is also now possible to easily check whether the +installed GPGME Python bindings were built from a development or beta +branch of the GPGME source code. + +The following code demonstrates how both of those methods may be used: + +#+BEGIN_SRC python -i +import gpg + +try: + if gpg.version.is_beta is True: + print("The installed GPGME Python bindings were built from beta code.") + else: + print("The installed GPGME Python bindings are a released version.") +except Exception as e: + print(e) + +try: + if gpg.version.versionintlist[0] == 1: + if gpg.version.versionintlist[1] == 12: + if gpg.version.versionintlist[2] == 1: + print("This is the minimum version for using versionintlist.") + elif gpg.version.versionintlist[2] > 1: + print("The versionintlist method is available.") + else: + pass + elif gpg.version.versionintlist[1] > 12: + print("The versionintlist method is available.") + else: + pass + elif gpg.version.versionintlist[0] > 1: + print("The versionintlist method is available.") + else: + pass +except Exception as e: + print(e) +#+END_SRC + +The points where =pass= is used in the above example will most likely +also produce an =Exception= error since those results should only +occur in versions which do not have the =gpgme.version.is_beta= and +=gpgme.version.versionintlist= methods available. + + +* Copyright and Licensing + :PROPERTIES: + :CUSTOM_ID: copyright-and-license + :END: + + +** Copyright + :PROPERTIES: + :CUSTOM_ID: copyright + :END: + +Copyright © The GnuPG Project, 2018. + + +** Draft Editions of this HOWTO + :PROPERTIES: + :CUSTOM_ID: draft-editions + :END: + +Draft editions of this HOWTO may be periodically available directly +from the author at any of the following URLs: + +- [[https://files.au.adversary.org/crypto/gpgme-python-howto.html][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 SSL)]] +- [[http://files.au.adversary.org/crypto/gpgme-python-howto.html][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 no SSL)]] +- [[https://files.au.adversary.org/crypto/gpgme-python-howto-split/index.html][GPGME Python Bindings HOWTO draft (HTML multiple files, AWS S3 SSL)]] +- [[http://files.au.adversary.org/crypto/gpgme-python-howto/index.html][GPGME Python Bindings HOWTO draft (HTML multiple files, AWS S3 no SSL)]] + +These draft versions have been generated from this document via GNU +Emacs [[https://orgmode.org/][Org mode]] to =.texi= and [[https://www.gnu.org/software/texinfo/][GNU Texinfo]] to HTML. Though it is +likely that the specific [[https://files.au.adversary.org/crypto/gpgme-python-howto][file]] [[http://files.au.adversary.org/crypto/gpgme-python-howto.org][version]] used will be on the same server +with the generated output formats. Occasionally I may include the Org +mode generated XHTML versions: + +- [[https://files.au.adversary.org/crypto/gpgme-python-howto.xhtml][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 SSL)]] +- [[http://files.au.adversary.org/crypto/gpgme-python-howto.xhtml][GPGME Python Bindings HOWTO draft (HTML single file, AWS S3 no SSL)]] + +That XHTML version, however, is exported in a way which inherits a +colour scheme from [[https://github.com/holomorph/emacs-zenburn][the author's Emacs theme]] (which is a higher contrast +version of [[http://kippura.org/zenburnpage/][Zenburn]] ported by [[https://github.com/holomorph][Holomorph]]). So it's fine for people who +prefer dark themed web pages, but not so great for everyone else. + +The GNU Texinfo and reStructured Text versions ship with the software, +while the GNU Emacs Info version is generated from the Texinfo +version using GNU Texinfo or GNU Makeinfo. The Texinfo format is +generated from the original Org mode source file in Org mode itself +either within GNU Emacs or via the command line by invoking Emacs in +batch mode: + +#+BEGIN_SRC shell + emacs gpgme-python-howto.org --batch -f org-texinfo-export-to-texinfo --kill + emacs gpgme-python-howto --batch -f org-texinfo-export-to-texinfo --kill +#+END_SRC + +The reStructuredText format is also generated from the Org mode source +file, except it is generated using [[https://pandoc.org][Pandoc]] with either of the following +commands (depending on the filename): + +#+BEGIN_SRC shell + pandoc -f org -t rst+smart -o gpgme-python-howto.rst gpgme-python-howto.org + pandoc -f org -t rst+smart -o gpgme-python-howto.rst gpgme-python-howto +#+END_SRC + +Note that the Org mode source files are identified as such via a mode +line at the top of each file and have had their =.org= file extensions +dropped in order to make scripted generation of output formats easier +and not require renaming files post-conversion. + +Due to a bug in Org mode's texinfo conversion method, the recommended +steps for generating the Texinfo files for all the files in the +=lang/python/doc/src/= directory are as follows: + +#+BEGIN_SRC shell + for x in * ; do + emacs $x --batch -f org-texinfo-export-to-texinfo --kill + cat $x.texi | sed -e 's/@documentencoding UTF-8/@documentencoding utf-8/g' > ../texinfo/$x.texi + pandoc -f org -t rst+smart -o ../rst/$x.rst $x + done ; + rm -fv *.texi + cd ../texinfo + mkdir info + mkdir html + for x in *.texi ; do + makeinfo -v $x + makeinfo --html --no-split $x + done ; + mv *.info info/ + mv *.html html/ +#+END_SRC + +This code snippet includes the generation of the reStructuredText +files and would be expected to be run from the =doc/src/= directory +containing the Org mode source files. It also assumes that the +commands are being run on POSIX compliant systems with basic tools +like sed, the Bourne shell and GNU Emacs[fn:6] available. The code +snippet also includes the steps for generating the Emacs Info files +and HTML files from the Texinfo files. Using reStructuredText files +with Sphinx is best left for the documentation of that project. + +In addition to these there is a significantly less frequently updated +version as a HTML [[https://files.au.adversary.org/crypto/gpgme-python/dita/webhelp/index.html][WebHelp site]] (AWS S3 SSL); generated from DITA XML +source files, which can be found in [[https://dev.gnupg.org/source/gpgme/browse/ben%252Fhowto-dita/][an alternative branch]] of the GPGME +git repository. + +Various generated output formats may occasionally be found in +subdirectories of the [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python][gpgme-python]] directory. In particular within +the [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python/dita][DITA]], [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python/rst][reStructuredText]] and [[https://s3.amazonaws.com/files.au.adversary.org/crypto/gpgme-python/texinfo][Texinfo]] subdirectories. The =rst= +directory contains output files generated with Sphinx and may include a +considerable number of its possible output formats, but there are no +guarantees as to how recent these are or even if they are present. + +These draft editions are not official documents and the version of +documentation in the master branch or which ships with released +versions is the only official documentation. Nevertheless, these +draft editions may occasionally be of use by providing more accessible +web versions which are updated between releases. They are provided on +the understanding that they may contain errors or may contain content +subject to change prior to an official release. + + +** License GPL compatible + :PROPERTIES: + :CUSTOM_ID: license + :END: + +This file is free software; as a special exception the author gives +unlimited permission to copy and/or distribute it, with or without +modifications, as long as this notice is preserved. + +This file is distributed in the hope that it will be useful, but +WITHOUT ANY WARRANTY, to the extent permitted by law; without even the +implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR +PURPOSE. + + +* Footnotes + +[fn:1] =short-history= and/or =short-history.html=. + +[fn:2] With no issues reported specific to Python 3.7, the release of +Python 3.7.1 at around the same time as GPGME 1.12.0 and the testing +with Python 3.7.1rc1, there is no reason to delay moving 3.7 ahead of +3.6 now. Production environments with more conservative requirements +will always enforce their own policies anyway and installation to each +supported minor release is quite possible too. + +[fn:3] Yes, even if you use virtualenv with everything you do in +Python. If you want to install this module as just your user account +then you will need to manually configure, compile and install the +/entire/ GnuPG stack as that user as well. This includes libraries +which are not often installed that way. It can be done and there are +circumstances under which it is worthwhile, but generally only on +POSIX systems which utilise single user mode (some even require it). + +[fn:4] You probably don't really want to do this. Searching the +keyservers for "gnupg.org" produces over 400 results, the majority of +which aren't actually at the gnupg.org domain, but just included a +comment regarding the project in their key somewhere. + +[fn:5] Such as with ProtonMail servers. This also means that +restricted servers which only advertise either HTTP or HTTPS end +points and not HKP or HKPS end points must still be identified as as +HKP or HKPS within the Python Code. The =hkp4py= module will rewrite +these appropriately when the connection is made to the server. + +[fn:6] Okay, Emacs might not necessarily qualify as a basic tool, but +it is common enough that having it installed on a system isn't too +great an expectation, nor is it difficult to add to most POSIX +systems, even if the users of those systems do not personally use it. diff --git a/lang/python/doc/src/index b/lang/python/doc/src/index deleted file mode 100644 index ec13c9a1..00000000 --- a/lang/python/doc/src/index +++ /dev/null @@ -1,29 +0,0 @@ -# -*- mode: org -*- -#+TITLE: GNU Privacy Guard (GnuPG) Made Easy Python Bindings -#+AUTHOR: Ben McGinnes -#+LATEX_COMPILER: xelatex -#+LATEX_CLASS: article -#+LATEX_CLASS_OPTIONS: [12pt] -#+LATEX_HEADER: \usepackage{xltxtra} -#+LATEX_HEADER: \usepackage[margin=1in]{geometry} -#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} -#+LATEX_HEADER: \author{Ben McGinnes } - - -* GPGME Python Bindings - :PROPERTIES: - :CUSTOM_ID: top - :END: - - -** Contents - :PROPERTIES: - :CUSTOM_ID: contents - :END: - - -- [[file:short-history][A short history of the project]] -- [[file:what-is-new][What's New]] - - [[file:maintenance-mode][Maintenance Mode]] (from January, 2019) -- [[file:what-was-new][What Was New]] -- [[file:gpgme-python-howto][GPGME Python Bindings HOWTO]] diff --git a/lang/python/doc/src/index.org b/lang/python/doc/src/index.org new file mode 100644 index 00000000..ec13c9a1 --- /dev/null +++ b/lang/python/doc/src/index.org @@ -0,0 +1,29 @@ +# -*- mode: org -*- +#+TITLE: GNU Privacy Guard (GnuPG) Made Easy Python Bindings +#+AUTHOR: Ben McGinnes +#+LATEX_COMPILER: xelatex +#+LATEX_CLASS: article +#+LATEX_CLASS_OPTIONS: [12pt] +#+LATEX_HEADER: \usepackage{xltxtra} +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} +#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} +#+LATEX_HEADER: \author{Ben McGinnes } + + +* GPGME Python Bindings + :PROPERTIES: + :CUSTOM_ID: top + :END: + + +** Contents + :PROPERTIES: + :CUSTOM_ID: contents + :END: + + +- [[file:short-history][A short history of the project]] +- [[file:what-is-new][What's New]] + - [[file:maintenance-mode][Maintenance Mode]] (from January, 2019) +- [[file:what-was-new][What Was New]] +- [[file:gpgme-python-howto][GPGME Python Bindings HOWTO]] diff --git a/lang/python/doc/src/maintenance-mode b/lang/python/doc/src/maintenance-mode deleted file mode 100644 index 4b4e0fca..00000000 --- a/lang/python/doc/src/maintenance-mode +++ /dev/null @@ -1,135 +0,0 @@ -# -*- mode: org -*- -#+TITLE: Maintenance Mode -#+AUTHOR: Ben McGinnes -#+LATEX_COMPILER: xelatex -#+LATEX_CLASS: article -#+LATEX_CLASS_OPTIONS: [12pt] -#+LATEX_HEADER: \usepackage{xltxtra} -#+LATEX_HEADER: \usepackage[margin=1in]{geometry} -#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} -#+LATEX_HEADER: \author{Ben McGinnes } - - -* Maintenance Mode from 2019 - :PROPERTIES: - :CUSTOM_ID: maintenance-mode - :END: - -| Version: | 0.0.1 | -| GPGME Version: | 1.13.0 | -| Author: | Ben McGinnes | -| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | -| Language: | Australian English, British English | -| xml:lang: | en-AU, en-GB, en | - -From the beginning of 2019 the Python bindings to GPGME will enter -maintenance mode, meaning that new features will not be added and only -bug fixes and security fixes will be made. This also means that -documentation beyond that existing at the end of 2018 will not be -developed further except to correct errors. - -Though use of these bindings appears to have been quite well received, -there has been no indication of what demand there is, if any for -either financial backing of the current Python bindings development or -support contracts with g10code GmbH citing the necessity of including -the bindings. - - -** Maintainer from 2019 onward - :PROPERTIES: - :CUSTOM_ID: maintenance-mode-bm - :END: - -How does this affect the position of GnuPG Python Bindings Maintainer? - -Well, I will remain as maintainer of the bindings; but without funding -for that position, the amount of time I will be able to dedicate -solely to this task will be limited and reduced to volunteered time. -As with all volunteered time and effort in free software projects, -this will be subject to numerous external imperatives. - - -** Using the Python Bindings from 2019 and beyond - :PROPERTIES: - :CUSTOM_ID: maintenance-mode-blade-runner - :END: - -For most, if not all, Python developers using these bindings; they -will continue to “just work” the same as they always have. Expansions -of GPGME itself are usually handled by SWIG with the existing code and -thus bindings are generated properly when the bindings are installed -alongside GPGME and when the latter is built from source. - -In the rare circumstances where that is not enough to address some new -addition to GPGME, then that is a bug and thus subject to the -maintenance mode provisions (i.e. it will be fixed following a bug -report being raised and your humble author will need to remember where -the timesheet template was filed, depending on how many years off such -an event is). - -All the GPGME functionality will continue to be accessible via the -lower level, dynamically generated methods which match the GPGME C -documentation. While the more intuitively Pythonic higher level layer -already covers the vast majority of functionality people require with -key generation, signatures, certifications (key signing), encryption, -decryption, verification, validation, trust levels and so on. - -Any wanted features lacking in the Python bindings are usually lacking -because they are missing from GPGME itself (e.g. revoking keys via the -API) and in such cases they are usually deliberately excluded. More -discussion of these issues can be found in the archives of the -[[https://lists.gnupg.org/mailman/listinfo/gnupg-devel][gnupg-devel mailing list]]. - -Any features existing in the dynamically generated layer for which -people want a specific, higher level function included to make it more -Pythonic (e.g. to avoid needing to learn or memorise cryptographic -mode values or GnuPG status code numbers), would be a feature request -and /not/ a bug. - -It is still worthwhile requesting it, but the addition of such a -feature would not be guaranteed and provided on a purely volunteer -basis. Expediting such a request would require funding that request. - -Those with a commercial interest in expediting such a feature request -already know how to [[https://gnupg.org/cgi-bin/procdonate.cgi?mode=preset][expedite it]] (use the message field to state what -feature is being requested). - - -** Documentation formats - :PROPERTIES: - :CUSTOM_ID: docs - :END: - -The documentation has been written in Org mode for GNU Emacs, with -both Texinfo and reStructuredText formats generated from that. The -Texinfo files are intended for use with the rest of the GnuPG -documentation; while the reStructuredText files are intended for use -with Docutils and Sphinx, as with other Python projects. - - -*** Cautionary Notes regarding Sphinx and EPUB - :PROPERTIES: - :CUSTOM_ID: sphinx-made-epubs-suck - :END: - -Though Python's Docutils in conjunction with Sphinx is capable of -generating some very useful HTML sites, as proven by [[https://readthedocs.org/][Read the Docs]] and -the [[https://docs.python.org/][Python documentation]], there are a number of output formats it does -not handle well. At the top of the list of things it manages to break -so atrociously as to be embarassing is the [[http://idpf.org/epub][EPUB 3]] format. - -The automatically generated EPUB of the CPython documentation always -contains hundreds of validation errors and even the modest amount of -documentation here [[https://files.au.adversary.org.s3.amazonaws.com/crypto/gpgme-python/rst/epub/GPGMEPythonBindings.epub][produced a file]] with approximately thirty -validation errors. As the volume of documentation content increases, -so does the induced errors. Whereas Texinfo doesn't produce EPUB -output at all, nor does Org-mode. - -Should there ever be genuine demand for this format, lodge a [[https://dev.gnupg.org/maniphest/task/edit/form/4/][feature -request]] case marked for [[https://dev.gnupg.org/p/BenM/][my]] attention. The means of generating such -files flawlessly is already available, but is not yet part of the -GnuPG build system. Nor is it integrated with a means of converting -Org mode input files to the relevant base format automatically, as can -already be done when converting Org to reStructuredText or Org to -Texinfo. As a certain amount of work would be required to get it -done, there would need to be clear demand for that work to be done. diff --git a/lang/python/doc/src/maintenance-mode.org b/lang/python/doc/src/maintenance-mode.org new file mode 100644 index 00000000..4b4e0fca --- /dev/null +++ b/lang/python/doc/src/maintenance-mode.org @@ -0,0 +1,135 @@ +# -*- mode: org -*- +#+TITLE: Maintenance Mode +#+AUTHOR: Ben McGinnes +#+LATEX_COMPILER: xelatex +#+LATEX_CLASS: article +#+LATEX_CLASS_OPTIONS: [12pt] +#+LATEX_HEADER: \usepackage{xltxtra} +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} +#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} +#+LATEX_HEADER: \author{Ben McGinnes } + + +* Maintenance Mode from 2019 + :PROPERTIES: + :CUSTOM_ID: maintenance-mode + :END: + +| Version: | 0.0.1 | +| GPGME Version: | 1.13.0 | +| Author: | Ben McGinnes | +| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | +| Language: | Australian English, British English | +| xml:lang: | en-AU, en-GB, en | + +From the beginning of 2019 the Python bindings to GPGME will enter +maintenance mode, meaning that new features will not be added and only +bug fixes and security fixes will be made. This also means that +documentation beyond that existing at the end of 2018 will not be +developed further except to correct errors. + +Though use of these bindings appears to have been quite well received, +there has been no indication of what demand there is, if any for +either financial backing of the current Python bindings development or +support contracts with g10code GmbH citing the necessity of including +the bindings. + + +** Maintainer from 2019 onward + :PROPERTIES: + :CUSTOM_ID: maintenance-mode-bm + :END: + +How does this affect the position of GnuPG Python Bindings Maintainer? + +Well, I will remain as maintainer of the bindings; but without funding +for that position, the amount of time I will be able to dedicate +solely to this task will be limited and reduced to volunteered time. +As with all volunteered time and effort in free software projects, +this will be subject to numerous external imperatives. + + +** Using the Python Bindings from 2019 and beyond + :PROPERTIES: + :CUSTOM_ID: maintenance-mode-blade-runner + :END: + +For most, if not all, Python developers using these bindings; they +will continue to “just work” the same as they always have. Expansions +of GPGME itself are usually handled by SWIG with the existing code and +thus bindings are generated properly when the bindings are installed +alongside GPGME and when the latter is built from source. + +In the rare circumstances where that is not enough to address some new +addition to GPGME, then that is a bug and thus subject to the +maintenance mode provisions (i.e. it will be fixed following a bug +report being raised and your humble author will need to remember where +the timesheet template was filed, depending on how many years off such +an event is). + +All the GPGME functionality will continue to be accessible via the +lower level, dynamically generated methods which match the GPGME C +documentation. While the more intuitively Pythonic higher level layer +already covers the vast majority of functionality people require with +key generation, signatures, certifications (key signing), encryption, +decryption, verification, validation, trust levels and so on. + +Any wanted features lacking in the Python bindings are usually lacking +because they are missing from GPGME itself (e.g. revoking keys via the +API) and in such cases they are usually deliberately excluded. More +discussion of these issues can be found in the archives of the +[[https://lists.gnupg.org/mailman/listinfo/gnupg-devel][gnupg-devel mailing list]]. + +Any features existing in the dynamically generated layer for which +people want a specific, higher level function included to make it more +Pythonic (e.g. to avoid needing to learn or memorise cryptographic +mode values or GnuPG status code numbers), would be a feature request +and /not/ a bug. + +It is still worthwhile requesting it, but the addition of such a +feature would not be guaranteed and provided on a purely volunteer +basis. Expediting such a request would require funding that request. + +Those with a commercial interest in expediting such a feature request +already know how to [[https://gnupg.org/cgi-bin/procdonate.cgi?mode=preset][expedite it]] (use the message field to state what +feature is being requested). + + +** Documentation formats + :PROPERTIES: + :CUSTOM_ID: docs + :END: + +The documentation has been written in Org mode for GNU Emacs, with +both Texinfo and reStructuredText formats generated from that. The +Texinfo files are intended for use with the rest of the GnuPG +documentation; while the reStructuredText files are intended for use +with Docutils and Sphinx, as with other Python projects. + + +*** Cautionary Notes regarding Sphinx and EPUB + :PROPERTIES: + :CUSTOM_ID: sphinx-made-epubs-suck + :END: + +Though Python's Docutils in conjunction with Sphinx is capable of +generating some very useful HTML sites, as proven by [[https://readthedocs.org/][Read the Docs]] and +the [[https://docs.python.org/][Python documentation]], there are a number of output formats it does +not handle well. At the top of the list of things it manages to break +so atrociously as to be embarassing is the [[http://idpf.org/epub][EPUB 3]] format. + +The automatically generated EPUB of the CPython documentation always +contains hundreds of validation errors and even the modest amount of +documentation here [[https://files.au.adversary.org.s3.amazonaws.com/crypto/gpgme-python/rst/epub/GPGMEPythonBindings.epub][produced a file]] with approximately thirty +validation errors. As the volume of documentation content increases, +so does the induced errors. Whereas Texinfo doesn't produce EPUB +output at all, nor does Org-mode. + +Should there ever be genuine demand for this format, lodge a [[https://dev.gnupg.org/maniphest/task/edit/form/4/][feature +request]] case marked for [[https://dev.gnupg.org/p/BenM/][my]] attention. The means of generating such +files flawlessly is already available, but is not yet part of the +GnuPG build system. Nor is it integrated with a means of converting +Org mode input files to the relevant base format automatically, as can +already be done when converting Org to reStructuredText or Org to +Texinfo. As a certain amount of work would be required to get it +done, there would need to be clear demand for that work to be done. diff --git a/lang/python/doc/src/short-history b/lang/python/doc/src/short-history deleted file mode 100644 index bc874aa2..00000000 --- a/lang/python/doc/src/short-history +++ /dev/null @@ -1,187 +0,0 @@ -# -*- mode: org -*- -#+TITLE: A Short History of the GPGME bindings for Python -#+LATEX_COMPILER: xelatex -#+LATEX_CLASS: article -#+LATEX_CLASS_OPTIONS: [12pt] -#+LATEX_HEADER: \usepackage{xltxtra} -#+LATEX_HEADER: \usepackage[margin=1in]{geometry} -#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} - -* Overview - :PROPERTIES: - :CUSTOM_ID: overview - :END: - -| Version: | 0.0.1 | -| GPGME Version: | 1.13.0 | -| Author: | Ben McGinnes | -| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | -| Language: | Australian English, British English | -| xml:lang: | en-AU, en-GB, en | - -The GPGME Python bindings passed through many hands and numerous -phases before, after a fifteen year journey, coming full circle to -return to the source. This is a short explanation of that journey. - -** In the beginning - :PROPERTIES: - :CUSTOM_ID: in-the-begining - :END: - - In 2002 John Goerzen released PyME; Python bindings for the GPGME - module which utilised the current release of Python of the time and - SWIG.[fn:1] Shortly after creating it and ensuring it worked he stopped - supporting it, though he left his work available on his Gopher - site. - - -** Keeping the flame alive - :PROPERTIES: - :CUSTOM_ID: keeping-the-flame-alive - :END: - - A couple of years later the project was picked up by Igor Belyi and - actively developed and maintained by him from 2004 to 2008. Igor's - whereabouts at the time of this document's creation are unknown, - but the current authors do hope he is well. We're assuming (or - hoping) that life did what life does and made continuing untenable. - - -** Passing the torch - :PROPERTIES: - :CUSTOM_ID: passing-the-torch - :END: - - In 2014 Martin Albrecht wanted to patch a bug in the PyME code and - discovered the absence of Igor. Following a discussion on the PyME - mailing list he became the new maintainer for PyME, releasing - version 0.9.0 in May of that year. He remains the maintainer of - the original PyME release in Python 2.6 and 2.7 (available via - PyPI). - - -** Coming full circle - :PROPERTIES: - :CUSTOM_ID: ouroboros - :END: - - In 2015 Ben McGinnes approached Martin about a Python 3 version, - while investigating how complex a task this would be the task ended - up being completed. A subsequent discussion with Werner Koch led - to the decision to fold the Python 3 port back into the original - GPGME release in the languages subdirectory for non-C bindings - under the module name of =pyme3=. - - In 2016 this PyME module was integrated back into the GPGME project - by Justus Winter. During the course of this work Justus adjusted - the port to restore limited support for Python 2, but not as many - minor point releases as the original PyME package supports. During - the course of this integration the package was renamed to more - accurately reflect its status as a component of GPGME. The =pyme3= - module was renamed to =gpg= and adopted by the upstream GnuPG team. - - In 2017 Justus departed G10code and the GnuPG team. Following this - Ben returned to maintain of gpgme Python bindings and continue - building them from that point. - - -* Relics of the past - :PROPERTIES: - :CUSTOM_ID: relics-past - :END: - -There are a few things, in addition to code specific factors, such as -SWIG itself, which are worth noting here. - -** The Annoyances of Git - :PROPERTIES: - :CUSTOM_ID: the-annoyances-of-git - :END: - - As anyone who has ever worked with git knows, submodules are - horrible way to deal with pretty much anything. In the interests - of avoiding migraines, that was skipped with addition of the PyME - code to GPGME. - - Instead the files were added to a subdirectory of the =lang/= - directory, along with a copy of the entire git log up to that point - as a separate file within the =lang/python/docs/= directory.[fn:2] - As the log for PyME is nearly 100KB and the log for GPGME is - approximately 1MB, this would cause considerable bloat, as well as - some confusion, should the two be merged. - - Hence the unfortunate, but necessary, step to simply move the - files. A regular repository version has been maintained should it - be possible to implement this better in the future. - - -** The Perils of PyPI - :PROPERTIES: - :CUSTOM_ID: the-perils-of-pypi - :END: - - The early port of the Python 2 =pyme= module as =pyme3= was never - added to PyPI while the focus remained on development and testing - during 2015 and early 2016. Later in 2016, however, when Justus - completed his major integration work and subsequently renamed the - module from =pyme3= to =gpg=, some prior releases were also - provided through PyPI. - - Since these bindings require a matching release of the GPGME - libraries in order to function, it was determined that there was - little benefit in also providing a copy through PyPI since anyone - obtaining the GPGME source code would obtain the Python bindings - source code at the same time. Whereas there was the potential to - sew confusion amongst Python users installing the module from PyPI, - only to discover that without the relevant C files, header files or - SWIG compiled binaries, the Python module did them little good. - - There are only two files on PyPI which might turn up in a search - for this module or a sample of its content: - - 1. gpg (1.8.0) - Python bindings for GPGME GnuPG cryptography library - 2. pyme (0.9.0) - Python support for GPGME GnuPG cryptography library - -*** GPG 1·8·0 - Python bindings for GPGME GnuPG cryptography library - :PROPERTIES: - :CUSTOM_ID: pypi-gpgme-180 - :END: - - This is the most recent version to reach PyPI and is the version - of the official Pyhon bindings which shipped with GPGME 1.8.0. If - you have GPGME 1.8.0 installed and /only/ 1.8.0 installed, then it - is probably safe to use this copy from PyPI. - - As there have been a lot of changes since the release of GPGME - 1.8.0, the GnuPG Project recommends not using this version of the - module and instead installing the current version of GPGME along - with the Python bindings included with that package. - - -*** PyME 0·9·0 - Python support for GPGME GnuPG cryptography library - :PROPERTIES: - :CUSTOM_ID: pypi-gpgme-90 - :END: - - This is the last release of the PyME bindings maintained by Martin - Albrecht and is only compatible with Python 2, it will not work - with Python 3. This is the version of the software from which the - port from Python 2 to Python 3 code was made in 2015. - - Users of the more recent Python bindings will recognise numerous - points of similarity, but also significant differences. It is - likely that the more recent official bindings will feel "more - pythonic." - - For those using Python 2, there is essentially no harm in using - this module, but it may lack a number of more recent features - added to GPGME. - - -* Footnotes - -[fn:1] In all likelihood this would have been Python 2.2 or possibly -Python 2.3. - -[fn:2] The entire PyME git log and other preceding VCS logs are -located in the =gpgme/lang/python/docs/old-commits.log= file. diff --git a/lang/python/doc/src/short-history.org b/lang/python/doc/src/short-history.org new file mode 100644 index 00000000..bc874aa2 --- /dev/null +++ b/lang/python/doc/src/short-history.org @@ -0,0 +1,187 @@ +# -*- mode: org -*- +#+TITLE: A Short History of the GPGME bindings for Python +#+LATEX_COMPILER: xelatex +#+LATEX_CLASS: article +#+LATEX_CLASS_OPTIONS: [12pt] +#+LATEX_HEADER: \usepackage{xltxtra} +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} +#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} + +* Overview + :PROPERTIES: + :CUSTOM_ID: overview + :END: + +| Version: | 0.0.1 | +| GPGME Version: | 1.13.0 | +| Author: | Ben McGinnes | +| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | +| Language: | Australian English, British English | +| xml:lang: | en-AU, en-GB, en | + +The GPGME Python bindings passed through many hands and numerous +phases before, after a fifteen year journey, coming full circle to +return to the source. This is a short explanation of that journey. + +** In the beginning + :PROPERTIES: + :CUSTOM_ID: in-the-begining + :END: + + In 2002 John Goerzen released PyME; Python bindings for the GPGME + module which utilised the current release of Python of the time and + SWIG.[fn:1] Shortly after creating it and ensuring it worked he stopped + supporting it, though he left his work available on his Gopher + site. + + +** Keeping the flame alive + :PROPERTIES: + :CUSTOM_ID: keeping-the-flame-alive + :END: + + A couple of years later the project was picked up by Igor Belyi and + actively developed and maintained by him from 2004 to 2008. Igor's + whereabouts at the time of this document's creation are unknown, + but the current authors do hope he is well. We're assuming (or + hoping) that life did what life does and made continuing untenable. + + +** Passing the torch + :PROPERTIES: + :CUSTOM_ID: passing-the-torch + :END: + + In 2014 Martin Albrecht wanted to patch a bug in the PyME code and + discovered the absence of Igor. Following a discussion on the PyME + mailing list he became the new maintainer for PyME, releasing + version 0.9.0 in May of that year. He remains the maintainer of + the original PyME release in Python 2.6 and 2.7 (available via + PyPI). + + +** Coming full circle + :PROPERTIES: + :CUSTOM_ID: ouroboros + :END: + + In 2015 Ben McGinnes approached Martin about a Python 3 version, + while investigating how complex a task this would be the task ended + up being completed. A subsequent discussion with Werner Koch led + to the decision to fold the Python 3 port back into the original + GPGME release in the languages subdirectory for non-C bindings + under the module name of =pyme3=. + + In 2016 this PyME module was integrated back into the GPGME project + by Justus Winter. During the course of this work Justus adjusted + the port to restore limited support for Python 2, but not as many + minor point releases as the original PyME package supports. During + the course of this integration the package was renamed to more + accurately reflect its status as a component of GPGME. The =pyme3= + module was renamed to =gpg= and adopted by the upstream GnuPG team. + + In 2017 Justus departed G10code and the GnuPG team. Following this + Ben returned to maintain of gpgme Python bindings and continue + building them from that point. + + +* Relics of the past + :PROPERTIES: + :CUSTOM_ID: relics-past + :END: + +There are a few things, in addition to code specific factors, such as +SWIG itself, which are worth noting here. + +** The Annoyances of Git + :PROPERTIES: + :CUSTOM_ID: the-annoyances-of-git + :END: + + As anyone who has ever worked with git knows, submodules are + horrible way to deal with pretty much anything. In the interests + of avoiding migraines, that was skipped with addition of the PyME + code to GPGME. + + Instead the files were added to a subdirectory of the =lang/= + directory, along with a copy of the entire git log up to that point + as a separate file within the =lang/python/docs/= directory.[fn:2] + As the log for PyME is nearly 100KB and the log for GPGME is + approximately 1MB, this would cause considerable bloat, as well as + some confusion, should the two be merged. + + Hence the unfortunate, but necessary, step to simply move the + files. A regular repository version has been maintained should it + be possible to implement this better in the future. + + +** The Perils of PyPI + :PROPERTIES: + :CUSTOM_ID: the-perils-of-pypi + :END: + + The early port of the Python 2 =pyme= module as =pyme3= was never + added to PyPI while the focus remained on development and testing + during 2015 and early 2016. Later in 2016, however, when Justus + completed his major integration work and subsequently renamed the + module from =pyme3= to =gpg=, some prior releases were also + provided through PyPI. + + Since these bindings require a matching release of the GPGME + libraries in order to function, it was determined that there was + little benefit in also providing a copy through PyPI since anyone + obtaining the GPGME source code would obtain the Python bindings + source code at the same time. Whereas there was the potential to + sew confusion amongst Python users installing the module from PyPI, + only to discover that without the relevant C files, header files or + SWIG compiled binaries, the Python module did them little good. + + There are only two files on PyPI which might turn up in a search + for this module or a sample of its content: + + 1. gpg (1.8.0) - Python bindings for GPGME GnuPG cryptography library + 2. pyme (0.9.0) - Python support for GPGME GnuPG cryptography library + +*** GPG 1·8·0 - Python bindings for GPGME GnuPG cryptography library + :PROPERTIES: + :CUSTOM_ID: pypi-gpgme-180 + :END: + + This is the most recent version to reach PyPI and is the version + of the official Pyhon bindings which shipped with GPGME 1.8.0. If + you have GPGME 1.8.0 installed and /only/ 1.8.0 installed, then it + is probably safe to use this copy from PyPI. + + As there have been a lot of changes since the release of GPGME + 1.8.0, the GnuPG Project recommends not using this version of the + module and instead installing the current version of GPGME along + with the Python bindings included with that package. + + +*** PyME 0·9·0 - Python support for GPGME GnuPG cryptography library + :PROPERTIES: + :CUSTOM_ID: pypi-gpgme-90 + :END: + + This is the last release of the PyME bindings maintained by Martin + Albrecht and is only compatible with Python 2, it will not work + with Python 3. This is the version of the software from which the + port from Python 2 to Python 3 code was made in 2015. + + Users of the more recent Python bindings will recognise numerous + points of similarity, but also significant differences. It is + likely that the more recent official bindings will feel "more + pythonic." + + For those using Python 2, there is essentially no harm in using + this module, but it may lack a number of more recent features + added to GPGME. + + +* Footnotes + +[fn:1] In all likelihood this would have been Python 2.2 or possibly +Python 2.3. + +[fn:2] The entire PyME git log and other preceding VCS logs are +located in the =gpgme/lang/python/docs/old-commits.log= file. diff --git a/lang/python/doc/src/what-is-new b/lang/python/doc/src/what-is-new deleted file mode 100644 index 2a993f2b..00000000 --- a/lang/python/doc/src/what-is-new +++ /dev/null @@ -1,54 +0,0 @@ -# -*- mode: org -*- -#+TITLE: What's New in the GPGME Python Bindings and Documentation -#+AUTHOR: Ben McGinnes -#+LATEX_COMPILER: xelatex -#+LATEX_CLASS: article -#+LATEX_CLASS_OPTIONS: [12pt] -#+LATEX_HEADER: \usepackage{xltxtra} -#+LATEX_HEADER: \usepackage[margin=1in]{geometry} -#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} -#+LATEX_HEADER: \author{Ben McGinnes } - - -* What's New - :PROPERTIES: - :CUSTOM_ID: new-stuff - :END: - -| Version: | 0.0.1 | -| GPGME Version: | 1.13.0 | -| Author: | Ben McGinnes | -| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | -| Language: | Australian English, British English | -| xml:lang: | en-AU, en-GB, en | - -Last time the most obviously new thing was adding the /What's New/ -section to the HOWTO. Now it's moving it out of the HOWTO. - - -** New in GPGME 1·13·0 - :PROPERTIES: - :CUSTOM_ID: new-stuff-1-13-0 - :END: - -Additions since GPGME 1.12.0 include: - -- Moving the /What's New/ section out of the basic [[file:gpgme-python-howto.org][HOWTO]] document and - into its own file so as to more readily include other documents - beyond that HOWTO. -- Moving the preceding, archival, segments into [[file:what-was-new.org][another file]]. -- Added =gpg.version.versionintlist= to make it easier for Python - developers to check for a specific version number, even with beta - versions (it will drop the "-betaN" part). -- Added expanded detail on issues pertaining to installing for Windows - users. -- Bindings enter [[file:maintenance-mode][maintenance mode]] from January, 2019. -- Added documentation on maintenance mode and what changes can be made - to the code when in that status. Essentially that boils down to bug - fixes only and no feature requests. -- The import-keys-hkp.py example script, which uses the =hkp4py= - module to search the SKS servers for a key, has been tightened up to - search for both hexadecimal key IDs and user ID strings with reduced - chance of unnecessary repitition. There may still be some - repetition if a key includes a user ID matching the hexadecimal - value of a key ID. diff --git a/lang/python/doc/src/what-is-new.org b/lang/python/doc/src/what-is-new.org new file mode 100644 index 00000000..2a993f2b --- /dev/null +++ b/lang/python/doc/src/what-is-new.org @@ -0,0 +1,54 @@ +# -*- mode: org -*- +#+TITLE: What's New in the GPGME Python Bindings and Documentation +#+AUTHOR: Ben McGinnes +#+LATEX_COMPILER: xelatex +#+LATEX_CLASS: article +#+LATEX_CLASS_OPTIONS: [12pt] +#+LATEX_HEADER: \usepackage{xltxtra} +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} +#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} +#+LATEX_HEADER: \author{Ben McGinnes } + + +* What's New + :PROPERTIES: + :CUSTOM_ID: new-stuff + :END: + +| Version: | 0.0.1 | +| GPGME Version: | 1.13.0 | +| Author: | Ben McGinnes | +| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | +| Language: | Australian English, British English | +| xml:lang: | en-AU, en-GB, en | + +Last time the most obviously new thing was adding the /What's New/ +section to the HOWTO. Now it's moving it out of the HOWTO. + + +** New in GPGME 1·13·0 + :PROPERTIES: + :CUSTOM_ID: new-stuff-1-13-0 + :END: + +Additions since GPGME 1.12.0 include: + +- Moving the /What's New/ section out of the basic [[file:gpgme-python-howto.org][HOWTO]] document and + into its own file so as to more readily include other documents + beyond that HOWTO. +- Moving the preceding, archival, segments into [[file:what-was-new.org][another file]]. +- Added =gpg.version.versionintlist= to make it easier for Python + developers to check for a specific version number, even with beta + versions (it will drop the "-betaN" part). +- Added expanded detail on issues pertaining to installing for Windows + users. +- Bindings enter [[file:maintenance-mode][maintenance mode]] from January, 2019. +- Added documentation on maintenance mode and what changes can be made + to the code when in that status. Essentially that boils down to bug + fixes only and no feature requests. +- The import-keys-hkp.py example script, which uses the =hkp4py= + module to search the SKS servers for a key, has been tightened up to + search for both hexadecimal key IDs and user ID strings with reduced + chance of unnecessary repitition. There may still be some + repetition if a key includes a user ID matching the hexadecimal + value of a key ID. diff --git a/lang/python/doc/src/what-was-new b/lang/python/doc/src/what-was-new deleted file mode 100644 index 8440373e..00000000 --- a/lang/python/doc/src/what-was-new +++ /dev/null @@ -1,106 +0,0 @@ -# -*- mode: org -*- -#+TITLE: What Was New in the GPGME Python Bindings and Documentation -#+AUTHOR: Ben McGinnes -#+LATEX_COMPILER: xelatex -#+LATEX_CLASS: article -#+LATEX_CLASS_OPTIONS: [12pt] -#+LATEX_HEADER: \usepackage{xltxtra} -#+LATEX_HEADER: \usepackage[margin=1in]{geometry} -#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} -#+LATEX_HEADER: \author{Ben McGinnes } - - -* What Was New - :PROPERTIES: - :CUSTOM_ID: new-stuff - :END: - -| Version: | 0.0.1 | -| GPGME Version: | 1.13.0 | -| Author: | Ben McGinnes | -| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | -| Language: | Australian English, British English | -| xml:lang: | en-AU, en-GB, en | - -The following are all the past /What's New/ sections for the Python -Bindings HOWTO and other documentation. - - -** What Was New in GPGME 1·12·0 - :PROPERTIES: - :CUSTOM_ID: gpgme-1-12-0 - :END: - -The most obviously new point for those reading this guide is this -section on other new things, but that’s hardly important. Not given -all the other things which spurred the need for adding this section -and its subsections. - - -*** New in GPGME 1·12·0 - :PROPERTIES: - :CUSTOM_ID: new-stuff-1-12-0 - :END: - -There have been quite a number of additions to GPGME and the Python -bindings to it since the last release of GPGME with versions 1.11.0 -and 1.11.1 in April, 2018. - -The bullet points of new additiions are: - -- an expanded section on [[file:gpgme-python-howto#installation][installing]] and [[file:gpgme-python-howto#snafu][troubleshooting]] the Python - bindings. -- The release of Python 3.7.0; which appears to be working just fine - with our bindings, in spite of intermittent reports of problems for - many other Python projects with that new release. -- Python 3.7 has been moved to the head of the specified python - versions list in the build process. -- In order to fix some other issues, there are certain underlying - functions which are more exposed through the [[file:gpgme-python-howto#howto-get-context][gpg.Context()]], but - ongoing documentation ought to clarify that or otherwise provide the - best means of using the bindings. Some additions to =gpg.core= and - the =Context()=, however, were intended (see below). -- Continuing work in identifying and confirming the cause of - oft-reported [[file:gpgme-python-howto#snafu-runtime-not-funtime][problems installing the Python bindings on Windows]]. -- GSOC: Google's Surreptitiously Ordered Conscription ... erm ... oh, - right; Google's Summer of Code. Though there were two hopeful - candidates this year; only one ended up involved with the GnuPG - Project directly, the other concentrated on an unrelated third party - project with closer ties to one of the GNU/Linux distributions than - to the GnuPG Project. Thus the Python bindings benefited from GSOC - participant Jacob Adams, who added the key_import function; building - on prior work by Tobias Mueller. -- Several new methods functions were added to the gpg.Context(), - including: [[file:gpgme-python-howto#howto-import-key][key_import]], [[file:gpgme-python-howto#howto-export-key][key_export]], [[file:gpgme-python-howto#howto-export-public-key][key_export_minimal]] and - [[file:gpgme-python-howto#howto-export-secret-key][key_export_secret]]. -- Importing and exporting examples include versions integrated with - Marcel Fest's recently released [[https://github.com/Selfnet/hkp4py][HKP for Python]] module. Some - [[file:gpgme-python-howto#hkp4py][additional notes on this module]] are included at the end of the HOWTO. -- Instructions for dealing with semi-walled garden implementations - like ProtonMail are also included. This is intended to make things - a little easier when communicating with users of ProtonMail's - services and should not be construed as an endorsement of said - service. The GnuPG Project neither favours, nor disfavours - ProtonMail and the majority of this deals with interacting with the - ProtonMail keyserver. -- Semi-formalised the location where [[file:gpgme-python-howto#draft-editions][draft versions]] of this HOWTO may - periodically be accessible. This is both for the reference of - others and testing the publishing of the document itself. Renamed - this file at around the same time. -- The Texinfo documentation build configuration has been replicated - from the parent project in order to make to maintain consistency - with that project (and actually ship with each release). -- a reStructuredText (=.rst=) version is also generated for Python - developers more used to and comfortable with that format as it is - the standard Python documentation format and Python developers may - wish to use it with Sphinx. Please note that there has been no - testing of the reStructuredText version with Sphinx at all. The - reST file was generated by the simple expedient of using [[https://pandoc.org/][Pandoc]]. -- Added a new section for [[file:gpgme-python-howto#advanced-use][advanced or experimental use]]. -- Began the advanced use cases with [[file:gpgme-python-howto#cython][a section]] on using the module with - [[https://cython.org/][Cython]]. -- Added a number of new scripts to the =example/howto/= directory; - some of which may be in advance of their planned sections of the - HOWTO (and some are just there because it seemed like a good idea at - the time). -- Cleaned up a lot of things under the hood. diff --git a/lang/python/doc/src/what-was-new.org b/lang/python/doc/src/what-was-new.org new file mode 100644 index 00000000..8440373e --- /dev/null +++ b/lang/python/doc/src/what-was-new.org @@ -0,0 +1,106 @@ +# -*- mode: org -*- +#+TITLE: What Was New in the GPGME Python Bindings and Documentation +#+AUTHOR: Ben McGinnes +#+LATEX_COMPILER: xelatex +#+LATEX_CLASS: article +#+LATEX_CLASS_OPTIONS: [12pt] +#+LATEX_HEADER: \usepackage{xltxtra} +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} +#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} +#+LATEX_HEADER: \author{Ben McGinnes } + + +* What Was New + :PROPERTIES: + :CUSTOM_ID: new-stuff + :END: + +| Version: | 0.0.1 | +| GPGME Version: | 1.13.0 | +| Author: | Ben McGinnes | +| Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | +| Language: | Australian English, British English | +| xml:lang: | en-AU, en-GB, en | + +The following are all the past /What's New/ sections for the Python +Bindings HOWTO and other documentation. + + +** What Was New in GPGME 1·12·0 + :PROPERTIES: + :CUSTOM_ID: gpgme-1-12-0 + :END: + +The most obviously new point for those reading this guide is this +section on other new things, but that’s hardly important. Not given +all the other things which spurred the need for adding this section +and its subsections. + + +*** New in GPGME 1·12·0 + :PROPERTIES: + :CUSTOM_ID: new-stuff-1-12-0 + :END: + +There have been quite a number of additions to GPGME and the Python +bindings to it since the last release of GPGME with versions 1.11.0 +and 1.11.1 in April, 2018. + +The bullet points of new additiions are: + +- an expanded section on [[file:gpgme-python-howto#installation][installing]] and [[file:gpgme-python-howto#snafu][troubleshooting]] the Python + bindings. +- The release of Python 3.7.0; which appears to be working just fine + with our bindings, in spite of intermittent reports of problems for + many other Python projects with that new release. +- Python 3.7 has been moved to the head of the specified python + versions list in the build process. +- In order to fix some other issues, there are certain underlying + functions which are more exposed through the [[file:gpgme-python-howto#howto-get-context][gpg.Context()]], but + ongoing documentation ought to clarify that or otherwise provide the + best means of using the bindings. Some additions to =gpg.core= and + the =Context()=, however, were intended (see below). +- Continuing work in identifying and confirming the cause of + oft-reported [[file:gpgme-python-howto#snafu-runtime-not-funtime][problems installing the Python bindings on Windows]]. +- GSOC: Google's Surreptitiously Ordered Conscription ... erm ... oh, + right; Google's Summer of Code. Though there were two hopeful + candidates this year; only one ended up involved with the GnuPG + Project directly, the other concentrated on an unrelated third party + project with closer ties to one of the GNU/Linux distributions than + to the GnuPG Project. Thus the Python bindings benefited from GSOC + participant Jacob Adams, who added the key_import function; building + on prior work by Tobias Mueller. +- Several new methods functions were added to the gpg.Context(), + including: [[file:gpgme-python-howto#howto-import-key][key_import]], [[file:gpgme-python-howto#howto-export-key][key_export]], [[file:gpgme-python-howto#howto-export-public-key][key_export_minimal]] and + [[file:gpgme-python-howto#howto-export-secret-key][key_export_secret]]. +- Importing and exporting examples include versions integrated with + Marcel Fest's recently released [[https://github.com/Selfnet/hkp4py][HKP for Python]] module. Some + [[file:gpgme-python-howto#hkp4py][additional notes on this module]] are included at the end of the HOWTO. +- Instructions for dealing with semi-walled garden implementations + like ProtonMail are also included. This is intended to make things + a little easier when communicating with users of ProtonMail's + services and should not be construed as an endorsement of said + service. The GnuPG Project neither favours, nor disfavours + ProtonMail and the majority of this deals with interacting with the + ProtonMail keyserver. +- Semi-formalised the location where [[file:gpgme-python-howto#draft-editions][draft versions]] of this HOWTO may + periodically be accessible. This is both for the reference of + others and testing the publishing of the document itself. Renamed + this file at around the same time. +- The Texinfo documentation build configuration has been replicated + from the parent project in order to make to maintain consistency + with that project (and actually ship with each release). +- a reStructuredText (=.rst=) version is also generated for Python + developers more used to and comfortable with that format as it is + the standard Python documentation format and Python developers may + wish to use it with Sphinx. Please note that there has been no + testing of the reStructuredText version with Sphinx at all. The + reST file was generated by the simple expedient of using [[https://pandoc.org/][Pandoc]]. +- Added a new section for [[file:gpgme-python-howto#advanced-use][advanced or experimental use]]. +- Began the advanced use cases with [[file:gpgme-python-howto#cython][a section]] on using the module with + [[https://cython.org/][Cython]]. +- Added a number of new scripts to the =example/howto/= directory; + some of which may be in advance of their planned sections of the + HOWTO (and some are just there because it seemed like a good idea at + the time). +- Cleaned up a lot of things under the hood. -- cgit v1.2.3