diff options
Diffstat (limited to 'lang/python/docs')
| -rw-r--r-- | lang/python/docs/GPGMEpythonHOWTOen.org | 1370 | ||||
| -rw-r--r-- | lang/python/docs/TODO.org | 77 |
2 files changed, 1446 insertions, 1 deletions
diff --git a/lang/python/docs/GPGMEpythonHOWTOen.org b/lang/python/docs/GPGMEpythonHOWTOen.org new file mode 100644 index 00000000..1e8dd9fa --- /dev/null +++ b/lang/python/docs/GPGMEpythonHOWTOen.org @@ -0,0 +1,1370 @@ +#+TITLE: GNU Privacy Guard (GnuPG) Made Easy Python Bindings HOWTO (English) +#+LATEX_COMPILER: xelatex +#+LATEX_CLASS: article +#+LATEX_CLASS_OPTIONS: [12pt] +#+LATEX_HEADER: \usepackage{xltxtra} +#+LATEX_HEADER: \usepackage[margin=1in]{geometry} +#+LATEX_HEADER: \setmainfont[Ligatures={Common}]{Times New Roman} +#+LATEX_HEADER: \author{Ben McGinnes <[email protected]>} + + +* Introduction + :PROPERTIES: + :CUSTOM_ID: intro + :END: + + | Version: | 0.1.0 | + | Author: | Ben McGinnes <[email protected]> | + | Author GPG Key: | DB4724E6FA4286C92B4E55C4321E4E2373590E5D | + | Language: | Australian English, British English | + | xml:lang: | en-AU, en-GB, en | + + This document provides basic instruction in how to use the GPGME + Python bindings to programmatically leverage the GPGME library. + + +** Python 2 versus Python 3 + :PROPERTIES: + :CUSTOM_ID: py2-vs-py3 + :END: + + Though the GPGME Python bindings themselves provide support for + both Python 2 and 3, the focus is unequivocally on Python 3 and + specifically from Python 3.4 and above. As a consequence all the + examples and instructions in this guide use Python 3 code. + + Much of it will work with Python 2, but much of it also deals with + Python 3 byte literals, particularly when reading and writing data. + Developers concentrating on Python 2.7, and possibly even 2.6, will + need to make the appropriate modifications to support the older + string and unicode types as opposed to bytes. + + There are multiple reasons for concentrating on Python 3; some of + which relate to the immediate integration of these bindings, some + of which relate to longer term plans for both GPGME and the python + bindings and some of which relate to the impending EOL period for + Python 2.7. Essentially, though, there is little value in tying + the bindings to a version of the language which is a dead end and + the advantages offered by Python 3 over Python 2 make handling the + data types with which GPGME deals considerably easier. + + +** Examples + :PROPERTIES: + :CUSTOM_ID: howto-python3-examples + :END: + + All of the examples found in this document can be found as Python 3 + scripts in the =lang/python/examples/howto= directory. + + +* GPGME Concepts + :PROPERTIES: + :CUSTOM_ID: gpgme-concepts + :END: + + +** A C API + :PROPERTIES: + :CUSTOM_ID: gpgme-c-api + :END: + + Unlike many modern APIs with which programmers will be more + familiar with these days, the GPGME API is a C API. The API is + intended for use by C coders who would be able to access its + features by including the =gpgme.h= header file with their own C + source code and then access its functions just as they would any + other C headers. + + This is a very effective method of gaining complete access to the + API and in the most efficient manner possible. It does, however, + have the drawback that it cannot be directly used by other + languages without some means of providing an interface to those + languages. This is where the need for bindings in various + languages stems. + + +** Python bindings + :PROPERTIES: + :CUSTOM_ID: gpgme-python-bindings + :END: + + The Python bindings for GPGME provide a higher level means of + accessing the complete feature set of GPGME itself. It also + provides a more pythonic means of calling these API functions. + + The bindings are generated dynamically with SWIG and the copy of + =gpgme.h= generated when GPGME is compiled. + + This means that a version of the Python bindings is fundamentally + tied to the exact same version of GPGME used to generate that copy + of =gpgme.h=. + + +** Difference between the Python bindings and other GnuPG Python packages + :PROPERTIES: + :CUSTOM_ID: gpgme-python-bindings-diffs + :END: + + There have been numerous attempts to add GnuPG support to Python + over the years. Some of the most well known are listed here, along + with what differentiates them. + + +*** The python-gnupg package maintained by Vinay Sajip + :PROPERTIES: + :CUSTOM_ID: diffs-python-gnupg + :END: + + This is arguably the most popular means of integrating GPG with + Python. The package utilises the =subprocess= module to implement + wrappers for the =gpg= and =gpg2= executables normally invoked on + the command line (=gpg.exe= and =gpg2.exe= on Windows). + + The popularity of this package stemmed from its ease of use and + capability in providing the most commonly required features. + + Unfortunately it has been beset by a number of security issues in + the past; most of which stemmed from using unsafe methods of + accessing the command line via the =subprocess= calls. While some + effort has been made over the last two to three years (as of 2018) + to mitigate this, particularly by no longer providing shell access + through those subprocess calls, the wrapper is still somewhat + limited in the scope of its GnuPG features coverage. + + The python-gnupg package is available under the MIT license. + + +*** The gnupg package created and maintained by Isis Lovecruft + :PROPERTIES: + :CUSTOM_ID: diffs-isis-gnupg + :END: + + In 2015 Isis Lovecruft from the Tor Project forked and then + re-implemented the python-gnupg package as just gnupg. This new + package also relied on subprocess to call the =gpg= or =gpg2= + binaries, but did so somewhat more securely. + + The naming and version numbering selected for this package, + however, resulted in conflicts with the original python-gnupg and + since its functions were called in a different manner to + python-gnupg, the release of this package also resulted in a great + deal of consternation when people installed what they thought was + an upgrade that subsequently broke the code relying on it. + + The gnupg package is available under the GNU General Public + License version 3.0 (or any later version). + + +*** The PyME package maintained by Martin Albrecht + :PROPERTIES: + :CUSTOM_ID: diffs-pyme + :END: + + This package is the origin of these bindings, though they are + somewhat different now. For details of when and how the PyME + package was folded back into GPGME itself see the /Short History/ + document[fn:1] in this Python bindings =docs= directory.[fn:2] + + The PyME package was first released in 2002 and was also the first + attempt to implement a low level binding to GPGME. In doing so it + provided access to considerably more functionality than either the + =python-gnupg= or =gnupg= packages. + + The PyME package is only available for Python 2.6 and 2.7. + + Porting the PyME package to Python 3.4 in 2015 is what resulted in + it being folded into the GPGME project and the current bindings + are the end result of that effort. + + The PyME package is available under the same dual licensing as + GPGME itself: the GNU General Public License version 2.0 (or any + later version) and the GNU Lesser General Public License version + 2.1 (or any later version). + + +* GPGME Python bindings installation + :PROPERTIES: + :CUSTOM_ID: gpgme-python-install + :END: + + +** No PyPI + :PROPERTIES: + :CUSTOM_ID: do-not-use-pypi + :END: + + Most third-party Python packages and modules are available and + distributed through the Python Package Installer, known as PyPI. + + Due to the nature of what these bindings are and how they work, it + is infeasible to install the GPGME Python bindings in the same way. + + This is because the bindings use SWIG to dynamically generate C + bindings against =gpgme.h= and =gpgme.h= is generated from + =gpgme.h.in= at compile time when GPGME is built from source. Thus + to include a package in PyPI which actually built correctly would + require either statically built libraries for every architecture + bundled with it or a full implementation of C for each + architecture. + + +** Requirements + :PROPERTIES: + :CUSTOM_ID: gpgme-python-requirements + :END: + + The GPGME Python bindings only have three requirements: + + 1. A suitable version of Python 2 or Python 3. With Python 2 that + means Python 2.7 and with Python 3 that means Python 3.4 or + higher. + 2. SWIG. + 3. GPGME itself. Which also means that all of GPGME's dependencies + must be installed too. + + +** Installation + :PROPERTIES: + :CUSTOM_ID: installation + :END: + + Installing the Python bindings is effectively achieved by compiling + and installing GPGME itself. + + Once SWIG is installed with Python and all the dependencies for + GPGME are installed you only need to confirm that the version(s) of + Python you want the bindings installed for are in your =$PATH=. + + By default GPGME will attempt to install the bindings for the most + recent or highest version number of Python 2 and Python 3 it + detects in =$PATH=. It specifically checks for the =python= and + =python3= executables first and then checks for specific version + numbers. + + For Python 2 it checks for these executables in this order: + =python=, =python2= and =python2.7=. + + For Python 3 it checks for these executables in this order: + =python3=, =python3.6=, =python3.5= and =python3.4=. + + +*** Installing GPGME + :PROPERTIES: + :CUSTOM_ID: install-gpgme + :END: + + See the GPGME =README= file for details of how to install GPGME from + source. + + +* Fundamentals + :PROPERTIES: + :CUSTOM_ID: howto-fund-a-mental + :END: + + Before we can get to the fun stuff, there are a few matters + regarding GPGME's design which hold true whether you're dealing with + the C code directly or these Python bindings. + + +** No REST + :PROPERTIES: + :CUSTOM_ID: no-rest-for-the-wicked + :END: + + The first part of which is or will be fairly blatantly obvious upon + viewing the first example, but it's worth reiterating anyway. That + being that this API is /*not*/ a REST API. Nor indeed could it + ever be one. + + Most, if not all, Python programmers (and not just Python + programmers) know how easy it is to work with a RESTful API. In + fact they've become so popular that many other APIs attempt to + emulate REST-like behaviour as much as they are able. Right down + to the use of JSON formatted output to facilitate the use of their + API without having to retrain developers. + + This API does not do that. It would not be able to do that and + also provide access to the entire C API on which it's built. It + does, however, provide a very pythonic interface on top of the + direct bindings and it's this pythonic layer with which this HOWTO + deals with. + + +** Context + :PROPERTIES: + :CUSTOM_ID: howto-get-context + :END: + + One of the reasons which prevents this API from being RESTful is + that most operations require more than one instruction to the API + to perform the task. Sure, there are certain functions which can + be performed simultaneously, particularly if the result known or + strongly anticipated (e.g. selecting and encrypting to a key known + to be in the public keybox). + + There are many more, however, which cannot be manipulated so + readily: they must be performed in a specific sequence and the + result of one operation has a direct bearing on the outcome of + subsequent operations. Not merely by generating an error either. + + When dealing with this type of persistent state on the web, full of + both the RESTful and REST-like, it's most commonly referred to as a + session. In GPGME, however, it is called a context and every + operation type has one. + + +* Working with keys + :PROPERTIES: + :CUSTOM_ID: howto-keys + :END: + + +** Key selection + :PROPERTIES: + :CUSTOM_ID: howto-keys-selection + :END: + + Selecting keys to encrypt to or to sign with will be a common + occurrence when working with GPGMe and the means available for + doing so are quite simple. + + They do depend on utilising a Context; however once the data is + recorded in another variable, that Context does not need to be the + same one which subsequent operations are performed. + + The easiest way to select a specific key is by searching for that + key's key ID or fingerprint, preferably the full fingerprint + without any spaces in it. A long key ID will probably be okay, but + is not advised and short key IDs are already a problem with some + being generated to match specific patterns. It does not matter + whether the pattern is upper or lower case. + + So this is the best method: + + #+begin_src python + import gpg + + k = gpg.Context().keylist(pattern="258E88DCBD3CD44D8E7AB43F6ECB6AF0DEADBEEF") + keys = list(k) + #+end_src + + This is passable and very likely to be common: + + #+begin_src python + import gpg + + k = gpg.Context().keylist(pattern="0x6ECB6AF0DEADBEEF") + keys = list(k) + #+end_src + + And this is a really bad idea: + + #+begin_src python + import gpg + + k = gpg.Context().keylist(pattern="0xDEADBEEF") + keys = list(k) + #+end_src + + Alternatively it may be that the intention is to create a list of + keys which all match a particular search string. For instance all + the addresses at a particular domain, like this: + + #+begin_src python + import gpg + + ncsc = gpg.Context().keylist(pattern="ncsc.mil") + nsa = list(ncsc) + #+end_src + + +*** Counting keys + :PROPERTIES: + :CUSTOM_ID: howto-keys-counting + :END: + + Counting the number of keys in your public keybox (=pubring.kbx=), + the format which has superseded the old keyring format + (=pubring.gpg= and =secring.gpg=), or the number of secret keys is + a very simple task. + + #+begin_src python + import gpg + + c = gpg.Context() + seckeys = c.keylist(pattern=None, secret=True) + pubkeys = c.keylist(pattern=None, secret=False) + + seclist = list(seckeys) + secnum = len(seclist) + + publist = list(pubkeys) + pubnum = len(publist) + + print(""" + Number of secret keys: {0} + Number of public keys: {1} + """.format(secnum, pubnum)) + #+end_src + + +** Get key + :PROPERTIES: + :CUSTOM_ID: howto-get-key + :END: + + An alternative method of getting a single key via its fingerprint + is available directly within a Context with =Context().get_key=. + This is the preferred method of selecting a key in order to modify + it, sign or certify it and for obtaining relevant data about a + single key as a part of other functions; when verifying a signature + made by that key, for instance. + + By default this method will select public keys, but it can select + secret keys as well. + + This first example demonstrates selecting the current key of Werner + Koch, which is due to expire at the end of 2018: + + #+begin_src python + import gpg + + fingerprint = "80615870F5BAD690333686D0F2AD85AC1E42B367" + key = gpg.Context().get_key(fingerprint) + #+end_src + + Whereas this example demonstrates selecting the author's current + key with the =secret= key word argument set to =True=: + + #+begin_src python + import gpg + + fingerprint = "DB4724E6FA4286C92B4E55C4321E4E2373590E5D" + key = gpg.Context().get_key(fingerprint, secret=True) + #+end_src + + It is, of course, quite possible to select expired, disabled and + revoked keys with this function, but only to effectively display + information about those keys. + + It is also possible to use both unicode or string literals and byte + literals with the fingerprint when getting a key in this way. + + +* Basic Functions + :PROPERTIES: + :CUSTOM_ID: howto-the-basics + :END: + + The most frequently called features of any cryptographic library + will be the most fundamental tasks for encryption software. In this + section we will look at how to programmatically encrypt data, + decrypt it, sign it and verify signatures. + + +** Encryption + :PROPERTIES: + :CUSTOM_ID: howto-basic-encryption + :END: + + Encrypting is very straight forward. In the first example below + the message, =text=, is encrypted to a single recipient's key. In + the second example the message will be encrypted to multiple + recipients. + + +*** Encrypting to one key + :PROPERTIES: + :CUSTOM_ID: howto-basic-encryption-single + :END: + + Once the the Context is set the main issues with encrypting data + is essentially reduced to key selection and the keyword arguments + specified in the =gpg.Context().encrypt()= method. + + Those keyword arguments are: =recipients=, a list of keys + encrypted to (covered in greater detail in the following section); + =sign=, whether or not to sign the plaintext data, see subsequent + sections on signing and verifying signatures below (defaults to + =True=); =sink=, to write results or partial results to a secure + sink instead of returning it (defaults to =None=); =passphrase=, + only used when utilising symmetric encryption (defaults to + =None=); =always_trust=, used to override the trust model settings + for recipient keys (defaults to =False=); =add_encrypt_to=, + utilises any preconfigured =encrypt-to= or =default-key= settings + in the user's =gpg.conf= file (defaults to =False=); =prepare=, + prepare for encryption (defaults to =False=); =expect_sign=, + prepare for signing (defaults to =False=); =compress=, compresses + the plaintext prior to encryption (defaults to =True=). + + #+begin_src python + import gpg + + a_key = "0x12345678DEADBEEF" + text = b"""Some text to test with. + + Since the text in this case must be bytes, it is most likely that + the input form will be a separate file which is opened with "rb" + as this is the simplest method of obtaining the correct data + format. + """ + + c = gpg.Context(armor=True) + rkey = list(c.keylist(pattern=a_key, secret=False)) + ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, sign=False) + + with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) + #+end_src + + Though this is even more likely to be used like this; with the + plaintext input read from a file, the recipient keys used for + encryption regardless of key trust status and the encrypted output + also encrypted to any preconfigured keys set in the =gpg.conf= + file: + + #+begin_src python + import gpg + + a_key = "0x12345678DEADBEEF" + + with open("secret_plans.txt", "rb") as afile: + text = afile.read() + + c = gpg.Context(armor=True) + rkey = list(c.keylist(pattern=a_key, secret=False)) + ciphertext, result, sign_result = c.encrypt(text, recipients=rkey, + sign=True, always_trust=True, + add_encrypt_to=True) + + with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) + #+end_src + + If the =recipients= paramater is empty then the plaintext is + encrypted symmetrically. If no =passphrase= is supplied as a + parameter or via a callback registered with the =Context()= then + an out-of-band prompt for the passphrase via pinentry will be + invoked. + + +*** Encrypting to multiple keys + :PROPERTIES: + :CUSTOM_ID: howto-basic-encryption-multiple + :END: + + Encrypting to multiple keys essentially just expands upon the key + selection process and the recipients from the previous examples. + + The following example encrypts a message (=text=) to everyone with + an email address on the =gnupg.org= domain,[fn:3] but does /not/ encrypt + to a default key or other key which is configured to normally + encrypt to. + + #+begin_src python + import gpg + + text = b"""Oh look, another test message. + + The same rules apply as with the previous example and more likely + than not, the message will actually be drawn from reading the + contents of a file or, maybe, from entering data at an input() + prompt. + + Since the text in this case must be bytes, it is most likely that + the input form will be a separate file which is opened with "rb" + as this is the simplest method of obtaining the correct data + format. + """ + + c = gpg.Context(armor=True) + rpattern = list(c.keylist(pattern="@gnupg.org", secret=False)) + logrus = [] + + for i in range(len(rpattern)): + if rpattern[i].can_encrypt == 1: + logrus.append(rpattern[i]) + + ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, sign=False, + always_trust=True) + + with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) + #+end_src + + All it would take to change the above example to sign the message + and also encrypt the message to any configured default keys would + be to change the =c.encrypt= line to this: + + #+begin_src python + ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, + always_trust=True, + add_encrypt_to=True) + #+end_src + + The only keyword arguments requiring modification are those for + which the default values are changing. The default value of + =sign= is =True=, the default of =always_trust= is =False=, the + default of =add_encrypt_to= is =False=. + + If =always_trust= is not set to =True= and any of the recipient + keys are not trusted (e.g. not signed or locally signed) then the + encryption will raise an error. It is possible to mitigate this + somewhat with something more like this: + + #+begin_src python + import gpg + + with open("secret_plans.txt.asc", "rb") as afile: + text = afile.read() + + c = gpg.Context(armor=True) + rpattern = list(c.keylist(pattern="@gnupg.org", secret=False)) + logrus = [] + + for i in range(len(rpattern)): + if rpattern[i].can_encrypt == 1: + logrus.append(rpattern[i]) + + try: + ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, add_encrypt_to=True) + except gpg.errors.InvalidRecipients as e: + for i in range(len(e.recipients)): + for n in range(len(logrus)): + if logrus[n].fpr == e.recipients[i].fpr: + logrus.remove(logrus[n]) + else: + pass + try: + ciphertext, result, sign_result = c.encrypt(text, recipients=logrus, add_encrypt_to=True) + except: + pass + + with open("secret_plans.txt.asc", "wb") as afile: + afile.write(ciphertext) + #+end_src + + This will attempt to encrypt to all the keys searched for, then + remove invalid recipients if it fails and try again. + + +** Decryption + :PROPERTIES: + :CUSTOM_ID: howto-basic-decryption + :END: + + Decrypting something encrypted to a key in one's secret keyring is + fairly straight forward. + + In this example code, however, preconfiguring either + =gpg.Context()= or =gpg.core.Context()= as =c= is unnecessary + because there is no need to modify the Context prior to conducting + the decryption and since the Context is only used once, setting it + to =c= simply adds lines for no gain. + + #+begin_src python + import gpg + + ciphertext = input("Enter path and filename of encrypted file: ") + newfile = input("Enter path and filename of file to save decrypted data to: ") + with open(ciphertext, "rb") as cfile: + plaintext, result, verify_result = gpg.Context().decrypt(cfile) + with open(newfile, "wb") as nfile: + nfile.write(plaintext) + #+end_src + + The data available in plaintext in this example is the decrypted + content as a byte object in =plaintext[0]=, the recipient key IDs + and algorithms in =plaintext[1]= and the results of verifying any + signatures of the data in =plaintext[0]=. + + +** Signing text and files + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing + :END: + + The following sections demonstrate how to specify keys to sign with. + + +*** Signing key selection + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-signers + :END: + + By default GPGME and the Python bindings will use the default key + configured for the user invoking the GPGME API. If there is no + default key specified and there is more than one secret key + available it may be necessary to specify the key or keys with + which to sign messages and files. + + #+begin_src python + import gpg + + logrus = input("Enter the email address or string to match signing keys to: ") + hancock = gpg.Context().keylist(pattern=logrus, secret=True) + sig_src = list(hancock) + #+end_src + + The signing examples in the following sections include the + explicitly designated =signers= parameter in two of the five + examples; once where the resulting signature would be ASCII + armoured and once where it would not be armoured. + + While it would be possible to enter a key ID or fingerprint here + to match a specific key, it is not possible to enter two + fingerprints and match two keys since the patten expects a string, + bytes or None and not a list. A string with two fingerprints + won't match any single key. + + +*** Normal or default signing messages or files + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-normal + :END: + + The normal or default signing process is essentially the same as + is most often invoked when also encrypting a message or file. So + when the encryption component is not utilised, the result is to + produce an encoded and signed output which may or may not be ASCII + armoured and which may or may not also be compressed. + + By default compression will be used unless GnuPG detects that the + plaintext is already compressed. ASCII armouring will be + determined according to the value of =gpg.Context().armor=. + + The compression algorithm is selected in much the same way as the + symmetric encryption algorithm or the hash digest algorithm is + when multiple keys are involved; from the preferences saved into + the key itself or by comparison with the preferences with all + other keys involved. + + #+begin_src python + import gpg + + text0 = """Declaration of ... something. + + """ + text = text0.encode() + + c = gpg.Context(armor=True, signers=sig_src) + signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL) + + with open("/path/to/statement.txt.asc", "w") as afile: + afile.write(signed_data.decode()) + #+end_src + + Though everything in this example is accurate, it is more likely + that reading the input data from another file and writing the + result to a new file will be performed more like the way it is done + in the next example. Even if the output format is ASCII armoured. + + #+begin_src python + import gpg + + with open("/path/to/statement.txt", "rb") as tfile: + text = tfile.read() + + c = gpg.Context() + signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.NORMAL) + + with open("/path/to/statement.txt.sig", "wb") as afile: + afile.write(signed_data) + #+end_src + + +*** Detached signing messages and files + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-detached + :END: + + Detached signatures will often be needed in programmatic uses of + GPGME, either for signing files (e.g. tarballs of code releases) + or as a component of message signing (e.g. PGP/MIME encoded + email). + + #+begin_src python + import gpg + + text0 = """Declaration of ... something. + + """ + text = text0.encode() + + c = gpg.Context(armor=True) + signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH) + + with open("/path/to/statement.txt.asc", "w") as afile: + afile.write(signed_data.decode()) + #+end_src + + As with normal signatures, detached signatures are best handled as + byte literals, even when the output is ASCII armoured. + + #+begin_src python + import gpg + + with open("/path/to/statement.txt", "rb") as tfile: + text = tfile.read() + + c = gpg.Context(signers=sig_src) + signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.DETACH) + + with open("/path/to/statement.txt.sig", "wb") as afile: + afile.write(signed_data) + #+end_src + + +*** Clearsigning messages or text + :PROPERTIES: + :CUSTOM_ID: howto-basic-signing-clear + :END: + + Though PGP/in-line messages are no longer encouraged in favour of + PGP/MIME, there is still sometimes value in utilising in-line + signatures. This is where clear-signed messages or text is of + value. + + #+begin_src python + import gpg + + text0 = """Declaration of ... something. + + """ + text = text0.encode() + + c = gpg.Context() + signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR) + + with open("/path/to/statement.txt.asc", "w") as afile: + afile.write(signed_data.decode()) + #+end_src + + In spite of the appearance of a clear-signed message, the data + handled by GPGME in signing it must still be byte literals. + + #+begin_src python + import gpg + + with open("/path/to/statement.txt", "rb") as tfile: + text = tfile.read() + + c = gpg.Context() + signed_data, result = c.sign(text, mode=gpg.constants.sig.mode.CLEAR) + + with open("/path/to/statement.txt.asc", "wb") as afile: + afile.write(signed_data) + #+end_src + + +** Signature verification + :PROPERTIES: + :CUSTOM_ID: howto-basic-verification + :END: + + Essentially there are two principal methods of verification of a + signature. The first of these is for use with the normal or + default signing method and for clear-signed messages. The second is + for use with files and data with detached signatures. + + The following example is intended for use with the default signing + method where the file was not ASCII armoured: + + #+begin_src python + import gpg + import time + + filename = "statement.txt" + gpg_file = "statement.txt.gpg" + + c = gpg.Context() + + try: + data, result = c.verify(open(gpg_file)) + verified = True + except gpg.errors.BadSignatures as e: + verified = False + print(e) + + if verified is True: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: + {0} + with key {1} + made at {2} + """.format(c.get_key(sign.fpr).uids[0].uid, + sign.fpr, time.ctime(sign.timestamp))) + else: + pass + #+end_src + + Whereas this next example, which is almost identical would work + with normal ASCII armoured files and with clear-signed files: + + #+begin_src python + import gpg + import time + + filename = "statement.txt" + asc_file = "statement.txt.asc" + + c = gpg.Context() + + try: + data, result = c.verify(open(asc_file)) + verified = True + except gpg.errors.BadSignatures as e: + verified = False + print(e) + + if verified is True: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: + {0} + with key {1} + made at {2} + """.format(c.get_key(sign.fpr).uids[0].uid, + sign.fpr, time.ctime(sign.timestamp))) + else: + pass + #+end_src + + In both of the previous examples it is also possible to compare the + original data that was signed against the signed data in =data= to + see if it matches with something like this: + + #+begin_src python + with open(filename, "rb") as afile: + text = afile.read() + + if text == data: + print("Good signature.") + else: + pass + #+end_src + + The following two examples, however, deal with detached signatures. + With his method of verification the data that was signed does not + get returned since it is already being explicitly referenced in the + first argument of =c.verify=. So =data= is =None= and only the + information in =result= is available. + + #+begin_src python + import gpg + import time + + filename = "statement.txt" + sig_file = "statement.txt.sig" + + c = gpg.Context() + + try: + data, result = c.verify(open(filename), open(sig_file)) + verified = True + except gpg.errors.BadSignatures as e: + verified = False + print(e) + + if verified is True: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: + {0} + with key {1} + made at {2} + """.format(c.get_key(sign.fpr).uids[0].uid, + sign.fpr, time.ctime(sign.timestamp))) + else: + pass + #+end_src + + #+begin_src python + import gpg + import time + + filename = "statement.txt" + asc_file = "statement.txt.asc" + + c = gpg.Context() + + try: + data, result = c.verify(open(filename), open(asc_file)) + verified = True + except gpg.errors.BadSignatures as e: + verified = False + print(e) + + if verified is not None: + for i in range(len(result.signatures)): + sign = result.signatures[i] + print("""Good signature from: + {0} + with key {1} + made at {2} + """.format(c.get_key(sign.fpr).uids[0].uid, + sign.fpr, time.ctime(sign.timestamp))) + else: + pass + #+end_src + + +* Creating keys and subkeys + :PROPERTIES: + :CUSTOM_ID: key-generation + :END: + + The one thing, aside from GnuPG itself, that GPGME depends on, of + course, is the keys themselves. So it is necessary to be able to + generate them and modify them by adding subkeys, revoking or + disabling them, sometimes deleting them and doing the same for user + IDs. + + In the following examples a key will be created for the world's + greatest secret agent, Danger Mouse. Since Danger Mouse is a secret + agent he needs to be able to protect information to =SECRET= level + clearance, so his keys will be 3072-bit keys. + + The pre-configured =gpg.conf= file which sets cipher, digest and + other preferences contains the following configuration parameters: + + #+begin_src conf + expert + allow-freeform-uid + allow-secret-key-import + trust-model tofu+pgp + tofu-default-policy unknown + enable-large-rsa + enable-dsa2 + # cert-digest-algo SHA256 + cert-digest-algo SHA512 + default-preference-list TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1 ZLIB BZIP2 ZIP Uncompressed + personal-cipher-preferences TWOFISH CAMELLIA256 AES256 CAMELLIA192 AES192 CAMELLIA128 AES BLOWFISH IDEA CAST5 3DES + personal-digest-preferences SHA512 SHA384 SHA256 SHA224 RIPEMD160 SHA1 + personal-compress-preferences ZLIB BZIP2 ZIP Uncompressed + #+end_src + + +** Primary key + :PROPERTIES: + :CUSTOM_ID: keygen-primary + :END: + + Generating a primary key uses the =create_key= method in a Context. + It contains multiple arguments and keyword arguments, including: + =userid=, =algorithm=, =expires_in=, =expires=, =sign=, =encrypt=, + =certify=, =authenticate=, =passphrase= and =force=. The defaults + for all of those except =userid=, =algorithm=, =expires_in=, + =expires= and =passphrase= is =False=. The defaults for + =algorithm= and =passphrase= is =None=. The default for + =expires_in= is =0=. The default for =expires= is =True=. There + is no default for =userid=. + + If =passphrase= is left as =None= then the key will not be + generated with a passphrase, if =passphrase= is set to a string + then that will be the passphrase and if =passphrase= is set to + =True= then gpg-agent will launch pinentry to prompt for a + passphrase. For the sake of convenience, these examples will keep + =passphrase= set to =None=. + + #+begin_src python + import gpg + + c = gpg.Context() + + c.home_dir = "~/.gnupg-dm" + userid = "Danger Mouse <[email protected]>" + + dmkey = c.create_key(userid, algorithm = "rsa3072", expires_in = 31536000, + sign = True, certify = True) + #+end_src + + One thing to note here is the use of setting the =c.home_dir= + parameter. This enables generating the key or keys in a different + location. In this case to keep the new key data created for this + example in a separate location rather than adding it to existing + and active key store data. As with the default directory, + =~/.gnupg=, any temporary or separate directory needs the + permissions set to only permit access by the directory owner. On + posix systems this means setting the directory permissions to 700. + + The successful generation of the key can be confirmed via the + returned =GenkeyResult= object, which includes the following data: + + #+begin_src python + print(""" + Fingerprint: {0} + Primary Key: {1} + Public Key: {2} + Secret Key: {3} + Sub Key: {4} + User IDs: {5} + """.format(dmkey.fpr, dmkey.primary, dmkey.pubkey, dmkey.seckey, dmkey.sub, + dmkey.uid)) + #+end_src + + Alternatively the information can be confirmed using the command + line program: + + #+begin_src shell + bash-4.4$ gpg --homedir ~/.gnupg-dm -K + ~/.gnupg-dm/pubring.kbx + ---------------------- + sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] + 177B7C25DB99745EE2EE13ED026D2F19E99E63AA + uid [ultimate] Danger Mouse <[email protected]> + + bash-4.4$ + #+end_src + + As with generating keys manually, to preconfigure expanded + preferences for the cipher, digest and compression algorithms, the + =gpg.conf= file must contain those details in the home directory in + which the new key is being generated. I used a cut down version of + my own =gpg.conf= file in order to be able to generate this: + + #+begin_src shell + bash-4.4$ gpg --homedir ~/.gnupg-dm --edit-key 177B7C25DB99745EE2EE13ED026D2F19E99E63AA showpref quit + Secret key is available. + + sec rsa3072/026D2F19E99E63AA + created: 2018-03-15 expires: 2019-03-15 usage: SC + trust: ultimate validity: ultimate + [ultimate] (1). Danger Mouse <[email protected]> + + [ultimate] (1). Danger Mouse <[email protected]> + Cipher: TWOFISH, CAMELLIA256, AES256, CAMELLIA192, AES192, CAMELLIA128, AES, BLOWFISH, IDEA, CAST5, 3DES + Digest: SHA512, SHA384, SHA256, SHA224, RIPEMD160, SHA1 + Compression: ZLIB, BZIP2, ZIP, Uncompressed + Features: MDC, Keyserver no-modify + + bash-4.4$ + #+end_src + + +** Subkeys + :PROPERTIES: + :CUSTOM_ID: keygen-subkeys + :END: + + Adding subkeys to a primary key is fairly similar to creating the + primary key with the =create_subkey= method. Most of the arguments + are the same, but not quite all. Instead of the =userid= argument + there is now a =key= argument for selecting which primary key to + add the subkey to. + + In the following example an encryption subkey will be added to the + primary key. Since Danger Mouse is a security conscious secret + agent, this subkey will only be valid for about six months, half + the length of the primary key. + + #+begin_src python + import gpg + + c = gpg.Context() + c.home_dir = "~/.gnupg-dm" + + key = c.get_key(dmkey.fpr, secret = True) + dmsub = c.create_subkey(key, algorithm = "rsa3072", expires_in = 15768000, + encrypt = True) + #+end_src + + As with the primary key, the results here can be checked with: + + #+begin_src python + print(""" + Fingerprint: {0} + Primary Key: {1} + Public Key: {2} + Secret Key: {3} + Sub Key: {4} + User IDs: {5} + """.format(dmsub.fpr, dmsub.primary, dmsub.pubkey, dmsub.seckey, dmsub.sub, + dmsub.uid)) + #+end_src + + As well as on the command line with: + + #+begin_src shell + bash-4.4$ gpg --homedir ~/.gnupg-dm -K + ~/.gnupg-dm/pubring.kbx + ---------------------- + sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] + 177B7C25DB99745EE2EE13ED026D2F19E99E63AA + uid [ultimate] Danger Mouse <[email protected]> + ssb rsa3072 2018-03-15 [E] [expires: 2018-09-13] + + bash-4.4$ + #+end_src + + +** User IDs + :PROPERTIES: + :CUSTOM_ID: keygen-uids + :END: + + By comparison to creating primary keys and subkeys, adding a new + user ID to an existing key is much simpler. The method used to do + this is =key_add_uid= and the only arguments it takes are for the + =key= and the new =uid=. + + #+begin_src python + import gpg + + c = gpg.Context() + c.home_dir = "~/.gnupg-dm" + + dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" + key = c.get_key(dmfpr, secret = True) + uid = "Danger Mouse <[email protected]>" + + c.key_add_uid(key, uid) + #+end_src + + Unsurprisingly the result of this is: + + #+begin_src shell + bash-4.4$ gpg --homedir ~/.gnupg-dm -K + ~/.gnupg-dm/pubring.kbx + ---------------------- + sec rsa3072 2018-03-15 [SC] [expires: 2019-03-15] + 177B7C25DB99745EE2EE13ED026D2F19E99E63AA + uid [ultimate] Danger Mouse <[email protected]> + uid [ultimate] Danger Mouse <[email protected]> + ssb rsa3072 2018-03-15 [E] [expires: 2018-09-13] + + bash-4.4$ + #+end_src + + +** Key certification + :PROPERTIES: + :CUSTOM_ID: key-sign + :END: + + Since key certification is more frequently referred to as key + signing, the method used to perform this function is =key_sign=. + + The =key_sign= method takes four arguments: =key=, =uids=, + =expires_in= and =local=. The default value of =uids= is =None= + and which results in all user IDs being selected. The default + values of =expires_in= snd =local= is =False=; which result in the + signature never expiring and being able to be exported. + + The =key= is the key being signed rather than the key doing the + signing. To change the key doing the signing refer to the signing + key selection above for signing messages and files. + + If the =uids= value is not =None= then it must either be a string + to match a single user ID or a list of strings to match multiple + user IDs. In this case the matching of those strings must be + precise and it is case sensitive. + + To sign Danger Mouse's key for just the initial user ID with a + signature which will last a little over a month, do this: + + #+begin_src python + import gpg + + c = gpg.Context() + uid = "Danger Mouse <[email protected]>" + + dmfpr = "177B7C25DB99745EE2EE13ED026D2F19E99E63AA" + key = c.get_key(dmfpr, secret = True) + c.key_sign(key, uids = uid, expires_in = 2764800) + #+end_src + + +* Miscellaneous work-arounds + :PROPERTIES: + :CUSTOM_ID: cheats-and-hacks + :END: + + +** Group lines + :PROPERTIES: + :CUSTOM_ID: group-lines + :END: + + There is not yet an easy way to access groups configured in the + gpg.conf file from within GPGME. As a consequence these central + groupings of keys cannot be shared amongst multiple programs, such + as MUAs readily. + + The following code, however, provides a work-around for obtaining + this information in Python. + + #+begin_src python + import subprocess + + lines = subprocess.getoutput("gpgconf --list-options gpg").splitlines() + + for i in range(len(lines)): + if lines[i].startswith("group") is True: + line = lines[i] + else: + pass + + groups = line.split(":")[-1].replace('"', '').split(',') + + group_lines = groups + for i in range(len(group_lines)): + group_lines[i] = group_lines[i].split("=") + + group_lists = group_lines + for i in range(len(group_lists)): + group_lists[i][1] = group_lists[i][1].split() + #+end_src + + The result of that code is that =group_lines= is a list of lists + where =group_lines[i][0]= is the name of the group and + =group_lines[i][1]= is the key IDs of the group as a string. + + The =group_lists= result is very similar in that it is a list of + lists. The first part, =group_lists[i][0]= matches + =group_lines[i][0]= as the name of the group, but + =group_lists[i][1]= is the key IDs of the group as a string. + + +* Copyright and Licensing + :PROPERTIES: + :CUSTOM_ID: copyright-and-license + :END: + + +** Copyright (C) The GnuPG Project, 2018 + :PROPERTIES: + :CUSTOM_ID: copyright + :END: + + Copyright © The GnuPG Project, 2018. + + +** License GPL compatible + :PROPERTIES: + :CUSTOM_ID: license + :END: + + This file is free software; as a special exception the author gives + unlimited permission to copy and/or distribute it, with or without + modifications, as long as this notice is preserved. + + This file is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY, to the extent permitted by law; without even + the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + PURPOSE. + + +* Footnotes + +[fn:1] =Short_History.org= and/or =Short_History.html=. + +[fn:2] The =lang/python/docs/= directory in the GPGME source. + +[fn:3] You probably don't really want to do this. Searching the +keyservers for "gnupg.org" produces over 400 results, the majority of +which aren't actually at the gnupg.org domain, but just included a +comment regarding the project in their key somewhere. diff --git a/lang/python/docs/TODO.org b/lang/python/docs/TODO.org index 9f039d81..add8f4ff 100644 --- a/lang/python/docs/TODO.org +++ b/lang/python/docs/TODO.org @@ -28,13 +28,74 @@ to produce reST versions via Pandoc and DITA XML can be reached through converting to either Markdown or XHTML first. -** TODO Documentation HOWTO + +** STARTED Documentation HOWTO :PROPERTIES: :CUSTOM_ID: todo-docs-howto :END: + - State "STARTED" from "TODO" [2018-03-08 Thu 13:59] \\ + Started yesterday. Write a HOWTO style guide for the current Python bindings. +*** DONE Start python bindings HOWTO + CLOSED: [2018-03-07 Wed 18:14] + :PROPERTIES: + :CUSTOM_ID: howto-start + :END: + + +*** STARTED Include certain specific instructions in the HOWTO + :PROPERTIES: + :CUSTOM_ID: howto-requests + :END: + + Note: moved the S/MIME bits out to their own section of the TODO + list and may be served better by separate HOWTO documentation + anyway. + + - State "STARTED" from "TODO" [2018-03-09 Fri 15:27] + Some functions can be worked out from the handful of examples + available, but many more can't and I've already begun receiving + requests for certain functions to be explained. + + +**** DONE Standard scenarios + CLOSED: [2018-03-19 Mon 12:34] + :PROPERTIES: + :CUSTOM_ID: howto-the-basics + :END: + + - State "DONE" from "STARTED" [2018-03-19 Mon 12:34] \\ + All four of those are done. + - State "STARTED" from "TODO" [2018-03-09 Fri 15:26] \\ + Began with the example code, now to add the text. + What everyone expects: encryption, decryption, signing and verifying. + + +**** STARTED Key control + :PROPERTIES: + :CUSTOM_ID: howto-key-control + :END: + + - State "STARTED" from "TODO" [2018-03-19 Mon 12:35] \\ + Generating keys and subkeys are done, but revocation is still to be done. + Generating keys, adding subkeys, revoking subkeys (and keeping + the cert key), adding and revoking UIDs, signing/certifying keys. + + +**** DONE More key control + CLOSED: [2018-03-19 Mon 12:36] + :PROPERTIES: + :CUSTOM_ID: howto-key-selection + :END: + + - State "DONE" from "TODO" [2018-03-19 Mon 12:36] \\ + Key selection, searching, matching and counting is done. + Selecting keys to encrypt to or manipulate in other ways (e.g. as + with key control or the basics). + + ** TODO Documentation SWIG :PROPERTIES: :CUSTOM_ID: todo-docs-swig @@ -47,6 +108,7 @@ something to be used in conjunction with the existing GPGME documentation which makes it easier for Python developers to use. + ** TODO GUI examples :PROPERTIES: :CUSTOM_ID: todo-gui-examples @@ -56,6 +118,7 @@ to either match or be similar to the old GTK2 examples available with PyME. + ** TODO Replace SWIG :PROPERTIES: :CUSTOM_ID: todo-replace-swig @@ -71,6 +134,7 @@ bindings using a more suitable means of interfacing with the GPGME C source code. + *** TODO Replacement for SWIG :PROPERTIES: :CUSTOM_ID: todo-replace-swig-replacement @@ -80,6 +144,7 @@ the most viable candidate, but some additional testing and checks are yet to be completed. + ** TODO API for an API :PROPERTIES: :CUSTOM_ID: todo-api-squared @@ -94,6 +159,16 @@ available or for which it is too difficult to create proper bindings. + +** TODO S/MIME + :PROPERTIES: + :CUSTOM_ID: s-mime + :END: + + Eventually add some of this, but the OpenPGP details are far more + important at the moment. + + * Project Task Details :PROPERTIES: :CUSTOM_ID: detailed-tasks |