Merge ad33aea18d into 815f880e34

Co-authored-by: Satsuoni <satsuoni@hotmail.com>

.github/workflows/main.yml

@@ -9,8 +9,10 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v2
+
       - name: Package
         run: python3 make_release.py
+
       - name: Upload
         uses: actions/upload-artifact@v2
         with: 
@@ -18,3 +20,46 @@ jobs:
           path: |
             DeDRM_tools_*.zip
             DeDRM_tools.zip
+
+#      - name: Delete old release
+#        uses: cb80/delrel@latest
+#        with:
+#          tag: autorelease
+#          token: ${{ github.token }}
+#
+#      - name: Delete old tag
+#        uses: dev-drprasad/delete-tag-and-release@v1.0
+#        with:
+#          tag_name: autorelease
+#          github_token: ${{ github.token }}
+#          delete_release: true
+#      
+#      - name: Prepare release
+#        run: cp DeDRM_tools.zip DeDRM_alpha_${{ github.sha }}.zip
+#
+#      - name: Auto-release
+#        id: autorelease
+#        uses: softprops/action-gh-release@v1
+#        with: 
+#          tag_name: autorelease
+#          token: ${{ github.token }}
+#          name: Automatic alpha release with latest changes
+#          body: |
+#            This release is automatically generated by Github for each commit. 
+#
+#            This means, every time a change is made to this repo, this release will be updated to contain an untested copy of the plugin at that stage. This will contain the most up-to-date code, but it's not tested at all and may be broken.
+#
+#            Last update based on Git commit ${{ github.sha }}.
+#          prerelease: true
+#          draft: true
+#          files: DeDRM_alpha_${{ github.sha }}.zip
+#
+#      - name: Make release public
+#        uses: irongut/EditRelease@v1.2.0
+#        with:
+#          token: ${{ github.token }}
+#          id: ${{ steps.autorelease.outputs.id }}
+#          draft: false
+#          prerelease: true
+#
+#

CHANGELOG.md

@@ -69,4 +69,26 @@ List of changes since the fork of Apprentice Harper's repository:
 
 ## Fixes on master (not yet released):
 
-- (None)
+- Fix a bug introduced with #48 that breaks DeDRM'ing on Calibre 4 (fixes #101).
+- Fix some more Calibre-6 bugs in the Obok plugin (should fix #114).
+- Fix a bug where invalid Adobe keys could cause the plugin to stop trying subsequent keys (partially fixes #109).
+- Fix DRM removal sometimes resetting the ZIP's internal "external_attr" value on Calibre 5 and newer.
+- Fix tons of PDF decryption issues (hopefully fixes #104 and other PDF-related issues).
+- Small Python 2 / Calibre 4 bugfix for Obok.
+- Removing ancient AlfCrypto machine code libraries, moving all encryption / decryption to Python code.
+- General cleanup and removal of dead code.
+- Fix a bug where ADE account keys weren't automatically imported from the DeACSM plugin when importing a PDF file.
+- Re-enable Xrefs in exported PDF files since the file corruption bug is hopefully fixed. Please open bug reports if you encounter new issues with PDF files.
+- Fix a bug that would sometimes cause corrupted keys to be added when adding them through the config dialog (fixes #145, #134, #119, #116, #115, #109).
+- Update the README (fixes #136) to indicate that Apprentice Harper's version is no longer being updated.
+- Fix a bug where PDFs with empty arrays (`<>`) in a PDF object failed to decrypt, fixes #183.
+- Automatically strip whitespace from entered Amazon Kindle serial numbers, should fix #158.
+- Obok: Add new setting option "Add new entry" for duplicate books to always add them to the Calibre database as a new book. Fixes #148.
+- Obok: Fix where changing the Calibre UI language to some languages would cause the "duplicate book" setting to reset.
+- Fix Python3 bug in stylexml2css.php script, fixes #232.
+- PDF: Ignore invalid PDF objids unless the script is running in strict mode. Fixes some PDFs, apparently. Fixes #233. 
+- Bugfix: EPUBs with remaining content in the encryption.xml after decryption weren't written correctly. 
+- Support for Adobe's 'aes128-cbc-uncompressed' encryption method (fixes #242).
+- Two bugfixes for Amazon DeDRM from Satuoni ( https://github.com/noDRM/DeDRM_tools/issues/315#issuecomment-1508305428 ) and andrewc12 ( https://github.com/andrewc12/DeDRM_tools/commit/d9233d61f00d4484235863969919059f4d0b2057 ) that might make the plugin work with newer versions.
+- Fix font decryption not working with some books (fixes #347), thanks for the patch @bydioeds. 
+- Fix a couple unicode errors for Python2 in Kindle and Nook code.

DeDRM_plugin/__init__.py

@@ -134,29 +134,8 @@ except:
     config_dir = ""
 
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get safely
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+import utilities
+
 
 PLUGIN_NAME = __version.PLUGIN_NAME
 PLUGIN_VERSION = __version.PLUGIN_VERSION
@@ -182,12 +161,8 @@ class DeDRM(FileTypePlugin):
     
     def initialize(self):
         """
-        Dynamic modules can't be imported/loaded from a zipfile.
-        So this routine will extract the appropriate
-        library for the target OS and copy it to the 'alfcrypto' subdirectory of
-        calibre's configuration directory. That 'alfcrypto' directory is then
-        inserted into the syspath (as the very first entry) in the run function
-        so the CDLL stuff will work in the alfcrypto.py script.
+        Extracting a couple Python scripts if running on Linux, 
+        just in case we need to run them in Wine.
 
         The extraction only happens once per version of the plugin
         Also perform upgrade of preferences once per version
@@ -208,15 +183,12 @@ class DeDRM(FileTypePlugin):
                 os.mkdir(self.alfdir)
             # only continue if we've never run this version of the plugin before
             self.verdir = os.path.join(self.maindir,PLUGIN_VERSION)
-            if not os.path.exists(self.verdir):
-                if iswindows:
-                    names = ["alfcrypto.dll","alfcrypto64.dll"]
-                elif isosx:
-                    names = ["libalfcrypto.dylib"]
-                else:
-                    names = ["libalfcrypto32.so","libalfcrypto64.so","kindlekey.py","adobekey.py","subasyncio.py"]
+            if not os.path.exists(self.verdir) and not iswindows and not isosx:
+
+                names = ["kindlekey.py","adobekey.py","ignoblekeyNookStudy.py","utilities.py","argv_utils.py"]
+
                 lib_dict = self.load_resources(names)
-                print("{0} v{1}: Copying needed library files from plugin's zip".format(PLUGIN_NAME, PLUGIN_VERSION))
+                print("{0} v{1}: Copying needed Python scripts from plugin's zip".format(PLUGIN_NAME, PLUGIN_VERSION))
 
                 for entry, data in lib_dict.items():
                     file_path = os.path.join(self.alfdir, entry)
@@ -228,7 +200,7 @@ class DeDRM(FileTypePlugin):
                     try:
                         open(file_path,'wb').write(data)
                     except:
-                        print("{0} v{1}: Exception when copying needed library files".format(PLUGIN_NAME, PLUGIN_VERSION))
+                        print("{0} v{1}: Exception when copying needed python scripts".format(PLUGIN_NAME, PLUGIN_VERSION))
                         traceback.print_exc()
                         pass
 
@@ -511,10 +483,10 @@ class DeDRM(FileTypePlugin):
                             continue
 
                         # Found matching key
-                        userkey = codecs.decode(userkeyhex, 'hex')
                         print("{0} v{1}: Trying UUID-matched encryption key {2:s}".format(PLUGIN_NAME, PLUGIN_VERSION, keyname))
                         of = self.temporary_file(".epub")
                         try: 
+                            userkey = codecs.decode(userkeyhex, 'hex')
                             result = ineptepub.decryptBook(userkey, inf.name, of.name)
                             of.close()
                             if result == 0:
@@ -531,12 +503,13 @@ class DeDRM(FileTypePlugin):
 
                 # Attempt to decrypt epub with each encryption key (generated or provided).
                 for keyname, userkeyhex in dedrmprefs['adeptkeys'].items():
-                    userkey = codecs.decode(userkeyhex, 'hex')
+                    
                     print("{0} v{1}: Trying Encryption key {2:s}".format(PLUGIN_NAME, PLUGIN_VERSION, keyname))
                     of = self.temporary_file(".epub")
 
                     # Give the user key, ebook and TemporaryPersistent file to the decryption function.
                     try:
+                        userkey = codecs.decode(userkeyhex, 'hex')
                         result = ineptepub.decryptBook(userkey, inf.name, of.name)
                     except ineptepub.ADEPTNewVersionError:
                         print("{0} v{1}: Book uses unsupported (too new) Adobe DRM.".format(PLUGIN_NAME, PLUGIN_VERSION, time.time()-self.starttime))
@@ -673,11 +646,11 @@ class DeDRM(FileTypePlugin):
                     continue
             
                 # Found matching key
-                userkey = codecs.decode(userkeyhex, 'hex')
                 print("{0} v{1}: Trying UUID-matched encryption key {2:s}".format(PLUGIN_NAME, PLUGIN_VERSION, keyname))
                 of = self.temporary_file(".pdf")
 
                 try: 
+                    userkey = codecs.decode(userkeyhex, 'hex')
                     result = ineptpdf.decryptBook(userkey, path_to_ebook, of.name)
                     of.close()
                     if result == 0:
@@ -760,10 +733,10 @@ class DeDRM(FileTypePlugin):
             if newkey is not None: 
                 if codecs.encode(newkey, 'hex').decode('ascii') not in dedrmprefs['adeptkeys'].values():
                     print("{0} v{1}: Found new key '{2}' in DeACSM plugin".format(PLUGIN_NAME, PLUGIN_VERSION, newname))
-                    newkeys.append(keyvalue)
+                    newkeys.append(newkey)
                     newnames.append(newname)
         except:
-            pass
+            traceback.print_exc()
 
         if len(newkeys) > 0:
             try:
@@ -797,7 +770,7 @@ class DeDRM(FileTypePlugin):
 
                     print("{0} v{1}: Failed to decrypt with new default key after {2:.1f} seconds".format(PLUGIN_NAME, PLUGIN_VERSION,time.time()-self.starttime))
             except Exception as e:
-                pass
+                traceback.print_exc()
 
 
         # Unable to decrypt the PDF with any of the existing keys. Is it a B&N PDF?
@@ -1025,8 +998,8 @@ class DeDRM(FileTypePlugin):
     def run(self, path_to_ebook):
 
         # make sure any unicode output gets converted safely with 'replace'
-        sys.stdout=SafeUnbuffered(sys.stdout)
-        sys.stderr=SafeUnbuffered(sys.stderr)
+        sys.stdout=utilities.SafeUnbuffered(sys.stdout)
+        sys.stderr=utilities.SafeUnbuffered(sys.stderr)
 
         print("{0} v{1}: Trying to decrypt {2}".format(PLUGIN_NAME, PLUGIN_VERSION, os.path.basename(path_to_ebook)))
         self.starttime = time.time()

DeDRM_plugin/_unused_activitybar.py

@@ -1,77 +0,0 @@
-# I think this file is unused?
-
-import sys
-import tkinter
-import tkinter.constants
-
-class ActivityBar(tkinter.Frame):
-
-    def __init__(self, master, length=300, height=20, barwidth=15, interval=50, bg='white', fillcolor='orchid1',\
-                 bd=2, relief=tkinter.constants.GROOVE, *args, **kw):
-        tkinter.Frame.__init__(self, master, bg=bg, width=length, height=height, *args, **kw)
-        self._master = master
-        self._interval = interval
-        self._maximum = length
-        self._startx = 0
-        self._barwidth = barwidth
-        self._bardiv = length / barwidth
-        if self._bardiv < 10:
-            self._bardiv = 10
-        stopx = self._startx + self._barwidth
-        if stopx > self._maximum:
-            stopx = self._maximum
-        # self._canv = Tkinter.Canvas(self, bg=self['bg'], width=self['width'], height=self['height'],\
-        #                             highlightthickness=0, relief='flat', bd=0)
-        self._canv = tkinter.Canvas(self, bg=self['bg'], width=self['width'], height=self['height'],\
-                                    highlightthickness=0, relief=relief, bd=bd)
-        self._canv.pack(fill='both', expand=1)
-        self._rect = self._canv.create_rectangle(0, 0, self._canv.winfo_reqwidth(), self._canv.winfo_reqheight(), fill=fillcolor, width=0)
-
-        self._set()
-        self.bind('<Configure>', self._update_coords)
-        self._running = False
-
-    def _update_coords(self, event):
-        '''Updates the position of the rectangle inside the canvas when the size of
-        the widget gets changed.'''
-        # looks like we have to call update_idletasks() twice to make sure
-        # to get the results we expect
-        self._canv.update_idletasks()
-        self._maximum = self._canv.winfo_width()
-        self._startx = 0
-        self._barwidth = self._maximum / self._bardiv
-        if self._barwidth < 2:
-            self._barwidth = 2
-        stopx = self._startx + self._barwidth
-        if stopx > self._maximum:
-            stopx = self._maximum
-        self._canv.coords(self._rect, 0, 0, stopx, self._canv.winfo_height())
-        self._canv.update_idletasks()
-
-    def _set(self):
-        if self._startx < 0:
-            self._startx = 0
-        if self._startx > self._maximum:
-            self._startx = self._startx % self._maximum
-        stopx = self._startx + self._barwidth
-        if stopx > self._maximum:
-            stopx = self._maximum
-        self._canv.coords(self._rect, self._startx, 0, stopx, self._canv.winfo_height())
-        self._canv.update_idletasks()
-
-    def start(self):
-        self._running = True
-        self.after(self._interval, self._step)
-
-    def stop(self):
-        self._running = False
-        self._set()
-
-    def _step(self):
-        if self._running:
-            stepsize = self._barwidth / 4
-            if stepsize < 2:
-                stepsize = 2
-            self._startx += stepsize
-            self._set()
-            self.after(self._interval, self._step)

DeDRM_plugin/_unused_scrolltextwidget.py

@@ -1,30 +0,0 @@
-#!/usr/bin/env python
-# vim:ts=4:sw=4:softtabstop=4:smarttab:expandtab
-
-# I think this file is unused?
-
-
-import tkinter
-import tkinter.constants
-
-# basic scrolled text widget
-class ScrolledText(tkinter.Text):
-    def __init__(self, master=None, **kw):
-        self.frame = tkinter.Frame(master)
-        self.vbar = tkinter.Scrollbar(self.frame)
-        self.vbar.pack(side=tkinter.constants.RIGHT, fill=tkinter.constants.Y)
-        kw.update({'yscrollcommand': self.vbar.set})
-        tkinter.Text.__init__(self, self.frame, **kw)
-        self.pack(side=tkinter.constants.LEFT, fill=tkinter.constants.BOTH, expand=True)
-        self.vbar['command'] = self.yview
-        # Copy geometry methods of self.frame without overriding Text
-        # methods = hack!
-        text_meths = list(vars(tkinter.Text).keys())
-        methods = list(vars(tkinter.Pack).keys()) + list(vars(tkinter.Grid).keys()) + list(vars(tkinter.Place).keys())
-        methods = set(methods).difference(text_meths)
-        for m in methods:
-            if m[0] != '_' and m != 'config' and m != 'configure':
-                setattr(self, m, getattr(self.frame, m))
-
-    def __str__(self):
-        return str(self.frame)

DeDRM_plugin/adobekey.py

@@ -45,29 +45,10 @@ import sys, os, struct, getopt
 from base64 import b64decode
 
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+
+from utilities import SafeUnbuffered
+from argv_utils import unicode_argv
+
 
 try:
     from calibre.constants import iswindows, isosx
@@ -75,41 +56,6 @@ except:
     iswindows = sys.platform.startswith('win')
     isosx = sys.platform.startswith('darwin')
 
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.  So use shell32.GetCommandLineArgvW to get sys.argv
-        # as a list of Unicode strings and encode them as utf-8
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["adobekey.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
 
 class ADEPTError(Exception):
     pass
@@ -507,7 +453,7 @@ def usage(progname):
 def cli_main():
     sys.stdout=SafeUnbuffered(sys.stdout)
     sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
+    argv=unicode_argv("adobekey.py")
     progname = os.path.basename(argv[0])
     print("{0} v{1}\nCopyright © 2009-2020 i♥cabbages, Apprentice Harper et al.".format(progname,__version__))
 
@@ -585,7 +531,7 @@ def gui_main():
             self.text.insert(tkinter.constants.END, text)
 
 
-    argv=unicode_argv()
+    argv=unicode_argv("adobekey.py")
     root = tkinter.Tk()
     root.withdraw()
     progpath, progname = os.path.split(argv[0])

DeDRM_plugin/alfcrypto.py

@@ -8,260 +8,90 @@
 # pbkdf2.py Copyright © 2009 Daniel Holth <dholth@fastmail.fm>
 # pbkdf2.py This code may be freely used and modified for any purpose.
 
-import sys, os
 import hmac
 from struct import pack
 import hashlib
+import aescbc
 
-# interface to needed routines libalfcrypto
-def _load_libalfcrypto():
-    import ctypes
-    from ctypes import CDLL, byref, POINTER, c_void_p, c_char_p, c_int, c_long, \
-        Structure, c_ulong, create_string_buffer, addressof, string_at, cast, sizeof
+class Pukall_Cipher(object):
+    def __init__(self):
+        self.key = None
 
-    pointer_size = ctypes.sizeof(ctypes.c_voidp)
-    name_of_lib = None
-    if sys.platform.startswith('darwin'):
-        name_of_lib = 'libalfcrypto.dylib'
-    elif sys.platform.startswith('win'):
-        if pointer_size == 4:
-            name_of_lib = 'alfcrypto.dll'
-        else:
-            name_of_lib = 'alfcrypto64.dll'
-    else:
-        if pointer_size == 4:
-            name_of_lib = 'libalfcrypto32.so'
-        else:
-            name_of_lib = 'libalfcrypto64.so'
-
-    # hard code to local location for libalfcrypto
-    libalfcrypto = os.path.join(sys.path[0],name_of_lib)
-    if not os.path.isfile(libalfcrypto):
-        libalfcrypto = os.path.join(sys.path[0], 'lib', name_of_lib)
-    if not os.path.isfile(libalfcrypto):
-        libalfcrypto = os.path.join('.',name_of_lib)
-    if not os.path.isfile(libalfcrypto):
-        raise Exception('libalfcrypto not found at %s' % libalfcrypto)
-
-    libalfcrypto = CDLL(libalfcrypto)
-
-    c_char_pp = POINTER(c_char_p)
-    c_int_p = POINTER(c_int)
-
-
-    def F(restype, name, argtypes):
-        func = getattr(libalfcrypto, name)
-        func.restype = restype
-        func.argtypes = argtypes
-        return func
-
-    # aes cbc decryption
-    #
-    # struct aes_key_st {
-    # unsigned long rd_key[4 *(AES_MAXNR + 1)];
-    # int rounds;
-    # };
-    #
-    # typedef struct aes_key_st AES_KEY;
-    #
-    # int AES_set_decrypt_key(const unsigned char *userKey, const int bits, AES_KEY *key);
-    #
-    #
-    # void AES_cbc_encrypt(const unsigned char *in, unsigned char *out,
-    # const unsigned long length, const AES_KEY *key,
-    # unsigned char *ivec, const int enc);
-
-    AES_MAXNR = 14
-
-    class AES_KEY(Structure):
-        _fields_ = [('rd_key', c_long * (4 * (AES_MAXNR + 1))), ('rounds', c_int)]
-
-    AES_KEY_p = POINTER(AES_KEY)
-    AES_cbc_encrypt = F(None, 'AES_cbc_encrypt',[c_char_p, c_char_p, c_ulong, AES_KEY_p, c_char_p, c_int])
-    AES_set_decrypt_key = F(c_int, 'AES_set_decrypt_key',[c_char_p, c_int, AES_KEY_p])
-
-
-
-    # Pukall 1 Cipher
-    # unsigned char *PC1(const unsigned char *key, unsigned int klen, const unsigned char *src,
-    #                unsigned char *dest, unsigned int len, int decryption);
-
-    PC1 = F(c_char_p, 'PC1', [c_char_p, c_ulong, c_char_p, c_char_p, c_ulong, c_ulong])
-
-    # Topaz Encryption
-    # typedef struct _TpzCtx {
-    #    unsigned int v[2];
-    # } TpzCtx;
-    #
-    # void topazCryptoInit(TpzCtx *ctx, const unsigned char *key, int klen);
-    # void topazCryptoDecrypt(const TpzCtx *ctx, const unsigned char *in, unsigned char *out, int len);
-
-    class TPZ_CTX(Structure):
-        _fields_ = [('v', c_long * 2)]
-
-    TPZ_CTX_p = POINTER(TPZ_CTX)
-    topazCryptoInit = F(None, 'topazCryptoInit', [TPZ_CTX_p, c_char_p, c_ulong])
-    topazCryptoDecrypt = F(None, 'topazCryptoDecrypt', [TPZ_CTX_p, c_char_p, c_char_p, c_ulong])
-
-
-    class AES_CBC(object):
-        def __init__(self):
-            self._blocksize = 0
-            self._keyctx = None
-            self._iv = 0
-
-        def set_decrypt_key(self, userkey, iv):
-            self._blocksize = len(userkey)
-            if (self._blocksize != 16) and (self._blocksize != 24) and (self._blocksize != 32) :
-                raise Exception('AES CBC improper key used')
-                return
-            keyctx = self._keyctx = AES_KEY()
-            self._iv = iv
-            rv = AES_set_decrypt_key(userkey, len(userkey) * 8, keyctx)
-            if rv < 0:
-                raise Exception('Failed to initialize AES CBC key')
-
-        def decrypt(self, data):
-            out = create_string_buffer(len(data))
-            mutable_iv = create_string_buffer(self._iv, len(self._iv))
-            rv = AES_cbc_encrypt(data, out, len(data), self._keyctx, mutable_iv, 0)
-            if rv == 0:
-                raise Exception('AES CBC decryption failed')
-            return out.raw
-
-    class Pukall_Cipher(object):
-        def __init__(self):
-            self.key = None
-
-        def PC1(self, key, src, decryption=True):
-            self.key = key
-            out = create_string_buffer(len(src))
-            de = 0
+    def PC1(self, key, src, decryption=True):
+        sum1 = 0;
+        sum2 = 0;
+        keyXorVal = 0;
+        if len(key)!=16:
+            raise Exception("PC1: Bad key length")
+        wkey = []
+        for i in range(8):
+            wkey.append(key[i*2]<<8 | key[i*2+1])
+        dst = bytearray(len(src))
+        for i in range(len(src)):
+            temp1 = 0;
+            byteXorVal = 0;
+            for j in range(8):
+                temp1 ^= wkey[j]
+                sum2  = (sum2+j)*20021 + sum1
+                sum1  = (temp1*346)&0xFFFF
+                sum2  = (sum2+sum1)&0xFFFF
+                temp1 = (temp1*20021+1)&0xFFFF
+                byteXorVal ^= temp1 ^ sum2
+            curByte = src[i]
+            if not decryption:
+                keyXorVal = curByte * 257;
+            curByte = ((curByte ^ (byteXorVal >> 8)) ^ byteXorVal) & 0xFF
             if decryption:
-                de = 1
-            rv = PC1(key, len(key), src, out, len(src), de)
-            return out.raw
+                keyXorVal = curByte * 257;
+            for j in range(8):
+                wkey[j] ^= keyXorVal;
+            dst[i] = curByte
+        return bytes(dst)
 
-    class Topaz_Cipher(object):
-        def __init__(self):
-            self._ctx = None
+class Topaz_Cipher(object):
+    def __init__(self):
+        self._ctx = None
 
-        def ctx_init(self, key):
-            tpz_ctx = self._ctx = TPZ_CTX()
-            topazCryptoInit(tpz_ctx, key, len(key))
-            return tpz_ctx
+    def ctx_init(self, key):
+        ctx1 = 0x0CAFFE19E
+        if isinstance(key, str):
+            key = key.encode('latin-1')
+        for keyByte in key:
+            ctx2 = ctx1
+            ctx1 = ((((ctx1 >>2) * (ctx1 >>7))&0xFFFFFFFF) ^ (keyByte * keyByte * 0x0F902007)& 0xFFFFFFFF )
+        self._ctx = [ctx1, ctx2]
+        return [ctx1,ctx2]
 
-        def decrypt(self, data,  ctx=None):
-            if ctx == None:
-                ctx = self._ctx
-            out = create_string_buffer(len(data))
-            topazCryptoDecrypt(ctx, data, out, len(data))
-            return out.raw
+    def decrypt(self, data,  ctx=None):
+        if ctx == None:
+            ctx = self._ctx
+        ctx1 = ctx[0]
+        ctx2 = ctx[1]
+        plainText = ""
+        if isinstance(data, str):
+            data = data.encode('latin-1')
+        for dataByte in data:
+            m = (dataByte ^ ((ctx1 >> 3) &0xFF) ^ ((ctx2<<3) & 0xFF)) &0xFF
+            ctx2 = ctx1
+            ctx1 = (((ctx1 >> 2) * (ctx1 >> 7)) &0xFFFFFFFF) ^((m * m * 0x0F902007) &0xFFFFFFFF)
+            plainText += chr(m)
+        return plainText
 
-    print("Using Library AlfCrypto DLL/DYLIB/SO")
-    return (AES_CBC, Pukall_Cipher, Topaz_Cipher)
+class AES_CBC(object):
+    def __init__(self):
+        self._key = None
+        self._iv = None
+        self.aes = None
 
+    def set_decrypt_key(self, userkey, iv):
+        self._key = userkey
+        self._iv = iv
+        self.aes = aescbc.AES_CBC(userkey, aescbc.noPadding(), len(userkey))
 
-def _load_python_alfcrypto():
-
-    import aescbc
-
-    class Pukall_Cipher(object):
-        def __init__(self):
-            self.key = None
-
-        def PC1(self, key, src, decryption=True):
-            sum1 = 0;
-            sum2 = 0;
-            keyXorVal = 0;
-            if len(key)!=16:
-                raise Exception('Pukall_Cipher: Bad key length.')
-            wkey = []
-            for i in range(8):
-                wkey.append(ord(key[i*2])<<8 | ord(key[i*2+1]))
-            dst = ""
-            for i in range(len(src)):
-                temp1 = 0;
-                byteXorVal = 0;
-                for j in range(8):
-                    temp1 ^= wkey[j]
-                    sum2  = (sum2+j)*20021 + sum1
-                    sum1  = (temp1*346)&0xFFFF
-                    sum2  = (sum2+sum1)&0xFFFF
-                    temp1 = (temp1*20021+1)&0xFFFF
-                    byteXorVal ^= temp1 ^ sum2
-                curByte = ord(src[i])
-                if not decryption:
-                    keyXorVal = curByte * 257;
-                curByte = ((curByte ^ (byteXorVal >> 8)) ^ byteXorVal) & 0xFF
-                if decryption:
-                    keyXorVal = curByte * 257;
-                for j in range(8):
-                    wkey[j] ^= keyXorVal;
-                dst+=chr(curByte)
-            return dst
-
-    class Topaz_Cipher(object):
-        def __init__(self):
-            self._ctx = None
-
-        def ctx_init(self, key):
-            ctx1 = 0x0CAFFE19E
-            if isinstance(key, str):
-                key = key.encode('latin-1')
-            for keyByte in key:
-                ctx2 = ctx1
-                ctx1 = ((((ctx1 >>2) * (ctx1 >>7))&0xFFFFFFFF) ^ (keyByte * keyByte * 0x0F902007)& 0xFFFFFFFF )
-            self._ctx = [ctx1, ctx2]
-            return [ctx1,ctx2]
-
-        def decrypt(self, data,  ctx=None):
-            if ctx == None:
-                ctx = self._ctx
-            ctx1 = ctx[0]
-            ctx2 = ctx[1]
-            plainText = ""
-            if isinstance(data, str):
-                data = data.encode('latin-1')
-            for dataByte in data:
-                m = (dataByte ^ ((ctx1 >> 3) &0xFF) ^ ((ctx2<<3) & 0xFF)) &0xFF
-                ctx2 = ctx1
-                ctx1 = (((ctx1 >> 2) * (ctx1 >> 7)) &0xFFFFFFFF) ^((m * m * 0x0F902007) &0xFFFFFFFF)
-                plainText += chr(m)
-            return plainText
-
-    class AES_CBC(object):
-        def __init__(self):
-            self._key = None
-            self._iv = None
-            self.aes = None
-
-        def set_decrypt_key(self, userkey, iv):
-            self._key = userkey
-            self._iv = iv
-            self.aes = aescbc.AES_CBC(userkey, aescbc.noPadding(), len(userkey))
-
-        def decrypt(self, data):
-            iv = self._iv
-            cleartext = self.aes.decrypt(iv + data)
-            return cleartext
-
-    print("Using Library AlfCrypto Python")
-    return (AES_CBC, Pukall_Cipher, Topaz_Cipher)
-
-
-def _load_crypto():
-    AES_CBC = Pukall_Cipher = Topaz_Cipher = None
-    cryptolist = (_load_libalfcrypto, _load_python_alfcrypto)
-    for loader in cryptolist:
-        try:
-            AES_CBC, Pukall_Cipher, Topaz_Cipher = loader()
-            break
-        except (ImportError, Exception):
-            pass
-    return AES_CBC, Pukall_Cipher, Topaz_Cipher
-
-AES_CBC, Pukall_Cipher, Topaz_Cipher = _load_crypto()
+    def decrypt(self, data):
+        iv = self._iv
+        cleartext = self.aes.decrypt(iv + data)
+        return cleartext
 
 
 class KeyIVGen(object):

DeDRM_plugin/androidkindlekey.py

@@ -41,71 +41,6 @@ except ImportError:
 
 # Routines common to Mac and PC
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
-
-try:
-    from calibre.constants import iswindows, isosx
-except:
-    iswindows = sys.platform.startswith('win')
-    isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.  So use shell32.GetCommandLineArgvW to get sys.argv
-        # as a list of Unicode strings and encode them as utf-8
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["kindlekey.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
 
 class DrmException(Exception):
     pass
@@ -342,9 +277,7 @@ def usage(progname):
 
 
 def cli_main():
-    sys.stdout=SafeUnbuffered(sys.stdout)
-    sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
+    argv=sys.argv
     progname = os.path.basename(argv[0])
     print("{0} v{1}\nCopyright © 2010-2020 Thom, Apprentice Harper et al.".format(progname,__version__))
 
@@ -463,7 +396,7 @@ def gui_main():
                 return
             self.status['text'] = "Select backup.ab file"
 
-    argv=unicode_argv()
+    argv=sys.argv()
     progpath, progname = os.path.split(argv[0])
     root = tkinter.Tk()
     root.title("Kindle for Android Key Extraction v.{0}".format(__version__))

DeDRM_plugin/argv_utils.py

@@ -1,14 +1,17 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 
-import sys, os
-import locale
-import codecs
-import importlib
+import sys
 
 # get sys.argv arguments and encode them into utf-8
-def unicode_argv():
-    if sys.platform.startswith('win'):
+def unicode_argv(default_name):
+
+    try:
+        from calibre.constants import iswindows
+    except:
+        iswindows = sys.platform.startswith('win')
+
+    if iswindows:
         # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
         # strings.
 
@@ -38,50 +41,8 @@ def unicode_argv():
                     range(start, argc.value)]
         # if we don't have any arguments at all, just pass back script name
         # this should never happen
-        return ["DeDRM.py"]
+        return [ default_name ]
     else:
         argvencoding = sys.stdin.encoding or "utf-8"
         return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
 
-
-def add_cp65001_codec():
-  try:
-    codecs.lookup('cp65001')
-  except LookupError:
-    codecs.register(
-        lambda name: name == 'cp65001' and codecs.lookup('utf-8') or None)
-  return
-
-
-def set_utf8_default_encoding():
-  if sys.getdefaultencoding() == 'utf-8':
-    return
-
-  # Regenerate setdefaultencoding.
-  importlib.reload(sys)
-  sys.setdefaultencoding('utf-8')
-
-  for attr in dir(locale):
-    if attr[0:3] != 'LC_':
-      continue
-    aref = getattr(locale, attr)
-    try:
-      locale.setlocale(aref, '')
-    except locale.Error:
-      continue
-    try:
-      lang = locale.getlocale(aref)[0]
-    except (TypeError, ValueError):
-      continue
-    if lang:
-      try:
-        locale.setlocale(aref, (lang, 'UTF-8'))
-      except locale.Error:
-        os.environ[attr] = lang + '.UTF-8'
-  try:
-    locale.setlocale(locale.LC_ALL, '')
-  except locale.Error:
-    pass
-  return
-
-

DeDRM_plugin/askfolder_ed.py

@@ -1,213 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-# vim:ts=4:sw=4:softtabstop=4:smarttab:expandtab
-
-# to work around tk_chooseDirectory not properly returning unicode paths on Windows
-# need to use a dialog that can be hacked up to actually return full unicode paths
-# originally based on AskFolder from EasyDialogs for Windows but modified to fix it
-# to actually use unicode for path
-
-# The original license for EasyDialogs is as follows
-#
-# Copyright (c) 2003-2005 Jimmy Retzlaff
-#
-# Permission is hereby granted, free of charge, to any person obtaining a
-# copy of this software and associated documentation files (the "Software"),
-# to deal in the Software without restriction, including without limitation
-# the rights to use, copy, modify, merge, publish, distribute, sublicense,
-# and/or sell copies of the Software, and to permit persons to whom the
-# Software is furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-# DEALINGS IN THE SOFTWARE.
-
-# Adjusted for Python 3, September 2020
-
-"""
-AskFolder(...) -- Ask the user to select a folder Windows specific
-"""
-
-import os
-
-import ctypes
-from ctypes import POINTER, byref, cdll, c_int, windll
-from ctypes.wintypes import LPCWSTR, LPWSTR
-import ctypes.wintypes as wintypes
-
-
-__all__ = ['AskFolder']
-
-# Load required Windows DLLs
-ole32 = ctypes.windll.ole32
-shell32 = ctypes.windll.shell32
-user32 = ctypes.windll.user32
-
-
-# Windows Constants
-BFFM_INITIALIZED = 1
-BFFM_SETOKTEXT = 1129
-BFFM_SETSELECTIONA = 1126
-BFFM_SETSELECTIONW = 1127
-BIF_EDITBOX = 16
-BS_DEFPUSHBUTTON = 1
-CB_ADDSTRING = 323
-CB_GETCURSEL = 327
-CB_SETCURSEL = 334
-CDM_SETCONTROLTEXT = 1128
-EM_GETLINECOUNT = 186
-EM_GETMARGINS = 212
-EM_POSFROMCHAR = 214
-EM_SETSEL = 177
-GWL_STYLE = -16
-IDC_STATIC = -1
-IDCANCEL = 2
-IDNO = 7
-IDOK = 1
-IDYES = 6
-MAX_PATH = 260
-OFN_ALLOWMULTISELECT = 512
-OFN_ENABLEHOOK = 32
-OFN_ENABLESIZING = 8388608
-OFN_ENABLETEMPLATEHANDLE = 128
-OFN_EXPLORER = 524288
-OFN_OVERWRITEPROMPT = 2
-OPENFILENAME_SIZE_VERSION_400 = 76
-PBM_GETPOS = 1032
-PBM_SETMARQUEE = 1034
-PBM_SETPOS = 1026
-PBM_SETRANGE = 1025
-PBM_SETRANGE32 = 1030
-PBS_MARQUEE = 8
-PM_REMOVE = 1
-SW_HIDE = 0
-SW_SHOW = 5
-SW_SHOWNORMAL = 1
-SWP_NOACTIVATE = 16
-SWP_NOMOVE = 2
-SWP_NOSIZE = 1
-SWP_NOZORDER = 4
-VER_PLATFORM_WIN32_NT = 2
-WM_COMMAND = 273
-WM_GETTEXT = 13
-WM_GETTEXTLENGTH = 14
-WM_INITDIALOG = 272
-WM_NOTIFY = 78
-
-# Windows function prototypes
-BrowseCallbackProc = ctypes.WINFUNCTYPE(ctypes.c_int, wintypes.HWND, ctypes.c_uint, wintypes.LPARAM, wintypes.LPARAM)
-
-# Windows types
-LPCTSTR = ctypes.c_char_p
-LPTSTR = ctypes.c_char_p
-LPVOID = ctypes.c_voidp
-TCHAR = ctypes.c_char
-
-class BROWSEINFO(ctypes.Structure):
-    _fields_ = [
-                ("hwndOwner", wintypes.HWND),
-                ("pidlRoot", LPVOID),
-                ("pszDisplayName", LPTSTR),
-                ("lpszTitle", LPCTSTR),
-                ("ulFlags", ctypes.c_uint),
-                ("lpfn", BrowseCallbackProc),
-                ("lParam", wintypes.LPARAM),
-                ("iImage", ctypes.c_int)
-               ]
-
-
-# Utilities
-def CenterWindow(hwnd):
-    desktopRect = GetWindowRect(user32.GetDesktopWindow())
-    myRect = GetWindowRect(hwnd)
-    x = width(desktopRect) // 2 - width(myRect) // 2
-    y = height(desktopRect) // 2 - height(myRect) // 2
-    user32.SetWindowPos(hwnd, 0,
-                        desktopRect.left + x,
-                        desktopRect.top + y,
-                        0, 0,
-                        SWP_NOACTIVATE | SWP_NOSIZE | SWP_NOZORDER
-                       )
-
-
-def GetWindowRect(hwnd):
-    rect = wintypes.RECT()
-    user32.GetWindowRect(hwnd, ctypes.byref(rect))
-    return rect
-
-def width(rect):
-    return rect.right-rect.left
-
-def height(rect):
-    return rect.bottom-rect.top
-
-
-def AskFolder(
-        message=None,
-        version=None,
-        defaultLocation=None,
-        location=None,
-        windowTitle=None,
-        actionButtonLabel=None,
-        cancelButtonLabel=None,
-        multiple=None):
-    """Display a dialog asking the user for select a folder.
-       modified to use unicode strings as much as possible
-       returns unicode path
-    """
-
-    def BrowseCallback(hwnd, uMsg, lParam, lpData):
-        if uMsg == BFFM_INITIALIZED:
-            if actionButtonLabel:
-                label = str(actionButtonLabel, errors='replace')
-                user32.SendMessageW(hwnd, BFFM_SETOKTEXT, 0, label)
-            if cancelButtonLabel:
-                label = str(cancelButtonLabel, errors='replace')
-                cancelButton = user32.GetDlgItem(hwnd, IDCANCEL)
-                if cancelButton:
-                    user32.SetWindowTextW(cancelButton, label)
-            if windowTitle:
-                title = str(windowTitle, errors='replace')
-                user32.SetWindowTextW(hwnd, title)
-            if defaultLocation:
-                user32.SendMessageW(hwnd, BFFM_SETSELECTIONW, 1, defaultLocation.replace('/', '\\'))
-            if location:
-                x, y = location
-                desktopRect = wintypes.RECT()
-                user32.GetWindowRect(0, ctypes.byref(desktopRect))
-                user32.SetWindowPos(hwnd, 0,
-                                  desktopRect.left + x,
-                                  desktopRect.top + y, 0, 0,
-                                  SWP_NOACTIVATE | SWP_NOSIZE | SWP_NOZORDER)
-            else:
-                CenterWindow(hwnd)
-        return 0
-
-    # This next line is needed to prevent gc of the callback
-    callback = BrowseCallbackProc(BrowseCallback)
-
-    browseInfo = BROWSEINFO()
-    browseInfo.pszDisplayName = ctypes.c_char_p('\0' * (MAX_PATH+1))
-    browseInfo.lpszTitle = message
-    browseInfo.lpfn = callback
-
-    pidl = shell32.SHBrowseForFolder(ctypes.byref(browseInfo))
-    if not pidl:
-        result = None
-    else:
-        path = LPCWSTR(" " * (MAX_PATH+1))
-        shell32.SHGetPathFromIDListW(pidl, path)
-        ole32.CoTaskMemFree(pidl)
-        result = path.value
-    return result
-
-
-
-

DeDRM_plugin/config.py

@@ -1152,7 +1152,8 @@ class AddAdeptDialog():
                 zip_function = zip
 
             for key, name in zip_function(defaultkeys, defaultnames):
-                if codecs.encode(key,'hex').decode("latin-1") in self.parent.plugin_keys.values():
+                key = codecs.encode(key,'hex').decode("latin-1")
+                if key in self.parent.plugin_keys.values():
                     print("Found key '{0}' in ADE - already present, skipping.".format(name))
                 else:
                     self.new_keys.append(key)
@@ -1167,8 +1168,8 @@ class AddAdeptDialog():
             key, name = checkForDeACSMkeys()
 
             if key is not None: 
-
-                if codecs.encode(key,'hex').decode("latin-1") in self.parent.plugin_keys.values():
+                key = codecs.encode(key,'hex').decode("latin-1")
+                if key in self.parent.plugin_keys.values():
                     print("Found key '{0}' in DeACSM - already present, skipping.".format(name))
                 else: 
                     # Found new key, add that.
@@ -1202,7 +1203,7 @@ class AddAdeptDialog():
 
     @property
     def key_value(self):
-        return codecs.encode(self.new_keys[0],'hex').decode("utf-8")
+        return codecs.encode(self.new_keys[0],'hex').decode("latin-1")
 
     
     @property
@@ -1325,7 +1326,7 @@ class AddSerialDialog(QDialog):
 
     @property
     def key_value(self):
-        return str(self.key_ledit.text()).replace(' ', '')
+        return str(self.key_ledit.text()).replace(' ', '').replace('\r', '').replace('\n', '').replace('\t', '')
 
     def accept(self):
         if len(self.key_name) == 0 or self.key_name.isspace():

DeDRM_plugin/convert2xml.py

@@ -5,36 +5,13 @@
 # For use with Topaz Scripts Version 2.6
 # Python 3, September 2020
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
 import sys
 import csv
 import os
 import getopt
-from struct import pack
-from struct import unpack
+from struct import pack, unpack
 
 class TpzDRMError(Exception):
     pass

DeDRM_plugin/epubfontdecrypt.py

@@ -2,7 +2,7 @@
 # -*- coding: utf-8 -*-
 
 # epubfontdecrypt.py
-# Copyright © 2021 by noDRM
+# Copyright © 2021-2023 by noDRM
 
 # Released under the terms of the GNU General Public Licence, version 3
 # <http://www.gnu.org/licenses/>
@@ -10,6 +10,7 @@
 
 # Revision history:
 #   1 - Initial release
+#   2 - Bugfix for multiple book IDs, reported at #347
 
 """
 Decrypts / deobfuscates font files in EPUB files
@@ -18,13 +19,14 @@ Decrypts / deobfuscates font files in EPUB files
 from __future__ import print_function
 
 __license__ = 'GPL v3'
-__version__ = "1"
+__version__ = "2"
 
 import os
 import traceback
 import zlib
 import zipfile
 from zipfile import ZipInfo, ZipFile, ZIP_STORED, ZIP_DEFLATED
+from zeroedzipinfo import ZeroedZipInfo
 from contextlib import closing
 from lxml import etree
 import itertools
@@ -192,9 +194,10 @@ def decryptFontsBook(inpath, outpath):
                 pass
 
             try: 
-                identify_element = container.find(packageNS("metadata")).find(metadataDCNS("identifier"))
-                if (secret_key_name is None or secret_key_name == identify_element.get("id")):
-                    font_master_key = identify_element.text
+                identify_elements = container.find(packageNS("metadata")).findall(metadataDCNS("identifier"))
+                for element in identify_elements:
+                    if (secret_key_name is None or secret_key_name == element.get("id")):
+                        font_master_key = element.text
             except: 
                 pass
 
@@ -298,13 +301,21 @@ def decryptFontsBook(inpath, outpath):
                         zi.internal_attr = oldzi.internal_attr
                         # external attributes are dependent on the create system, so copy both.
                         zi.external_attr = oldzi.external_attr
+                        zi.volume = oldzi.volume
                         zi.create_system = oldzi.create_system
+                        zi.create_version = oldzi.create_version
+
                         if any(ord(c) >= 128 for c in path) or any(ord(c) >= 128 for c in zi.comment):
                             # If the file name or the comment contains any non-ASCII char, set the UTF8-flag
                             zi.flag_bits |= 0x800
                     except:
                         pass
 
+                    # Python 3 has a bug where the external_attr is reset to `0o600 << 16`
+                    # if it's NULL, so we need a workaround:
+                    if zi.external_attr == 0: 
+                        zi = ZeroedZipInfo(zi)
+
                     if path == "mimetype":
                         outf.writestr(zi, inf.read('mimetype'))
                     elif path == "META-INF/encryption.xml":

DeDRM_plugin/epubtest.py

@@ -53,75 +53,13 @@ import sys, struct, os, traceback
 import zlib
 import zipfile
 import xml.etree.ElementTree as etree
+from argv_utils import unicode_argv
 
 NSMAP = {'adept': 'http://ns.adobe.com/adept',
          'enc': 'http://www.w3.org/2001/04/xmlenc#'}
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
-try:
-    from calibre.constants import iswindows, isosx
-except:
-    iswindows = sys.platform.startswith('win')
-    isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.  So use shell32.GetCommandLineArgvW to get sys.argv
-        # as a list of Unicode strings and encode them as utf-8
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["epubtest.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
 
 _FILENAME_LEN_OFFSET = 26
 _EXTRA_LEN_OFFSET = 28
@@ -219,7 +157,7 @@ def encryption(infile):
     return encryption
 
 def main():
-    argv=unicode_argv()
+    argv=unicode_argv("epubtest.py")
     if len(argv) < 2:
         print("Give an ePub file as a parameter.")
     else:

DeDRM_plugin/epubwatermark.py

@@ -16,6 +16,7 @@ Removes various watermarks from EPUB files
 
 import traceback
 from zipfile import ZipInfo, ZipFile, ZIP_STORED, ZIP_DEFLATED
+from zeroedzipinfo import ZeroedZipInfo
 from contextlib import closing
 from lxml import etree
 import re
@@ -133,13 +134,22 @@ def removeHTMLwatermarks(object, path_to_ebook):
                         zi.extra = oldzi.extra
                         zi.internal_attr = oldzi.internal_attr
                         zi.external_attr = oldzi.external_attr
+                        zi.volume = oldzi.volume
                         zi.create_system = oldzi.create_system
+                        zi.create_version = oldzi.create_version
+
                         if any(ord(c) >= 128 for c in path) or any(ord(c) >= 128 for c in zi.comment):
                             # If the file name or the comment contains any non-ASCII char, set the UTF8-flag
                             zi.flag_bits |= 0x800
                     except:
                         pass
 
+                    # Python 3 has a bug where the external_attr is reset to `0o600 << 16`
+                    # if it's NULL, so we need a workaround:
+                    if zi.external_attr == 0: 
+                        zi = ZeroedZipInfo(zi)
+
+
                     outf.writestr(zi, data)
         except:
             traceback.print_exc()
@@ -249,13 +259,21 @@ def removeOPFwatermarks(object, path_to_ebook):
                         zi.extra = oldzi.extra
                         zi.internal_attr = oldzi.internal_attr
                         zi.external_attr = oldzi.external_attr
+                        zi.volume = oldzi.volume
                         zi.create_system = oldzi.create_system
+                        zi.create_version = oldzi.create_version
+
                         if any(ord(c) >= 128 for c in path) or any(ord(c) >= 128 for c in zi.comment):
                             # If the file name or the comment contains any non-ASCII char, set the UTF8-flag
                             zi.flag_bits |= 0x800
                     except:
                         pass
 
+                    # Python 3 has a bug where the external_attr is reset to `0o600 << 16`
+                    # if it's NULL, so we need a workaround:
+                    if zi.external_attr == 0: 
+                        zi = ZeroedZipInfo(zi)
+
                     outf.writestr(zi, data)
         except:
             traceback.print_exc()
@@ -301,13 +319,21 @@ def removeCDPwatermark(object, path_to_ebook):
                     zi.extra = oldzi.extra
                     zi.internal_attr = oldzi.internal_attr
                     zi.external_attr = oldzi.external_attr
+                    zi.volume = oldzi.volume
                     zi.create_system = oldzi.create_system
+                    zi.create_version = oldzi.create_version
+
                     if any(ord(c) >= 128 for c in path) or any(ord(c) >= 128 for c in zi.comment):
                         # If the file name or the comment contains any non-ASCII char, set the UTF8-flag
                         zi.flag_bits |= 0x800
                 except:
                     pass
 
+                # Python 3 has a bug where the external_attr is reset to `0o600 << 16`
+                # if it's NULL, so we need a workaround:
+                if zi.external_attr == 0: 
+                    zi = ZeroedZipInfo(zi)
+
                 outf.writestr(zi, data)
         
         print("Watermark: Successfully removed cdp.info watermark")

DeDRM_plugin/erdr2pml.py

@@ -79,68 +79,12 @@ except ImportError:
 
 #@@CALIBRE_COMPAT_CODE@@
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
 iswindows = sys.platform.startswith('win')
 isosx = sys.platform.startswith('darwin')
 
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.
-
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["mobidedrm.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
+from argv_utils import unicode_argv
 
 import cgi
 import logging
@@ -507,7 +451,7 @@ def getuser_key(name,cc):
 def cli_main():
     print("eRdr2Pml v{0}. Copyright © 2009–2020 The Dark Reverser et al.".format(__version__))
 
-    argv=unicode_argv()
+    argv=unicode_argv("erdr2pml.py")
     try:
         opts, args = getopt.getopt(argv[1:], "hp", ["make-pmlz"])
     except getopt.GetoptError as err:

DeDRM_plugin/genbook.py

@@ -4,29 +4,7 @@
 # Python 3 for calibre 5.0
 from __future__ import print_function
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
 import sys
 import csv

DeDRM_plugin/ignoblekeyGenPassHash.py

@@ -50,72 +50,9 @@ try:
 except ImportError:
     from Crypto.Cipher import AES
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
-
-try:
-    from calibre.constants import iswindows, isosx
-except:
-    iswindows = sys.platform.startswith('win')
-    isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.  So use shell32.GetCommandLineArgvW to get sys.argv
-        # as a list of Unicode strings and encode them as utf-8
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["ignoblekeygen.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
+from utilities import SafeUnbuffered
 
+from argv_utils import unicode_argv
 
 class IGNOBLEError(Exception):
     pass
@@ -148,7 +85,7 @@ def generate_key(name, ccn):
 def cli_main():
     sys.stdout=SafeUnbuffered(sys.stdout)
     sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
+    argv=unicode_argv("ignoblekeyGenPassHash.py")
     progname = os.path.basename(argv[0])
     if len(argv) != 4:
         print("usage: {0} <Name> <CC#> <keyfileout.b64>".format(progname))

DeDRM_plugin/ignoblekeyNookStudy.py

@@ -27,71 +27,14 @@ import hashlib
 import getopt
 import re
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
 try:
-    from calibre.constants import iswindows, isosx
+    from calibre.constants import iswindows
 except:
     iswindows = sys.platform.startswith('win')
-    isosx = sys.platform.startswith('darwin')
 
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.  So use shell32.GetCommandLineArgvW to get sys.argv
-        # as a list of Unicode strings and encode them as utf-8
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["ignoblekey.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
+from argv_utils import unicode_argv
 
 class DrmException(Exception):
     pass
@@ -111,15 +54,26 @@ def getNookLogFiles():
         paths = set()
         if 'LOCALAPPDATA' in os.environ.keys():
             # Python 2.x does not return unicode env. Use Python 3.x
-            path = winreg.ExpandEnvironmentStrings("%LOCALAPPDATA%")
+            if sys.version_info[0] == 2:
+                path = winreg.ExpandEnvironmentStrings(u"%LOCALAPPDATA%")
+            else:
+                path = winreg.ExpandEnvironmentStrings("%LOCALAPPDATA%")
             if os.path.isdir(path):
                 paths.add(path)
         if 'USERPROFILE' in os.environ.keys():
             # Python 2.x does not return unicode env. Use Python 3.x
-            path = winreg.ExpandEnvironmentStrings("%USERPROFILE%")+"\\AppData\\Local"
+            if sys.version_info[0] == 2:
+                path = winreg.ExpandEnvironmentStrings(u"%USERPROFILE%")+u"\\AppData\\Local"
+            else:
+                path = winreg.ExpandEnvironmentStrings("%USERPROFILE%")+"\\AppData\\Local"
+
             if os.path.isdir(path):
                 paths.add(path)
-            path = winreg.ExpandEnvironmentStrings("%USERPROFILE%")+"\\AppData\\Roaming"
+
+            if sys.version_info[0] == 2:  
+                path = winreg.ExpandEnvironmentStrings(u"%USERPROFILE%")+u"\\AppData\\Roaming"
+            else:
+                path = winreg.ExpandEnvironmentStrings("%USERPROFILE%")+"\\AppData\\Roaming"
             if os.path.isdir(path):
                 paths.add(path)
         # User Shell Folders show take precedent over Shell Folders if present
@@ -244,7 +198,7 @@ def usage(progname):
 def cli_main():
     sys.stdout=SafeUnbuffered(sys.stdout)
     sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
+    argv=unicode_argv("ignoblekeyNookStudy.py")
     progname = os.path.basename(argv[0])
     print("{0} v{1}\nCopyright © 2015 Apprentice Alf".format(progname,__version__))
 
@@ -305,7 +259,7 @@ def gui_main():
             self.text.insert(tkinter.constants.END, text)
 
 
-    argv=unicode_argv()
+    argv=unicode_argv("ignoblekeyNookStudy.py")
     root = tkinter.Tk()
     root.withdraw()
     progpath, progname = os.path.split(argv[0])

DeDRM_plugin/ignoblekeyfetch.py

@@ -1,265 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-
-# ignoblekeyfetch.py
-# Copyright © 2015-2020 Apprentice Harper et al.
-
-# Released under the terms of the GNU General Public Licence, version 3
-# <http://www.gnu.org/licenses/>
-
-# Based on discoveries by "Nobody You Know"
-# Code partly based on ignoblekeygen.py by several people.
-
-# Windows users: Before running this program, you must first install Python.
-#   We recommend ActiveState Python 2.7.X for Windows from
-#   http://www.activestate.com/activepython/downloads.
-#   Then save this script file as ignoblekeyfetch.pyw and double-click on it to run it.
-#
-# Mac OS X users: Save this script file as ignoblekeyfetch.pyw.  You can run this
-#   program from the command line (python ignoblekeyfetch.pyw) or by double-clicking
-#   it when it has been associated with PythonLauncher.
-
-# Revision history:
-#   1.0 - Initial  version
-#   1.1 - Try second URL if first one fails
-#   2.0 - Python 3 for calibre 5.0
-
-"""
-Fetch Barnes & Noble EPUB user key from B&N servers using email and password.
-
-NOTE: This script used to work in the past, but the server it uses is long gone.
-It can no longer be used to download keys from B&N servers, it is no longer
-supported by the Calibre plugin, and it will be removed in the future. 
-
-"""
-
-__license__ = 'GPL v3'
-__version__ = "2.0"
-
-import sys
-import os
-
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
-
-try:
-    from calibre.constants import iswindows, isosx
-except:
-    iswindows = sys.platform.startswith('win')
-    isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.  So use shell32.GetCommandLineArgvW to get sys.argv
-        # as a list of Unicode strings and encode them as utf-8
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["ignoblekeyfetch.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
-
-
-class IGNOBLEError(Exception):
-    pass
-
-def fetch_key(email, password):
-    # change email and password to utf-8 if unicode
-    if type(email)==str:
-        email = email.encode('utf-8')
-    if type(password)==str:
-        password = password.encode('utf-8')
-
-    import random
-    random = "%030x" % random.randrange(16**30)
-
-    import urllib.parse, urllib.request, re
-
-    # try the URL from nook for PC
-    fetch_url = "https://cart4.barnesandnoble.com/services/service.aspx?Version=2&acctPassword="
-    fetch_url += urllib.parse.quote(password,'')+"&devID=PC_BN_2.5.6.9575_"+random+"&emailAddress="
-    fetch_url += urllib.parse.quote(email,"")+"&outFormat=5&schema=1&service=1&stage=deviceHashB"
-    #print fetch_url
-
-    found = ''
-    try:
-        response = urllib.request.urlopen(fetch_url)
-        the_page = response.read()
-        #print the_page
-        found = re.search('ccHash>(.+?)</ccHash', the_page).group(1)
-    except:
-        found = ''
-    if len(found)!=28:
-        # try the URL from android devices
-        fetch_url = "https://cart4.barnesandnoble.com/services/service.aspx?Version=2&acctPassword="
-        fetch_url += urllib.parse.quote(password,'')+"&devID=hobbes_9.3.50818_"+random+"&emailAddress="
-        fetch_url += urllib.parse.quote(email,"")+"&outFormat=5&schema=1&service=1&stage=deviceHashB"
-        #print fetch_url
-
-        found = ''
-        try:
-            response = urllib.request.urlopen(fetch_url)
-            the_page = response.read()
-            #print the_page
-            found = re.search('ccHash>(.+?)</ccHash', the_page).group(1)
-        except:
-            found = ''
-
-    return found
-
-
-
-
-def cli_main():
-    sys.stdout=SafeUnbuffered(sys.stdout)
-    sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
-    progname = os.path.basename(argv[0])
-    if len(argv) != 4:
-        print("usage: {0} <email> <password> <keyfileout.b64>".format(progname))
-        return 1
-    email, password, keypath = argv[1:]
-    userkey = fetch_key(email, password)
-    if len(userkey) == 28:
-        open(keypath,'wb').write(userkey)
-        return 0
-    print("Failed to fetch key.")
-    return 1
-
-
-def gui_main():
-    try:
-        import tkinter
-        import tkinter.filedialog
-        import tkinter.constants
-        import tkinter.messagebox
-        import traceback
-    except:
-        return cli_main()
-
-    class DecryptionDialog(tkinter.Frame):
-        def __init__(self, root):
-            tkinter.Frame.__init__(self, root, border=5)
-            self.status = tkinter.Label(self, text="Enter parameters")
-            self.status.pack(fill=tkinter.constants.X, expand=1)
-            body = tkinter.Frame(self)
-            body.pack(fill=tkinter.constants.X, expand=1)
-            sticky = tkinter.constants.E + tkinter.constants.W
-            body.grid_columnconfigure(1, weight=2)
-            tkinter.Label(body, text="Account email address").grid(row=0)
-            self.name = tkinter.Entry(body, width=40)
-            self.name.grid(row=0, column=1, sticky=sticky)
-            tkinter.Label(body, text="Account password").grid(row=1)
-            self.ccn = tkinter.Entry(body, width=40)
-            self.ccn.grid(row=1, column=1, sticky=sticky)
-            tkinter.Label(body, text="Output file").grid(row=2)
-            self.keypath = tkinter.Entry(body, width=40)
-            self.keypath.grid(row=2, column=1, sticky=sticky)
-            self.keypath.insert(2, "bnepubkey.b64")
-            button = tkinter.Button(body, text="...", command=self.get_keypath)
-            button.grid(row=2, column=2)
-            buttons = tkinter.Frame(self)
-            buttons.pack()
-            botton = tkinter.Button(
-                buttons, text="Fetch", width=10, command=self.generate)
-            botton.pack(side=tkinter.constants.LEFT)
-            tkinter.Frame(buttons, width=10).pack(side=tkinter.constants.LEFT)
-            button = tkinter.Button(
-                buttons, text="Quit", width=10, command=self.quit)
-            button.pack(side=tkinter.constants.RIGHT)
-
-        def get_keypath(self):
-            keypath = tkinter.filedialog.asksaveasfilename(
-                parent=None, title="Select B&N ePub key file to produce",
-                defaultextension=".b64",
-                filetypes=[('base64-encoded files', '.b64'),
-                           ('All Files', '.*')])
-            if keypath:
-                keypath = os.path.normpath(keypath)
-                self.keypath.delete(0, tkinter.constants.END)
-                self.keypath.insert(0, keypath)
-            return
-
-        def generate(self):
-            email = self.name.get()
-            password = self.ccn.get()
-            keypath = self.keypath.get()
-            if not email:
-                self.status['text'] = "Email address not given"
-                return
-            if not password:
-                self.status['text'] = "Account password not given"
-                return
-            if not keypath:
-                self.status['text'] = "Output keyfile path not set"
-                return
-            self.status['text'] = "Fetching..."
-            try:
-                userkey = fetch_key(email, password)
-            except Exception as e:
-                self.status['text'] = "Error: {0}".format(e.args[0])
-                return
-            if len(userkey) == 28:
-                open(keypath,'wb').write(userkey)
-                self.status['text'] = "Keyfile fetched successfully"
-            else:
-                self.status['text'] = "Keyfile fetch failed."
-
-    root = tkinter.Tk()
-    root.title("Barnes & Noble ePub Keyfile Fetch v.{0}".format(__version__))
-    root.resizable(True, False)
-    root.minsize(300, 0)
-    DecryptionDialog(root).pack(fill=tkinter.constants.X, expand=1)
-    root.mainloop()
-    return 0
-
-if __name__ == '__main__':
-    if len(sys.argv) > 1:
-        sys.exit(cli_main())
-    sys.exit(gui_main())

DeDRM_plugin/ineptepub.py

@@ -48,6 +48,7 @@ import base64
 import zlib
 import zipfile
 from zipfile import ZipInfo, ZipFile, ZIP_STORED, ZIP_DEFLATED
+from zeroedzipinfo import ZeroedZipInfo
 from contextlib import closing
 from lxml import etree
 from uuid import UUID
@@ -69,69 +70,9 @@ def unpad(data, padding=16):
 
     return data[:-pad_len]
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
-try:
-    from calibre.constants import iswindows, isosx
-except:
-    iswindows = sys.platform.startswith('win')
-    isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.
-
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        return ["ineptepub.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
+from argv_utils import unicode_argv
 
 
 class ADEPTError(Exception):
@@ -148,15 +89,16 @@ class Decryptor(object):
     def __init__(self, bookkey, encryption):
         enc = lambda tag: '{%s}%s' % (NSMAP['enc'], tag)
         self._aes = AES.new(bookkey, AES.MODE_CBC, b'\x00'*16)
-        encryption = etree.fromstring(encryption)
+        self._encryption = etree.fromstring(encryption)
         self._encrypted = encrypted = set()
+        self._encryptedForceNoDecomp = encryptedForceNoDecomp = set()
         self._otherData = otherData = set()
 
         self._json_elements_to_remove = json_elements_to_remove = set()
         self._has_remaining_xml = False
         expr = './%s/%s/%s' % (enc('EncryptedData'), enc('CipherData'),
                                enc('CipherReference'))
-        for elem in encryption.findall(expr):
+        for elem in self._encryption.findall(expr):
             path = elem.get('URI', None)
             encryption_type_url = (elem.getparent().getparent().find("./%s" % (enc('EncryptionMethod'))).get('Algorithm', None))
             if path is not None:
@@ -165,6 +107,11 @@ class Decryptor(object):
                     path = path.encode('utf-8')
                     encrypted.add(path)
                     json_elements_to_remove.add(elem.getparent().getparent())
+                elif (encryption_type_url == "http://ns.adobe.com/adept/xmlenc#aes128-cbc-uncompressed"):
+                    # Adobe uncompressed, for stuff like video files
+                    path = path.encode('utf-8')
+                    encryptedForceNoDecomp.add(path)
+                    json_elements_to_remove.add(elem.getparent().getparent())
                 else:
                     path = path.encode('utf-8')
                     otherData.add(path)
@@ -193,14 +140,15 @@ class Decryptor(object):
         return decompressed_bytes
 
     def decrypt(self, path, data):
-        if path.encode('utf-8') in self._encrypted:
+        if path.encode('utf-8') in self._encrypted or path.encode('utf-8') in self._encryptedForceNoDecomp:
             data = self._aes.decrypt(data)[16:]
             if type(data[-1]) != int:
                 place = ord(data[-1])
             else:
                 place = data[-1]
             data = data[:-place]
-            data = self.decompress(data)
+            if not path.encode('utf-8') in self._encryptedForceNoDecomp:
+                data = self.decompress(data)
         return data
 
 # check file to make check whether it's probably an Adobe Adept encrypted ePub
@@ -302,7 +250,7 @@ def decryptBook(userkey, inpath, outpath):
 
             if len(bookkey) != 64:
                 # Normal or "hardened" Adobe ADEPT
-                rsakey = RSA.import_key(userkey) # parses the ASN1 structure
+                rsakey = RSA.importKey(userkey) # parses the ASN1 structure
                 bookkey = base64.b64decode(bookkey)
                 if int(keytype, 10) > 2:
                     bookkey = removeHardening(rights, keytype, bookkey)
@@ -356,12 +304,23 @@ def decryptBook(userkey, inpath, outpath):
                         zi.internal_attr = oldzi.internal_attr
                         # external attributes are dependent on the create system, so copy both.
                         zi.external_attr = oldzi.external_attr
+
+                        zi.volume = oldzi.volume
                         zi.create_system = oldzi.create_system
+                        zi.create_version = oldzi.create_version
+
                         if any(ord(c) >= 128 for c in path) or any(ord(c) >= 128 for c in zi.comment):
                             # If the file name or the comment contains any non-ASCII char, set the UTF8-flag
                             zi.flag_bits |= 0x800
                     except:
                         pass
+
+                    # Python 3 has a bug where the external_attr is reset to `0o600 << 16`
+                    # if it's NULL, so we need a workaround:
+                    if zi.external_attr == 0: 
+                        zi = ZeroedZipInfo(zi)
+
+
                     if path == "META-INF/encryption.xml":
                         outf.writestr(zi, data)
                     else:
@@ -375,7 +334,7 @@ def decryptBook(userkey, inpath, outpath):
 def cli_main():
     sys.stdout=SafeUnbuffered(sys.stdout)
     sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
+    argv=unicode_argv("ineptepub.py")
     progname = os.path.basename(argv[0])
     if len(argv) != 4:
         print("usage: {0} <keyfile.der> <inbook.epub> <outbook.epub>".format(progname))

DeDRM_plugin/ineptpdf.py

@@ -51,13 +51,14 @@
 #   9.1.1 - Only support PyCryptodome; clean up the code
 #   10.0.0 - Add support for "hardened" Adobe DRM (RMSDK >= 10)
 #   10.0.2 - Fix some Python2 stuff
+#   10.0.4 - Fix more Python2 stuff
 
 """
 Decrypts Adobe ADEPT-encrypted PDF files.
 """
 
 __license__ = 'GPL v3'
-__version__ = "10.0.2"
+__version__ = "10.0.4"
 
 import codecs
 import hashlib
@@ -92,67 +93,12 @@ def unpad(data, padding=16):
     return data[:-pad_len]
 
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
 iswindows = sys.platform.startswith('win')
 isosx = sys.platform.startswith('darwin')
 
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.
-
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        return ["ineptpdf.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
-
+from argv_utils import unicode_argv
 
 class ADEPTError(Exception):
     pass
@@ -171,7 +117,7 @@ def SHA256(message):
 # 1 = only if present in input
 # 2 = always
 
-GEN_XREF_STM = 0
+GEN_XREF_STM = 1
 
 # This is the value for the current document
 gen_xref_stm = False # will be set in PDFSerializer
@@ -200,7 +146,10 @@ def nunpack(s, default=0):
     elif l == 2:
         return struct.unpack('>H', s)[0]
     elif l == 3:
-        return struct.unpack('>L', bytes([0]) + s)[0]
+        if sys.version_info[0] == 2:
+            return struct.unpack('>L', '\x00'+s)[0]
+        else: 
+            return struct.unpack('>L', bytes([0]) + s)[0]
     elif l == 4:
         return struct.unpack('>L', s)[0]
     else:
@@ -321,6 +270,11 @@ END_STRING = re.compile(br'[()\\]')
 OCT_STRING = re.compile(br'[0-7]')
 ESC_STRING = { b'b':8, b't':9, b'n':10, b'f':12, b'r':13, b'(':40, b')':41, b'\\':92 }
 
+class EmptyArrayValue(object):
+    def __str__(self):
+        return "<>"
+
+
 class PSBaseParser(object):
 
     '''
@@ -459,7 +413,10 @@ class PSBaseParser(object):
             self.hex += c
             return (self.parse_literal_hex, i+1)
         if self.hex:
-            self.token += bytes([int(self.hex, 16)])
+            if sys.version_info[0] == 2: 
+                self.token += chr(int(self.hex, 16))
+            else: 
+                self.token += bytes([int(self.hex, 16)])
         return (self.parse_literal, i)
 
     def parse_number(self, s, i):
@@ -543,10 +500,18 @@ class PSBaseParser(object):
             self.oct += c
             return (self.parse_string_1, i+1)
         if self.oct:
-            self.token += bytes([int(self.oct, 8)])
+            if sys.version_info[0] == 2:
+                self.token += chr(int(self.oct, 8))
+            else: 
+                self.token += bytes([int(self.oct, 8)])  
             return (self.parse_string, i)
         if c in ESC_STRING:
-            self.token += bytes([ESC_STRING[c]])
+
+            if sys.version_info[0] == 2:
+                self.token += chr(ESC_STRING[c])
+            else: 
+                self.token += bytes([ESC_STRING[c]])
+                
         return (self.parse_string, i+1)
 
     def parse_wopen(self, s, i):
@@ -559,6 +524,13 @@ class PSBaseParser(object):
         if c == b'<':
             self.add_token(KEYWORD_DICT_BEGIN)
             i += 1
+        if c == b'>':
+            # Empty array without any contents. Why though?
+            # We need to add some dummy python object that will serialize to 
+            # nothing, otherwise the code removes the whole array.
+            self.add_token(EmptyArrayValue())
+            i += 1
+
         return (self.parse_main, i)
 
     def parse_wclose(self, s, i):
@@ -572,13 +544,17 @@ class PSBaseParser(object):
         return (self.parse_main, i)
 
     def parse_hexstring(self, s, i):
-        m1 = END_HEX_STRING.search(s, i)
-        if not m1:
+        m = END_HEX_STRING.search(s, i)
+        if not m:
             self.token += s[i:]
             return (self.parse_hexstring, len(s))
-        j = m1.start(0)
+        j = m.start(0)
         self.token += s[i:j]
-        token = HEX_PAIR.sub(lambda m2: bytes([int(m2.group(0), 16)]),
+        if sys.version_info[0] == 2:
+            token = HEX_PAIR.sub(lambda m: chr(int(m.group(0), 16)),
+                                                 SPC.sub('', self.token))
+        else: 
+            token = HEX_PAIR.sub(lambda m: bytes([int(m.group(0), 16)]),
                                                  SPC.sub(b'', self.token))
         self.add_token(token)
         return (self.parse_main, j)
@@ -600,7 +576,11 @@ class PSBaseParser(object):
         while 1:
             self.fillbuf()
             if eol:
-                c = bytes([self.buf[self.charpos]])
+                if sys.version_info[0] == 2: 
+                    c = self.buf[self.charpos]
+                else: 
+                    c = bytes([self.buf[self.charpos]])
+
                 # handle '\r\n'
                 if c == b'\n':
                     linebuf += c
@@ -610,10 +590,17 @@ class PSBaseParser(object):
             if m:
                 linebuf += self.buf[self.charpos:m.end(0)]
                 self.charpos = m.end(0)
-                if bytes([linebuf[-1]]) == b'\r':
-                    eol = True
-                else:
-                    break
+                if sys.version_info[0] == 2:
+                    if linebuf[-1] == b'\r':
+                        eol = True
+                    else:
+                        break
+                else: 
+                    if bytes([linebuf[-1]]) == b'\r':
+                        eol = True
+                    else:
+                        break
+                    
             else:
                 linebuf += self.buf[self.charpos:]
                 self.charpos = len(self.buf)
@@ -989,9 +976,14 @@ class PDFStream(PDFObject):
                     for i in range(0, len(data), columns+1):
                         pred = data[i]
                         ent1 = data[i+1:i+1+columns]
-                        if pred == 2:
-                            ent1 = b''.join(bytes([(a+b) & 255]) \
-                                           for (a,b) in zip(ent0,ent1))
+                        if sys.version_info[0] == 2:
+                            if pred == '\x02':
+                                ent1 = ''.join(chr((ord(a)+ord(b)) & 255) \
+                                               for (a,b) in zip(ent0,ent1))
+                        else: 
+                            if pred == 2:
+                                ent1 = b''.join(bytes([(a+b) & 255]) \
+                                            for (a,b) in zip(ent0,ent1))
                         buf += ent1
                         ent0 = ent1
                     data = buf
@@ -1439,7 +1431,10 @@ class PDFDocument(object):
         x = ARC4.new(hash).decrypt(Odata) # 4
         if R >= 3:
             for i in range(1,19+1):
-                k = b''.join(bytes([c ^ i]) for c in hash )
+                if sys.version_info[0] == 2:
+                    k = b''.join(chr(ord(c) ^ i) for c in hash )
+                else: 
+                    k = b''.join(bytes([c ^ i]) for c in hash )
                 x = ARC4.new(k).decrypt(x)
 
 
@@ -1497,7 +1492,10 @@ class PDFDocument(object):
             hash.update(docid[0]) # 3
             x = ARC4.new(key).decrypt(hash.digest()[:16]) # 4
             for i in range(1,19+1):
-                k = b''.join(bytes([c ^ i]) for c in key )
+                if sys.version_info[0] == 2:
+                    k = b''.join(chr(ord(c) ^ i) for c in key )
+                else: 
+                    k = b''.join(bytes([c ^ i]) for c in key )
                 x = ARC4.new(k).decrypt(x)
             u1 = x+x # 32bytes total
         if R == 2:
@@ -1525,8 +1523,8 @@ class PDFDocument(object):
 
         # check owner pass:
         retval = self.check_owner_password(password, docid, param)
-        if retval is True or retval is not None:
-            #print("Owner pass is valid - " + str(retval))
+        if retval is True or (retval is not False and retval is not None):
+            #print("Owner pass is valid")
             if retval is True:
                 self.decrypt_key = self.recover_encryption_key_with_password(password, docid, param)
             else:
@@ -1535,7 +1533,7 @@ class PDFDocument(object):
         if self.decrypt_key is None or self.decrypt_key is True or self.decrypt_key is False:
             # That's not the owner password. Check if it's the user password.
             retval = self.check_user_password(password, docid, param)
-            if retval is True or retval is not None:
+            if retval is True or (retval is not False and retval is not None):
                 #print("User pass is valid")
                 if retval is True:
                     self.decrypt_key = self.recover_encryption_key_with_password(password, docid, param)
@@ -1604,7 +1602,13 @@ class PDFDocument(object):
 
     def initialize_ebx_ignoble(self, keyb64, docid, param):
         self.is_printable = self.is_modifiable = self.is_extractable = True
-        key = keyb64.decode('base64')[:16]
+
+        try: 
+            key = keyb64.decode('base64')[:16]
+            # This will probably always error, but I'm not 100% sure, so lets leave the old code in.
+        except AttributeError: 
+            key = codecs.decode(keyb64.encode("ascii"), 'base64')[:16]
+
 
         length = int_value(param.get('Length', 0)) / 8
         rights = codecs.decode(str_value(param.get('ADEPT_LICENSE')), "base64")
@@ -1632,13 +1636,15 @@ class PDFDocument(object):
             else:
                 print("ebx_V is %d  and ebx_type is %d" % (ebx_V, ebx_type))
                 print("length is %d and len(bookkey) is %d" % (length, len(bookkey)))
-                print("bookkey[0] is %d" % bookkey[0])
+                if len(bookkey) > 0:
+                    print("bookkey[0] is %d" % bookkey[0])
                 raise ADEPTError('error decrypting book session key - mismatched length')
         else:
             # proper length unknown try with whatever you have
             print("ebx_V is %d  and ebx_type is %d" % (ebx_V, ebx_type))
             print("length is %d and len(bookkey) is %d" % (length, len(bookkey)))
-            print("bookkey[0] is %d" % ord(bookkey[0]))
+            if len(bookkey) > 0:
+                print("bookkey[0] is %d" % ord(bookkey[0]))
             if ebx_V == 3:
                 V = 3
             else:
@@ -1669,7 +1675,7 @@ class PDFDocument(object):
 
     def initialize_ebx_inept(self, password, docid, param):
         self.is_printable = self.is_modifiable = self.is_extractable = True
-        rsakey = RSA.import_key(password) # parses the ASN1 structure
+        rsakey = RSA.importKey(password) # parses the ASN1 structure
         length = int_value(param.get('Length', 0)) // 8
         rights = codecs.decode(param.get('ADEPT_LICENSE'), 'base64')
         rights = zlib.decompress(rights, -15)
@@ -1704,13 +1710,15 @@ class PDFDocument(object):
             else:
                 print("ebx_V is %d  and ebx_type is %d" % (ebx_V, ebx_type))
                 print("length is %d and len(bookkey) is %d" % (length, len(bookkey)))
-                print("bookkey[0] is %d" % bookkey[0])
+                if len(bookkey) > 0:
+                    print("bookkey[0] is %d" % bookkey[0])
                 raise ADEPTError('error decrypting book session key - mismatched length')
         else:
             # proper length unknown try with whatever you have
             print("ebx_V is %d  and ebx_type is %d" % (ebx_V, ebx_type))
             print("length is %d and len(bookkey) is %d" % (length, len(bookkey)))
-            print("bookkey[0] is %d" % bookkey[0])
+            if len(bookkey) > 0:
+                print("bookkey[0] is %d" % bookkey[0])
             if ebx_V == 3:
                 V = 3
             else:
@@ -1758,7 +1766,11 @@ class PDFDocument(object):
         data = data[16:]
         plaintext = AES.new(key,AES.MODE_CBC,ivector).decrypt(data)
         # remove pkcs#5 aes padding
-        cutter = -1 * plaintext[-1]
+        if sys.version_info[0] == 2: 
+            cutter = -1 * ord(plaintext[-1])
+        else: 
+            cutter = -1 * plaintext[-1]
+
         plaintext = plaintext[:cutter]
         return plaintext
 
@@ -1819,7 +1831,19 @@ class PDFDocument(object):
                 try:
                     obj = objs[i]
                 except IndexError:
-                    raise PDFSyntaxError('Invalid object number: objid=%r' % (objid))
+                    # This IndexError used to just raise an exception.
+                    # Unfortunately that seems to break some PDFs, see this issue:
+                    # https://github.com/noDRM/DeDRM_tools/issues/233
+                    # I'm not sure why this is the case, but lets try only raising that exception
+                    # when in STRICT mode, and make it a warning otherwise.
+                    if STRICT:
+                        raise PDFSyntaxError('Invalid object number: objid=%r' % (objid))
+
+                    print('Invalid object number: objid=%r' % (objid))
+                    print("Continuing anyways?")
+                    print("If the resulting PDF is corrupted, please open a bug report.")
+                    return None
+
                 if isinstance(obj, PDFStream):
                     obj.set_objid(objid, 0)
             else:
@@ -2234,7 +2258,7 @@ class PDFSerializer(object):
         elif isinstance(obj, bytearray):
             self.write(b'(%s)' % self.escape_string(obj))
         elif isinstance(obj, bytes):
-            self.write(b'(%s)' % self.escape_string(obj))
+            self.write(b'<%s>' % binascii.hexlify(obj).upper())
         elif isinstance(obj, str):
             self.write(b'(%s)' % self.escape_string(obj.encode('utf-8')))
         elif isinstance(obj, bool):
@@ -2261,6 +2285,20 @@ class PDFSerializer(object):
                 self.write(b'(deleted)')
             else:
                 data = obj.get_decdata()
+
+                # Fix length:
+                # We've decompressed and then recompressed the PDF stream.
+                # Depending on the algorithm, the implementation, and the compression level, 
+                # the resulting recompressed stream is unlikely to have the same length as the original.
+                # So we need to update the PDF object to contain the new proper length.
+
+                # Without this change, all PDFs exported by this plugin are slightly corrupted - 
+                # even though most if not all PDF readers can correct that on-the-fly.
+
+                if 'Length' in obj.dic: 
+                    obj.dic['Length'] = len(data)
+
+
                 self.serialize_object(obj.dic)
                 self.write(b'stream\n')
                 self.write(data)
@@ -2307,7 +2345,7 @@ def getPDFencryptionType(inpath):
 def cli_main():
     sys.stdout=SafeUnbuffered(sys.stdout)
     sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
+    argv=unicode_argv("ineptpdf.py")
     progname = os.path.basename(argv[0])
     if len(argv) != 4:
         print("usage: {0} <keyfile.der> <inbook.pdf> <outbook.pdf>".format(progname))

DeDRM_plugin/ion.py

@@ -57,6 +57,7 @@ except ImportError:
                 # Windows-friendly choice: pylzma wheels
                 import pylzma as lzma
 
+from kfxtables import *
 
 TID_NULL = 0
 TID_BOOLEAN = 1
@@ -769,6 +770,7 @@ def pkcs7unpad(msg, blocklen):
 
 
 # every VoucherEnvelope version has a corresponding "word" and magic number, used in obfuscating the shared secret
+# 4-digit versions use their own obfuscation/scramble. It does not seem to depend on the "word" and number
 OBFUSCATION_TABLE = {
     "V1":    (0x00, None),
     "V2":    (0x05, b'Antidisestablishmentarianism'),
@@ -779,26 +781,26 @@ OBFUSCATION_TABLE = {
     "V7":    (0x05, b'\x10\x1bJ\x18\nh!\x10"\x03>Z\'\r\x01]W\x06\x1c\x1e?\x0f\x13'),
     "V8":    (0x09, b"K\x0c6\x1d\x1a\x17pO}Rk\x1d'w1^\x1f$\x1c{C\x02Q\x06\x1d`"),
     "V9":    (0x05, b'X.\x0eW\x1c*K\x12\x12\t\n\n\x17Wx\x01\x02Yf\x0f\x18\x1bVXPi\x01'),
-    "V10":   (0x07, b'z3\n\x039\x12\x13`\x06=v,\x02MTK\x1e%}L\x1c\x1f\x15\x0c\x11\x02\x0c\n8\x17p'),
+    "V10":   (0x07, b'z3\n\x039\x12\x13`\x06=v;\x02MTK\x1e%}L\x1c\x1f\x15\x0c\x11\x02\x0c\n8\x17p'),
     "V11":   (0x05, b'L=\nhVm\x07go\n6\x14\x06\x16L\r\x02\x0b\x0c\x1b\x04#p\t'),
-    "V12":   (0x06, b',n\x1d\rl\x13\x1c\x13\x16p\x14\x07U\x0c\x1f\x19w\x16\x16\x1d5T'),
+    "V12":   (0x06, b';n\x1d\rl\x13\x1c\x13\x16p\x14\x07U\x0c\x1f\x19w\x16\x16\x1d5T'),
     "V13":   (0x07, b'I\x05\t\x08\x03r)\x01$N\x0fr3n\x0b062D\x0f\x13'),
     "V14":   (0x05, b"\x03\x02\x1c9\x19\x15\x15q\x1057\x08\x16\x0cF\x1b.Fw\x01\x12\x03\x13\x02\x17S'hk6"),
     "V15":   (0x0A, b'&,4B\x1dcI\x0bU\x03I\x07\x04\x1c\t\x05c\x07%ws\x0cj\t\x1a\x08\x0f'),
-    "V16":   (0x0A, b'\x06\x18`h,b><\x06PqR\x02Zc\x034\n\x16\x1e\x18\x06#e'),
+    "V16":   (0x0A, b'\x06\x18`h;b><\x06PqR\x02Zc\x034\n\x16\x1e\x18\x06#e'),
     "V17":   (0x07, b'y\r\x12\x08fw.[\x02\t\n\x13\x11\x0c\x11b\x1e8L\x10(\x13<Jx6c\x0f'),
-    "V18":   (0x07, b'I\x0b\x0e,\x19\x1aIa\x10s\x19g\\\x1b\x11!\x18yf\x0f\t\x1d7[bSp\x03'),
+    "V18":   (0x07, b'I\x0b\x0e;\x19\x1aIa\x10s\x19g\\\x1b\x11!\x18yf\x0f\t\x1d7[bSp\x03'),
     "V19":   (0x05, b'\n6>)N\x02\x188\x016s\x13\x14\x1b\x16jeN\n\x146\x04\x18\x1c\x0c\x19\x1f,\x02]'),
-    "V20":   (0x08, b'_\r\x01\x12]\\\x14*\x17i\x14\r\t!\x1e,~hZ\x12jK\x17\x1e*1'),
+    "V20":   (0x08, b'_\r\x01\x12]\\\x14*\x17i\x14\r\t!\x1e;~hZ\x12jK\x17\x1e*1'),
     "V21":   (0x07, b'e\x1d\x19|\ty\x1di|N\x13\x0e\x04\x1bj<h\x13\x15k\x12\x08=\x1f\x16~\x13l'),
     "V22":   (0x08, b'?\x17yi$k7Pc\tEo\x0c\x07\x07\t\x1f,*i\x12\x0cI0\x10I\x1a?2\x04'),
     "V23":   (0x08, b'\x16+db\x13\x04\x18\rc%\x14\x17\x0f\x13F\x0c[\t9\x1ay\x01\x1eH'),
-    "V24":   (0x06, b'|6\\\x1a\r\x10\nP\x07\x0fu\x1f\t,\rr`uv\\~55\x11]N'),
-    "V25":   (0x09, b'\x07\x14w\x1e,^y\x01:\x08\x07\x1fr\tU#j\x16\x12\x1eB\x04\x16=\x06fZ\x07\x02\x06'),
+    "V24":   (0x06, b'|6\\\x1a\r\x10\nP\x07\x0fu\x1f\t;\rr`uv\\~55\x11]N'),
+    "V25":   (0x09, b'\x07\x14w\x1e;^y\x01:\x08\x07\x1fr\tU#j\x16\x12\x1eB\x04\x16=\x06fZ\x07\x02\x06'),
     "V26":   (0x06, b'\x03IL\x1e"K\x1f\x0f\x1fp0\x01`X\x02z0`\x03\x0eN\x07'),
     "V27":   (0x07, b'Xk\x10y\x02\x18\x10\x17\x1d,\x0e\x05e\x10\x15"e\x0fh(\x06s\x1c\x08I\x0c\x1b\x0e'),
     "V28":   (0x0A, b'6P\x1bs\x0f\x06V.\x1cM\x14\x02\n\x1b\x07{P0:\x18zaU\x05'),
-    "V9708": (0x05, b'\x1diIm\x08a\x17\x1e!am\x1d\x1aQ.\x16!\x06*\}x04\x11\t\x06\x04?'),
+    "V9708": (0x05, b'\x1diIm\x08a\x17\x1e!am\x1d\x1aQ.\x16!\x06*\x04\x11\t\x06\x04?'),
     "V1031": (0x08, b'Antidisestablishmentarianism'),
     "V2069": (0x07, b'Floccinaucinihilipilification'),
     "V9041": (0x06, b'>\x14\x0c\x12\x10-\x13&\x18U\x1d\x05Rlt\x03!\x19\x1b\x13\x04]Y\x19,\t\x1b'),
@@ -807,10 +809,367 @@ OBFUSCATION_TABLE = {
     "V9479": (0x09, b'\x10\x1bJ\x18\nh!\x10"\x03>Z\'\r\x01]W\x06\x1c\x1e?\x0f\x13'),
     "V9888": (0x05, b"K\x0c6\x1d\x1a\x17pO}Rk\x1d'w1^\x1f$\x1c{C\x02Q\x06\x1d`"),
     "V4648": (0x07, b'X.\x0eW\x1c*K\x12\x12\t\n\n\x17Wx\x01\x02Yf\x0f\x18\x1bVXPi\x01'),
-    "V5683": (0x05, b'z3\n\x039\x12\x13`\x06=v,\x02MTK\x1e%}L\x1c\x1f\x15\x0c\x11\x02\x0c\n8\x17p'),
+    "V5683": (0x05, b'z3\n\x039\x12\x13`\x06=v;\x02MTK\x1e%}L\x1c\x1f\x15\x0c\x11\x02\x0c\n8\x17p'),
 }
 
 
+#common str:  "PIDv3AESAES/CBC/PKCS5PaddingHmacSHA256"
+class workspace(object):
+  def __init__(self,initial_list):
+    self.work=initial_list
+  def shuffle(self,shuflist):
+    ll=len(shuflist)
+    rt=[]
+    for i in range(ll):
+      rt.append(self.work[shuflist[i]])
+    self.work=rt
+  def sbox(self,table,matrix,skplist=[]): #table is list of 4-byte integers
+    offset=0
+    nwork=list(self.work)
+    wo=0
+    toff=0
+    while offset<0x6000:
+      uv5=table[toff+nwork[wo+0]]
+      uv1=table[toff+nwork[wo+1]+0x100]
+      uv2=table[toff+nwork[wo+2]+0x200]
+      uv3=table[toff+nwork[wo+3]+0x300]
+      moff=0
+      if 0 in skplist:
+        moff+=0x400
+      else:
+        nib1=matrix[moff+offset+(uv1>>0x1c)|( (uv5>>0x18)&0xf0)]
+        moff+=0x100
+        nib2=matrix[moff+offset+(uv3>>0x1c)|( (uv2>>0x18)&0xf0)]
+        moff+=0x100
+        nib3=matrix[moff+offset+((uv1>>0x18)&0xf) |( (uv5>>0x14)&0xf0)]
+        moff+=0x100
+        nib4=matrix[moff+offset+((uv3>>0x18)&0xf) |( (uv2>>0x14)&0xf0)]
+        moff+=0x100
+      rnib1=matrix[moff+offset+nib1*0x10+nib2]
+      moff+=0x100
+      rnib2=matrix[moff+offset+nib3*0x10+nib4]
+      moff+=0x100
+      nwork[wo+0]=rnib1*0x10+rnib2
+      if 1 in skplist:
+        moff+=0x400
+      else:
+        nib1=matrix[moff+offset+((uv1>>0x14)&0xf)|( (uv5>>0x10)&0xf0)]
+        moff+=0x100
+        nib2=matrix[moff+offset+((uv3>>0x14)&0xf)|( (uv2>>0x10)&0xf0)]
+        moff+=0x100
+        nib3=matrix[moff+offset+((uv1>>0x10)&0xf) |( (uv5>>0xc)&0xf0)]
+        moff+=0x100
+        nib4=matrix[moff+offset+((uv3>>0x10)&0xf) |( (uv2>>0xc)&0xf0)]
+        moff+=0x100
+
+      rnib1=matrix[moff+offset+nib1*0x10+nib2]
+      moff+=0x100
+      rnib2=matrix[moff+offset+nib3*0x10+nib4]
+      moff+=0x100
+      nwork[wo+1]=rnib1*0x10+rnib2
+      if 2 in skplist:
+        moff+=0x400
+      else:
+        nib1=matrix[moff+offset+((uv1>>0xc)&0xf)|( (uv5>>0x8)&0xf0)]
+        moff+=0x100
+        nib2=matrix[moff+offset+((uv3>>0xc)&0xf)|( (uv2>>0x8)&0xf0)]
+        moff+=0x100
+        nib3=matrix[moff+offset+((uv1>>0x8)&0xf) |( (uv5>>0x4)&0xf0)]
+        moff+=0x100
+        nib4=matrix[moff+offset+((uv3>>0x8)&0xf) |( (uv2>>0x4)&0xf0)]
+        moff+=0x100
+      rnib1=matrix[moff+offset+nib1*0x10+nib2]
+      moff+=0x100
+      rnib2=matrix[moff+offset+nib3*0x10+nib4]
+      moff+=0x100
+      nwork[wo+2]=rnib1*0x10+rnib2
+      if 3 in skplist:
+        moff+=0x400
+      else:
+        nib1=matrix[moff+offset+((uv1>>0x4)&0xf)|( (uv5)&0xf0)]
+        moff+=0x100
+        nib2=matrix[moff+offset+((uv3>>0x4)&0xf)|( (uv2)&0xf0)]
+        moff+=0x100
+        nib3=matrix[moff+offset+((uv1)&0xf)|( (uv5<<4)&0xf0) ]
+        moff+=0x100
+        nib4=matrix[moff+offset+((uv3)&0xf)|( (uv2<<4)&0xf0) ]
+        moff+=0x100
+      ##############
+      rnib1=matrix[moff+offset+nib1*0x10+nib2]
+      moff+=0x100
+      rnib2=matrix[moff+offset+nib3*0x10+nib4]
+      moff+=0x100
+      nwork[wo+3]=rnib1*0x10+rnib2
+      offset = offset + 0x1800
+      wo+=4
+      toff+=0x400
+    self.work=nwork
+  def lookup(self,ltable):
+    for a in range(len(self.work)):
+      self.work[a]=ltable[a]
+  def exlookup(self,ltable):
+    lookoffs=0
+    for a in range(len(self.work)):
+      self.work[a]=ltable[self.work[a]+lookoffs]
+      lookoffs+=0x100
+  def mask(self, chunk):
+    out=[]
+    for a in range(len(chunk)):
+      self.work[a]=self.work[a]^chunk[a]
+      out.append(self.work[a])
+    return out
+
+def process_V9708(st):
+  #e9c457a7dae6aa24365e7ef219b934b17ed58ee7d5329343fc3aea7860ed51f9a73de14351c9
+  ws=workspace([0x11]*16)
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.shuffle(repl)
+    ws.sbox(d0x6a06ea70,d0x6a0dab50)
+    ws.sbox(d0x6a073a70,d0x6a0dab50)
+    ws.shuffle(repl)
+    ws.exlookup(d0x6a072a70)
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16;
+  return bytes(out)
+
+def process_V1031(st):
+  #d53efea7fdd0fda3e1e0ebbae87cad0e8f5ef413c471c3ae81f39222a9ec8b8ed582e045918c
+  ws=workspace([0x06,0x18,0x60,0x68,0x3b,0x62,0x3e,0x3c,0x06,0x50,0x71,0x52,0x02,0x5a,0x63,0x03])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0797c0,d0x6a0dab50,[3])
+    ws.sbox(d0x6a07e7c0,d0x6a0dab50,[3])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0797c0,d0x6a0dab50,[3])
+    ws.sbox(d0x6a07e7c0,d0x6a0dab50,[3])
+    ws.exlookup(d0x6a07d7c0)
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+    #break
+  return bytes(out)
+
+def process_V2069(st):
+  #8e6196d754a304c9354e91b5d79f07b048026d31c7373a8691e513f2c802c706742731caa858
+  ws=workspace([0x79,0x0d,0x12,0x08,0x66,0x77,0x2e,0x5b,0x02,0x09,0x0a,0x13,0x11,0x0c,0x11,0x62])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.sbox(d0x6a084498,d0x6a0dab50,[2])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a089498,d0x6a0dab50,[2])
+    ws.sbox(d0x6a089498,d0x6a0dab50,[2])
+    ws.sbox(d0x6a084498,d0x6a0dab50,[2])
+    ws.shuffle(repl)
+    ws.exlookup(d0x6a088498)
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+  return bytes(out)
+
+
+def process_V9041(st):
+  #11f7db074b24e560dfa6fae3252b383c3b936e51f6ded570dc936cb1da9f4fc4a97ec686e7d8
+  ws=workspace([0x49,0x0b,0x0e,0x3b,0x19,0x1a,0x49,0x61,0x10,0x73,0x19,0x67,0x5c,0x1b,0x11,0x21])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.sbox(d0x6a094170,d0x6a0dab50,[1])
+    ws.shuffle(repl)
+    ws.shuffle(repl)
+    ws.sbox(d0x6a08f170,d0x6a0dab50,[1])
+    ws.sbox(d0x6a08f170,d0x6a0dab50,[1])
+    ws.sbox(d0x6a094170,d0x6a0dab50,[1])
+
+    ws.exlookup(d0x6a093170)
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+    #break
+  return bytes(out)
+
+def process_V3646(st):
+  #d468aa362b44479282291983243b38197c4b4aa24c2c58e62c76ec4b81e08556ca0c54301664
+  ws=workspace([0x0a,0x36,0x3e,0x29,0x4e,0x02,0x18,0x38,0x01,0x36,0x73,0x13,0x14,0x1b,0x16,0x6a])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.shuffle(repl)
+    ws.sbox(d0x6a099e48,d0x6a0dab50,[2,3])
+    ws.sbox(d0x6a09ee48,d0x6a0dab50,[2,3])
+    ws.sbox(d0x6a09ee48,d0x6a0dab50,[2,3])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a099e48,d0x6a0dab50,[2,3])
+    ws.sbox(d0x6a099e48,d0x6a0dab50,[2,3])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a09ee48,d0x6a0dab50,[2,3])
+    ws.exlookup(d0x6a09de48)
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+  return bytes(out)
+
+
+def process_V6052(st):
+  #d683c8c4e4f46ae45812196f37e218eabce0fae08994f25fabb01d3e569b8bf3866b99d36f57
+  ws=workspace([0x5f,0x0d,0x01,0x12,0x5d,0x5c,0x14,0x2a,0x17,0x69,0x14,0x0d,0x09,0x21,0x1e,0x3b])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0a4b20,d0x6a0dab50,[1,3])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0a4b20,d0x6a0dab50,[1,3])
+    ws.sbox(d0x6a0a9b20,d0x6a0dab50,[1,3])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0a9b20,d0x6a0dab50,[1,3])
+    ws.sbox(d0x6a0a9b20,d0x6a0dab50,[1,3])
+    ws.sbox(d0x6a0a4b20,d0x6a0dab50,[1,3])
+
+    ws.exlookup(d0x6a0a8b20)
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+  return bytes(out)
+
+def process_V9479(st):
+  #925635db434bccd3f4791eb87b89d2dfc7c93be06e794744eb9de58e6d721e696980680ab551
+  ws=workspace([0x65,0x1d,0x19,0x7c,0x09,0x79,0x1d,0x69,0x7c,0x4e,0x13,0x0e,0x04,0x1b,0x6a,0x3c ])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.sbox(d0x6a0af7f8,d0x6a0dab50,[1,2,3])
+    ws.sbox(d0x6a0af7f8,d0x6a0dab50,[1,2,3])
+    ws.sbox(d0x6a0b47f8,d0x6a0dab50,[1,2,3])
+    ws.sbox(d0x6a0af7f8,d0x6a0dab50,[1,2,3])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0b47f8,d0x6a0dab50,[1,2,3])
+    ws.shuffle(repl)
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0b47f8,d0x6a0dab50,[1,2,3])
+    ws.exlookup(d0x6a0b37f8)
+
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+  return bytes(out)
+
+def process_V9888(st):
+  #54c470723f8c105ba0186b6319050869de673ce31a5ec15d4439921d4cd05c5e860cb2a41fea
+  ws=workspace([0x3f,0x17,0x79,0x69,0x24,0x6b,0x37,0x50,0x63,0x09,0x45,0x6f,0x0c,0x07,0x07,0x09])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.sbox(d0x6a0ba4d0,d0x6a0dab50,[1,2])
+    ws.sbox(d0x6a0bf4d0,d0x6a0dab50,[1,2])
+    ws.sbox(d0x6a0bf4d0,d0x6a0dab50,[1,2])
+    ws.sbox(d0x6a0ba4d0,d0x6a0dab50,[1,2])
+    ws.shuffle(repl)
+    ws.shuffle(repl)    
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0bf4d0,d0x6a0dab50,[1,2])
+    ws.sbox(d0x6a0ba4d0,d0x6a0dab50,[1,2])
+    ws.exlookup(d0x6a0be4d0)    
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+  return bytes(out)
+
+def process_V4648(st):
+  #705bd4cd8b61d4596ef4ca40774d68e71f1f846c6e94bd23fd26e5c127e0beaa650a50171f1b
+  ws=workspace([0x16,0x2b,0x64,0x62,0x13,0x04,0x18,0x0d,0x63,0x25,0x14,0x17,0x0f,0x13,0x46,0x0c])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.sbox(d0x6a0ca1a8,d0x6a0dab50,[1,3])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0ca1a8,d0x6a0dab50,[1,3])
+    ws.sbox(d0x6a0c51a8,d0x6a0dab50,[1,3])
+    ws.sbox(d0x6a0ca1a8,d0x6a0dab50,[1,3])
+    ws.sbox(d0x6a0c51a8,d0x6a0dab50,[1,3])
+    ws.sbox(d0x6a0c51a8,d0x6a0dab50,[1,3])
+    ws.shuffle(repl)
+    ws.shuffle(repl)
+    ws.exlookup(d0x6a0c91a8)    
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+  return bytes(out)
+
+def process_V5683(st):
+  #1f5af733423e5104afb9d5594e682ecf839a776257f33747c9beee671c57ab3f84943f69d8fd
+  ws=workspace([0x7c,0x36,0x5c,0x1a,0x0d,0x10,0x0a,0x50,0x07,0x0f,0x75,0x1f,0x09,0x3b,0x0d,0x72])
+  repl=[0,5,10,15,4,9,14,3,8,13,2,7,12,1,6,11]
+  remln=len(st)
+  sto=0
+  out=[]
+  while(remln>0):
+    ws.sbox(d0x6a0d4e80,d0x6a0dab50,[])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0cfe80,d0x6a0dab50,[])
+    ws.sbox(d0x6a0d4e80,d0x6a0dab50,[])
+    ws.sbox(d0x6a0cfe80,d0x6a0dab50,[])
+    ws.sbox(d0x6a0d4e80,d0x6a0dab50,[])
+    ws.shuffle(repl)
+    ws.sbox(d0x6a0cfe80,d0x6a0dab50,[])
+    ws.shuffle(repl)
+    ws.exlookup(d0x6a0d3e80)    
+    dat=ws.mask(st[sto:sto+16])
+    out+=dat
+    sto+=16
+    remln-=16
+  return bytes(out)
+
+
+# def a2hex(arr):
+#   ax=[]
+#   ha="0123456789abcdef"
+#   for a in arr:
+#     if a<0: a=256+a
+#     ax.append(ha[(a>>4)]+ha[a%16])
+#   return "".join(ax)
+# 
+# def memhex(adr,sz):
+#   emu=EmulatorHelper(currentProgram)
+#   arr=emu.readMemory(getAddress(adr),sz)
+#   return a2hex(arr)
+# 
+
+
+
+
 # obfuscate shared secret according to the VoucherEnvelope version
 def obfuscate(secret, version):
     if version == 1:  # v1 does not use obfuscation
@@ -835,6 +1194,107 @@ def obfuscate(secret, version):
     return obfuscated
 
 
+
+# scramble() and obfuscate2() from https://github.com/andrewc12/DeDRM_tools/commit/d9233d61f00d4484235863969919059f4d0b2057
+
+def scramble(st,magic):
+    ret=bytearray(len(st))
+    padlen=len(st)
+    for counter in range(len(st)):
+        ivar2=(padlen//2)-2*(counter%magic)+magic+counter-1
+        ret[ivar2%padlen]=st[counter]
+    return ret
+
+
+def obfuscate2(secret, version):
+    if version == 1:  # v1 does not use obfuscation
+        return secret
+    magic, word = OBFUSCATION_TABLE["V%d" % version]
+    # extend secret so that its length is divisible by the magic number
+    if len(secret) % magic != 0:
+        secret = secret + b'\x00' * (magic - len(secret) % magic)
+    obfuscated = bytearray(len(secret))
+    wordhash = bytearray(hashlib.sha256(word).digest()[16:])
+    #print(wordhash.hex())
+    shuffled = bytearray(scramble(secret,magic))
+    for i in range(0, len(secret)):
+        obfuscated[i] = shuffled[i] ^ wordhash[i % 16]
+    return obfuscated
+
+# scramble3() and obfuscate3() from https://github.com/Satsuoni/DeDRM_tools/commit/da6b6a0c911b6d45fe1b13042b690daebc1cc22f
+
+def scramble3(st,magic):
+  ret=bytearray(len(st))
+  padlen=len(st)
+  divs = padlen // magic
+  cntr = 0
+  iVar6 = 0
+  offset = 0
+  if (0 < ((magic - 1) + divs)):
+    while True:
+      if (offset & 1) == 0 :
+        uVar4 = divs - 1
+        if offset < divs:
+          iVar3 = 0
+          uVar4 = offset
+        else:
+          iVar3 = (offset - divs) + 1
+        if uVar4>=0:
+          iVar5 = uVar4 * magic
+          index =  ((padlen - 1) - cntr)
+          while True:
+            if (magic <= iVar3): break
+            ret[index] = st[iVar3 + iVar5]
+            iVar3 = iVar3 + 1
+            cntr = cntr + 1
+            uVar4 = uVar4 - 1
+            iVar5 = iVar5 - magic
+            index -= 1
+            if uVar4<=-1: break
+      else: 
+        if (offset < magic):
+          iVar3 = 0
+        else :
+          iVar3 = (offset - magic) + 1
+        if (iVar3 < divs):
+          uVar4 = offset
+          if (magic <= offset):
+            uVar4 = magic - 1
+
+          index = ((padlen - 1) - cntr)
+          iVar5 = iVar3 * magic
+          while True:
+            if (uVar4 < 0) : break
+            iVar3 += 1
+            ret[index] = st[uVar4 + iVar5]
+            uVar4 -= 1
+            index=index-1
+            iVar5 = iVar5 + magic;
+            cntr += 1;
+            if iVar3>=divs: break 
+      offset = offset + 1
+      if offset >= ((magic - 1) + divs) :break 
+  return ret
+
+#not sure if the third variant is used anywhere, but it is in Kindle, so I tried to add it
+def obfuscate3(secret, version):
+    if version == 1:  # v1 does not use obfuscation
+        return secret
+    magic, word = OBFUSCATION_TABLE["V%d" % version]
+    # extend secret so that its length is divisible by the magic number
+    if len(secret) % magic != 0:
+        secret = secret + b'\x00' * (magic - len(secret) % magic)
+    #secret = bytearray(secret)
+    obfuscated = bytearray(len(secret))
+    wordhash = bytearray(hashlib.sha256(word).digest())
+    #print(wordhash.hex())
+    shuffled=bytearray(scramble3(secret,magic))
+    #print(shuffled)
+    # shuffle secret and xor it with the first half of the word hash
+    for i in range(0, len(secret)):
+        obfuscated[i] = shuffled[i] ^ wordhash[i % 16]
+    return obfuscated
+
 class DrmIonVoucher(object):
     envelope = None
     version = None
@@ -878,18 +1338,34 @@ class DrmIonVoucher(object):
             else:
                 _assert(False, "Unknown lock parameter: %s" % param)
 
-        sharedsecret = obfuscate(shared, self.version)
 
-        key = hmac.new(sharedsecret, b"PIDv3", digestmod=hashlib.sha256).digest()
-        aes = AES.new(key[:32], AES.MODE_CBC, self.cipheriv[:16])
-        b = aes.decrypt(self.ciphertext)
-        b = pkcs7unpad(b, 16)
+        # i know that version maps to scramble pretty much 1 to 1, but there was precendent where they changed it, so... 
+        sharedsecrets = [obfuscate(shared, self.version),obfuscate2(shared, self.version),obfuscate3(shared, self.version),
+                         process_V9708(shared), process_V1031(shared), process_V2069(shared), process_V9041(shared),
+                         process_V3646(shared), process_V6052(shared), process_V9479(shared), process_V9888(shared),
+                         process_V4648(shared), process_V5683(shared)]
+        
+        decrypted=False
+        ex=None
+        for sharedsecret in sharedsecrets:
+            key = hmac.new(sharedsecret, b"PIDv3", digestmod=hashlib.sha256).digest()
+            aes = AES.new(key[:32], AES.MODE_CBC, self.cipheriv[:16])
+            try:
+                b = aes.decrypt(self.ciphertext)
+                b = pkcs7unpad(b, 16)
+                self.drmkey = BinaryIonParser(BytesIO(b))
+                addprottable(self.drmkey)
 
-        self.drmkey = BinaryIonParser(BytesIO(b))
-        addprottable(self.drmkey)
+                _assert(self.drmkey.hasnext() and self.drmkey.next() == TID_LIST and self.drmkey.gettypename() == "com.amazon.drm.KeySet@1.0",
+                    "Expected KeySet, got %s" % self.drmkey.gettypename())
+                decrypted=True 
 
-        _assert(self.drmkey.hasnext() and self.drmkey.next() == TID_LIST and self.drmkey.gettypename() == "com.amazon.drm.KeySet@1.0",
-                "Expected KeySet, got %s" % self.drmkey.gettypename())
+                print("Decryption succeeded")
+                break
+            except Exception as ex:
+                print("Decryption failed, trying next fallback ")
+        if not decrypted:
+            raise ex
 
         self.drmkey.stepin()
         while self.drmkey.hasnext():

DeDRM_plugin/k4mobidedrm.py

@@ -88,68 +88,10 @@ import kgenpids
 import androidkindlekey
 import kfxdedrm
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
-iswindows = sys.platform.startswith('win')
-isosx = sys.platform.startswith('darwin')
+from argv_utils import unicode_argv
 
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.
-
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["mobidedrm.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
 
 # cleanup unicode filenames
 # borrowed from calibre from calibre/src/calibre/__init__.py
@@ -311,7 +253,7 @@ def usage(progname):
 # Main
 #
 def cli_main():
-    argv=unicode_argv()
+    argv=unicode_argv("k4mobidedrm.py")
     progname = os.path.basename(argv[0])
     print("K4MobiDeDrm v{0}.\nCopyright © 2008-2020 Apprentice Harper et al.".format(__version__))
 

DeDRM_plugin/kindlekey.py

@@ -62,29 +62,7 @@ except NameError:
 
 # Routines common to Mac and PC
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
 try:
     from calibre.constants import iswindows, isosx
@@ -92,41 +70,7 @@ except:
     iswindows = sys.platform.startswith('win')
     isosx = sys.platform.startswith('darwin')
 
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.  So use shell32.GetCommandLineArgvW to get sys.argv
-        # as a list of Unicode strings and encode them as utf-8
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["kindlekey.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
+from argv_utils import unicode_argv
 
 class DrmException(Exception):
     pass
@@ -335,7 +279,10 @@ if iswindows:
         path = ""
         if 'LOCALAPPDATA' in os.environ.keys():
             # Python 2.x does not return unicode env. Use Python 3.x
-            path = winreg.ExpandEnvironmentStrings("%LOCALAPPDATA%")
+            if sys.version_info[0] == 2:
+                path = winreg.ExpandEnvironmentStrings(u"%LOCALAPPDATA%")
+            else:
+                path = winreg.ExpandEnvironmentStrings("%LOCALAPPDATA%")
             # this is just another alternative.
             # path = getEnvironmentVariable('LOCALAPPDATA')
             if not os.path.isdir(path):
@@ -989,7 +936,7 @@ def usage(progname):
 def cli_main():
     sys.stdout=SafeUnbuffered(sys.stdout)
     sys.stderr=SafeUnbuffered(sys.stderr)
-    argv=unicode_argv()
+    argv=unicode_argv("kindlekey.py")
     progname = os.path.basename(argv[0])
     print("{0} v{1}\nCopyright © 2010-2020 by some_updates, Apprentice Harper et al.".format(progname,__version__))
 
@@ -1050,7 +997,7 @@ def gui_main():
             self.text.insert(tkinter.constants.END, text)
 
 
-    argv=unicode_argv()
+    argv=unicode_argv("kindlekey.py")
     root = tkinter.Tk()
     root.withdraw()
     progpath, progname = os.path.split(argv[0])

DeDRM_plugin/kindlepid.py

@@ -16,68 +16,9 @@
 import sys
 import binascii
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
-iswindows = sys.platform.startswith('win')
-isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.
-
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["kindlepid.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
+from argv_utils import unicode_argv
 
 letters = 'ABCDEFGHIJKLMNPQRSTUVWXYZ123456789'
 
@@ -117,7 +58,7 @@ def pidFromSerial(s, l):
 
 def cli_main():
     print("Mobipocket PID calculator for Amazon Kindle. Copyright © 2007, 2009 Igor Skochinsky")
-    argv=unicode_argv()
+    argv=unicode_argv("kindlepid.py")
     if len(argv)==2:
         serial = argv[1]
     else:

DeDRM_plugin/mobidedrm.py

@@ -80,73 +80,11 @@ import sys
 import os
 import struct
 import binascii
-try:
-    from alfcrypto import Pukall_Cipher
-except:
-    print("AlfCrypto not found. Using python PC1 implementation.")
+from alfcrypto import Pukall_Cipher
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from utilities import SafeUnbuffered
 
-iswindows = sys.platform.startswith('win')
-isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.
-
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["mobidedrm.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
+from argv_utils import unicode_argv
 
 
 class DrmException(Exception):
@@ -162,41 +100,8 @@ def PC1(key, src, decryption=True):
     # if we can get it from alfcrypto, use that
     try:
         return Pukall_Cipher().PC1(key,src,decryption)
-    except NameError:
-        pass
-    except TypeError:
-        pass
-
-    # use slow python version, since Pukall_Cipher didn't load
-    sum1 = 0;
-    sum2 = 0;
-    keyXorVal = 0;
-    if len(key)!=16:
-         DrmException ("PC1: Bad key length")
-    wkey = []
-    for i in range(8):
-        wkey.append(key[i*2]<<8 | key[i*2+1])
-    dst = bytearray(len(src))
-    for i in range(len(src)):
-        temp1 = 0;
-        byteXorVal = 0;
-        for j in range(8):
-            temp1 ^= wkey[j]
-            sum2  = (sum2+j)*20021 + sum1
-            sum1  = (temp1*346)&0xFFFF
-            sum2  = (sum2+sum1)&0xFFFF
-            temp1 = (temp1*20021+1)&0xFFFF
-            byteXorVal ^= temp1 ^ sum2
-        curByte = src[i]
-        if not decryption:
-            keyXorVal = curByte * 257;
-        curByte = ((curByte ^ (byteXorVal >> 8)) ^ byteXorVal) & 0xFF
-        if decryption:
-            keyXorVal = curByte * 257;
-        for j in range(8):
-            wkey[j] ^= keyXorVal;
-        dst[i] = curByte
-    return bytes(dst)
+    except: 
+        raise
 
 # accepts unicode returns unicode
 def checksumPid(s):
@@ -254,12 +159,7 @@ class MobiBook:
         pass
 
     def __init__(self, infile):
-        print("MobiDeDrm v{0:s}.\nCopyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.".format(__version__))
-
-        try:
-            from alfcrypto import Pukall_Cipher
-        except:
-            print("AlfCrypto not found. Using python PC1 implementation.")
+        print("MobiDeDrm v{0:s}.\nCopyright © 2008-2022 The Dark Reverser, Apprentice Harper et al.".format(__version__))
 
         # initial sanity check on file
         self.data_file = open(infile, 'rb').read()
@@ -544,7 +444,7 @@ def getUnencryptedBook(infile,pidlist):
 
 
 def cli_main():
-    argv=unicode_argv()
+    argv=unicode_argv("mobidedrm.py")
     progname = os.path.basename(argv[0])
     if len(argv)<3 or len(argv)>4:
         print("MobiDeDrm v{0:s}.\nCopyright © 2008-2020 The Dark Reverser, Apprentice Harper et al.".format(__version__))

DeDRM_plugin/simpleprefs.py

@@ -1,81 +0,0 @@
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-# vim:ts=4:sw=4:softtabstop=4:smarttab:expandtab
-
-import sys
-import os, os.path
-import shutil
-
-class SimplePrefsError(Exception):
-    pass
-
-class SimplePrefs(object):
-    def __init__(self, target, description):
-        self.prefs = {}
-        self.key2file={}
-        self.file2key={}
-        for keyfilemap in description:
-            [key, filename] = keyfilemap
-            self.key2file[key] = filename
-            self.file2key[filename] = key
-        self.target = target + 'Prefs'
-        if sys.platform.startswith('win'):
-            try:
-                import winreg
-            except ImportError:
-                import _winreg as winreg
-            regkey = winreg.OpenKey(winreg.HKEY_CURRENT_USER, "Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Shell Folders\\")
-            path = winreg.QueryValueEx(regkey, 'Local AppData')[0]
-            prefdir = path + os.sep + self.target
-        elif sys.platform.startswith('darwin'):
-            home = os.getenv('HOME')
-            prefdir = os.path.join(home,'Library','Preferences','org.' + self.target)
-        else:
-            # linux and various flavors of unix
-            home = os.getenv('HOME')
-            prefdir = os.path.join(home,'.' + self.target)
-        if not os.path.exists(prefdir):
-            os.makedirs(prefdir)
-        self.prefdir = prefdir
-        self.prefs['dir'] = self.prefdir
-        self._loadPreferences()
-
-    def _loadPreferences(self):
-        filenames = os.listdir(self.prefdir)
-        for filename in filenames:
-            if filename in self.file2key:
-                key = self.file2key[filename]
-                filepath = os.path.join(self.prefdir,filename)
-                if os.path.isfile(filepath):
-                    try :
-                        data = file(filepath,'rb').read()
-                        self.prefs[key] = data
-                    except Exception as e:
-                        pass
-
-    def getPreferences(self):
-        return self.prefs
-
-    def setPreferences(self, newprefs={}):
-        if 'dir' not in newprefs:
-            raise SimplePrefsError('Error: Attempt to Set Preferences in unspecified directory')
-        if newprefs['dir'] != self.prefs['dir']:
-            raise SimplePrefsError('Error: Attempt to Set Preferences in unspecified directory')
-        for key in newprefs:
-            if key != 'dir':
-                if key in self.key2file:
-                    filename = self.key2file[key]
-                    filepath = os.path.join(self.prefdir,filename)
-                    data = newprefs[key]
-                    if data != None:
-                        data = str(data)
-                    if data == None or data == '':
-                        if os.path.exists(filepath):
-                            os.remove(filepath)
-                    else:
-                        try:
-                            file(filepath,'wb').write(data)
-                        except Exception as e:
-                            pass
-        self.prefs = newprefs
-        return

DeDRM_plugin/stylexml2css.py

@@ -181,7 +181,7 @@ class DocParser(object):
                                 print("Scale not defined!")
                                 scale = 1.0
 
-                            if val == "":
+                            if not val:
                                 val = 0
 
                             if not ((attr == b'hang') and (int(val) == 0)):

DeDRM_plugin/subasyncio.py

@@ -1,148 +0,0 @@
-#!/usr/bin/env python
-# vim:ts=4:sw=4:softtabstop=4:smarttab:expandtab
-
-import os, sys
-import signal
-import threading
-import subprocess
-from subprocess import Popen, PIPE, STDOUT
-
-# **heavily** chopped up and modfied version of asyncproc.py
-# to make it actually work on Windows as well as Mac/Linux
-# For the original see:
-# "http://www.lysator.liu.se/~bellman/download/"
-# author is  "Thomas Bellman <bellman@lysator.liu.se>"
-# available under GPL version 3 or Later
-
-# create an asynchronous subprocess whose output can be collected in
-# a non-blocking manner
-
-# What a mess!  Have to use threads just to get non-blocking io
-# in a cross-platform manner
-
-# luckily all thread use is hidden within this class
-
-class Process(object):
-    def __init__(self, *params, **kwparams):
-        if len(params) <= 3:
-            kwparams.setdefault('stdin', subprocess.PIPE)
-        if len(params) <= 4:
-            kwparams.setdefault('stdout', subprocess.PIPE)
-        if len(params) <= 5:
-            kwparams.setdefault('stderr', subprocess.PIPE)
-        self.__pending_input = []
-        self.__collected_outdata = []
-        self.__collected_errdata = []
-        self.__exitstatus = None
-        self.__lock = threading.Lock()
-        self.__inputsem = threading.Semaphore(0)
-        self.__quit = False
-
-        self.__process = subprocess.Popen(*params, **kwparams)
-
-        if self.__process.stdin:
-            self.__stdin_thread = threading.Thread(
-                name="stdin-thread",
-                target=self.__feeder, args=(self.__pending_input,
-                                            self.__process.stdin))
-            self.__stdin_thread.setDaemon(True)
-            self.__stdin_thread.start()
-
-        if self.__process.stdout:
-            self.__stdout_thread = threading.Thread(
-                name="stdout-thread",
-                target=self.__reader, args=(self.__collected_outdata,
-                                            self.__process.stdout))
-            self.__stdout_thread.setDaemon(True)
-            self.__stdout_thread.start()
-
-        if self.__process.stderr:
-            self.__stderr_thread = threading.Thread(
-                name="stderr-thread",
-                target=self.__reader, args=(self.__collected_errdata,
-                                            self.__process.stderr))
-            self.__stderr_thread.setDaemon(True)
-            self.__stderr_thread.start()
-
-    def pid(self):
-        return self.__process.pid
-
-    def kill(self, signal):
-        self.__process.send_signal(signal)
-
-    # check on subprocess (pass in 'nowait') to act like poll
-    def wait(self, flag):
-        if flag.lower() == 'nowait':
-            rc = self.__process.poll()
-        else:
-            rc = self.__process.wait()
-        if rc != None:
-            if self.__process.stdin:
-                self.closeinput()
-            if self.__process.stdout:
-                self.__stdout_thread.join()
-            if self.__process.stderr:
-                self.__stderr_thread.join()
-        return self.__process.returncode
-
-    def terminate(self):
-        if self.__process.stdin:
-            self.closeinput()
-        self.__process.terminate()
-
-    # thread gets data from subprocess stdout
-    def __reader(self, collector, source):
-        while True:
-            data = os.read(source.fileno(), 65536)
-            self.__lock.acquire()
-            collector.append(data)
-            self.__lock.release()
-            if data == "":
-                source.close()
-                break
-        return
-
-    # thread feeds data to subprocess stdin
-    def __feeder(self, pending, drain):
-        while True:
-            self.__inputsem.acquire()
-            self.__lock.acquire()
-            if not pending  and self.__quit:
-                drain.close()
-                self.__lock.release()
-                break
-            data = pending.pop(0)
-            self.__lock.release()
-            drain.write(data)
-
-    # non-blocking read of data from subprocess stdout
-    def read(self):
-        self.__lock.acquire()
-        outdata = "".join(self.__collected_outdata)
-        del self.__collected_outdata[:]
-        self.__lock.release()
-        return outdata
-
-    # non-blocking read of data from subprocess stderr
-    def readerr(self):
-        self.__lock.acquire()
-        errdata = "".join(self.__collected_errdata)
-        del self.__collected_errdata[:]
-        self.__lock.release()
-        return errdata
-
-    # non-blocking write to stdin of subprocess
-    def write(self, data):
-        if self.__process.stdin is None:
-            raise ValueError("Writing to process with stdin not a pipe")
-        self.__lock.acquire()
-        self.__pending_input.append(data)
-        self.__inputsem.release()
-        self.__lock.release()
-
-    # close stdinput of subprocess
-    def closeinput(self):
-        self.__lock.acquire()
-        self.__quit = True
-        self.__inputsem.release()
-        self.__lock.release()

DeDRM_plugin/topazextract.py

@@ -1,6 +1,8 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 
+from __future__ import print_function
+
 # topazextract.py
 # Mostly written by some_updates based on code from many others
 
@@ -23,69 +25,10 @@ from struct import pack
 from struct import unpack
 
 from alfcrypto import Topaz_Cipher
+from utilities import SafeUnbuffered
 
-# Wrap a stream so that output gets flushed immediately
-# and also make sure that any unicode strings get
-# encoded using "replace" before writing them.
-class SafeUnbuffered:
-    def __init__(self, stream):
-        self.stream = stream
-        self.encoding = stream.encoding
-        if self.encoding == None:
-            self.encoding = "utf-8"
-    def write(self, data):
-        if isinstance(data,str) or isinstance(data,unicode):
-            # str for Python3, unicode for Python2
-            data = data.encode(self.encoding,"replace")
-        try:
-            buffer = getattr(self.stream, 'buffer', self.stream)
-            # self.stream.buffer for Python3, self.stream for Python2
-            buffer.write(data)
-            buffer.flush()
-        except:
-            # We can do nothing if a write fails
-            raise
-    def __getattr__(self, attr):
-        return getattr(self.stream, attr)
+from argv_utils import unicode_argv
 
-iswindows = sys.platform.startswith('win')
-isosx = sys.platform.startswith('darwin')
-
-def unicode_argv():
-    if iswindows:
-        # Uses shell32.GetCommandLineArgvW to get sys.argv as a list of Unicode
-        # strings.
-
-        # Versions 2.x of Python don't support Unicode in sys.argv on
-        # Windows, with the underlying Windows API instead replacing multi-byte
-        # characters with '?'.
-
-
-        from ctypes import POINTER, byref, cdll, c_int, windll
-        from ctypes.wintypes import LPCWSTR, LPWSTR
-
-        GetCommandLineW = cdll.kernel32.GetCommandLineW
-        GetCommandLineW.argtypes = []
-        GetCommandLineW.restype = LPCWSTR
-
-        CommandLineToArgvW = windll.shell32.CommandLineToArgvW
-        CommandLineToArgvW.argtypes = [LPCWSTR, POINTER(c_int)]
-        CommandLineToArgvW.restype = POINTER(LPWSTR)
-
-        cmd = GetCommandLineW()
-        argc = c_int(0)
-        argv = CommandLineToArgvW(cmd, byref(argc))
-        if argc.value > 0:
-            # Remove Python executable and commands if present
-            start = argc.value - len(sys.argv)
-            return [argv[i] for i in
-                    range(start, argc.value)]
-        # if we don't have any arguments at all, just pass back script name
-        # this should never happen
-        return ["mobidedrm.py"]
-    else:
-        argvencoding = sys.stdin.encoding or "utf-8"
-        return [arg if (isinstance(arg, str) or isinstance(arg,unicode)) else str(arg, argvencoding) for arg in sys.argv]
 
 #global switch
 debug = False
@@ -458,7 +401,7 @@ def usage(progname):
 
 # Main
 def cli_main():
-    argv=unicode_argv()
+    argv=unicode_argv("topazextract.py")
     progname = os.path.basename(argv[0])
     print("TopazExtract v{0}.".format(__version__))
 

DeDRM_plugin/utilities.py

@@ -3,23 +3,10 @@
 
 #@@CALIBRE_COMPAT_CODE@@
 
-from ignoblekeyGenPassHash import generate_key
 import sys
 
 __license__ = 'GPL v3'
 
-DETAILED_MESSAGE = \
-'You have personal information stored in this plugin\'s customization '+ \
-'string from a previous version of this plugin.\n\n'+ \
-'This new version of the plugin can convert that info '+ \
-'into key data that the new plugin can then use (which doesn\'t '+ \
-'require personal information to be stored/displayed in an insecure '+ \
-'manner like the old plugin did).\n\nIf you choose NOT to migrate this data at this time '+ \
-'you will be prompted to save that personal data to a file elsewhere; and you\'ll have '+ \
-'to manually re-configure this plugin with your information.\n\nEither way... ' + \
-'this new version of the plugin will not be responsible for storing that personal '+ \
-'info in plain sight any longer.'
-
 def uStrCmp (s1, s2, caseless=False):
     import unicodedata as ud
     if sys.version_info[0] == 2:
@@ -34,14 +21,29 @@ def uStrCmp (s1, s2, caseless=False):
     else:
         return ud.normalize('NFC', str1) == ud.normalize('NFC', str2)
 
-def parseCustString(keystuff):
-    userkeys = []
-    ar = keystuff.split(':')
-    for i in ar:
+
+
+# Wrap a stream so that output gets flushed immediately
+# and also make sure that any unicode strings get safely
+# encoded using "replace" before writing them.
+class SafeUnbuffered:
+    def __init__(self, stream):
+        self.stream = stream
+        self.encoding = stream.encoding
+        if self.encoding == None:
+            self.encoding = "utf-8"
+    def write(self, data):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
+            data = data.encode(self.encoding,"replace")
         try:
-            name, ccn = i.split(',')
-            # Generate Barnes & Noble EPUB user key from name and credit card number.
-            userkeys.append(generate_key(name, ccn))
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
         except:
-            pass
-    return userkeys
+            # We can do nothing if a write fails
+            raise
+    def __getattr__(self, attr):
+        return getattr(self.stream, attr)
+        

DeDRM_plugin/zeroedzipinfo.py

@@ -0,0 +1,30 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+
+"""
+Python 3's "zipfile" has an annoying bug where the `external_attr` field 
+of a ZIP file cannot be set to 0. However, if the original DRMed ZIP has 
+that set to 0 then we want the DRM-free ZIP to have that as 0, too. 
+See https://github.com/python/cpython/issues/87713
+
+We cannot just set the "external_attr" to 0 as the code to save the ZIP
+resets that variable. 
+
+So, here's a class that inherits from ZipInfo and ensures that EVERY 
+read access to that variable will return a 0 ...
+
+"""
+
+import zipfile
+
+class ZeroedZipInfo(zipfile.ZipInfo):
+    def __init__(self, zinfo):
+        for k in self.__slots__:
+            if hasattr(zinfo, k):
+                setattr(self, k, getattr(zinfo, k))
+
+    def __getattribute__(self, name):
+        if name == "external_attr":
+            return 0
+        return object.__getattribute__(self, name)

DeDRM_plugin/zipfilerugged.py

@@ -394,6 +394,19 @@ class ZipInfo (object):
             extra = extra[ln+4:]
 
 
+class ZeroedZipInfo(ZipInfo):
+    def __init__(self, zinfo):
+        for k in self.__slots__:
+            if hasattr(zinfo, k):
+                setattr(self, k, getattr(zinfo, k))
+
+    def __getattribute__(self, name):
+        if name == "external_attr":
+            return 0
+        return object.__getattribute__(self, name)
+
+
+
 class _ZipDecrypter:
     """Class to handle decryption of files stored within a ZIP archive.
 

DeDRM_plugin/zipfix.py

@@ -26,6 +26,7 @@ import sys, os
 
 import zlib
 import zipfilerugged
+from zipfilerugged import ZipInfo, ZeroedZipInfo
 import getopt
 from struct import unpack
 
@@ -36,12 +37,6 @@ _FILENAME_OFFSET = 30
 _MAX_SIZE = 64 * 1024
 _MIMETYPE = 'application/epub+zip'
 
-class ZipInfo(zipfilerugged.ZipInfo):
-    def __init__(self, *args, **kwargs):
-        if 'compress_type' in kwargs:
-            compress_type = kwargs.pop('compress_type')
-        super(ZipInfo, self).__init__(*args, **kwargs)
-        self.compress_type = compress_type
 
 class fixZip:
     def __init__(self, zinput, zoutput):
@@ -117,7 +112,8 @@ class fixZip:
         # if epub write mimetype file first, with no compression
         if self.ztype == 'epub':
             # first get a ZipInfo with current time and no compression
-            mimeinfo = ZipInfo(b'mimetype',compress_type=zipfilerugged.ZIP_STORED)
+            mimeinfo = ZipInfo(b'mimetype')
+            mimeinfo.compress_type = zipfilerugged.ZIP_STORED
             mimeinfo.internal_attr = 1 # text file
             try:
                 # if the mimetype is present, get its info, including time-stamp
@@ -129,8 +125,16 @@ class fixZip:
                 mimeinfo.internal_attr = oldmimeinfo.internal_attr
                 mimeinfo.external_attr = oldmimeinfo.external_attr
                 mimeinfo.create_system = oldmimeinfo.create_system
+                mimeinfo.create_version = oldmimeinfo.create_version
+                mimeinfo.volume = oldmimeinfo.volume
             except:
                 pass
+
+            # Python 3 has a bug where the external_attr is reset to `0o600 << 16`
+            # if it's NULL, so we need a workaround:
+            if mimeinfo.external_attr == 0: 
+                mimeinfo = ZeroedZipInfo(mimeinfo)
+
             self.outzip.writestr(mimeinfo, _MIMETYPE.encode('ascii'))
 
         # write the rest of the files
@@ -145,13 +149,23 @@ class fixZip:
                     zinfo.filename = local_name
 
                 # create new ZipInfo with only the useful attributes from the old info
-                nzinfo = ZipInfo(zinfo.filename, zinfo.date_time, compress_type=zinfo.compress_type)
+                nzinfo = ZipInfo(zinfo.filename)
+                nzinfo.date_time = zinfo.date_time
+                nzinfo.compress_type = zinfo.compress_type
                 nzinfo.comment=zinfo.comment
                 nzinfo.extra=zinfo.extra
                 nzinfo.internal_attr=zinfo.internal_attr
                 nzinfo.external_attr=zinfo.external_attr
                 nzinfo.create_system=zinfo.create_system
+                nzinfo.create_version = zinfo.create_version
+                nzinfo.volume = zinfo.volume
                 nzinfo.flag_bits = zinfo.flag_bits & 0x800  # preserve UTF-8 flag
+
+                # Python 3 has a bug where the external_attr is reset to `0o600 << 16`
+                # if it's NULL, so we need a workaround:
+                if nzinfo.external_attr == 0: 
+                    nzinfo = ZeroedZipInfo(nzinfo)
+
                 self.outzip.writestr(nzinfo,data)
 
         self.bzf.close()

FAQs.md

@@ -31,7 +31,7 @@ Verify the one of the following cryptographic hash values, using software of you
 * SHA-1: 7AB9A86B954CB23D622BD79E3257F8E2182D791C
 * SHA-256: 28DC21246A9C7CDEDD2D6F0F4082E6BF7EF9DB9CE9D485548E8A9E1D19EAE2AC
 
-You will need to go to the preferences and uncheck the auto update checkbox. Then download and install 1.17 over the top of the newer installation. You'll also need to delete the KFX folders from your My Kindle Content folder. You may also need to take further action to prevent an auto update. The simplest wayis to find the 'updates' folder and replace it with a file. See [this thread] (http://www.mobileread.com/forums/showthread.php?t=283371) at MobileRead for a Script to do this on a PC. On a Mac you can find the folder at ~/Library/Application Support/Kindle/. Make the 'updates' folder read-only, or delete it and save a blank text file called 'updates' in its place.
+You will need to go to the preferences and uncheck the auto update checkbox. Then download and install 1.17 over the top of the newer installation. You'll also need to delete the KFX folders from your My Kindle Content folder. You may also need to take further action to prevent an auto update. The simplest way is to find the 'updates' folder and replace it with a file. See [this thread] (http://www.mobileread.com/forums/showthread.php?t=283371) at MobileRead for a Script to do this on a PC. On a Mac you can find the folder at ~/Library/Application Support/Kindle/. Make the 'updates' folder read-only, or delete it and save a blank text file called 'updates' in its place.
 
 Another possible solution is to use 1.19 or later, but disable KFX by renaming or disabling a necessary component of the application. This may or may not work on versions after 1.25. In a command window, enter the following commands when Kindle for PC/Mac is not running:
 
@@ -183,7 +183,7 @@ Amazon turned off backup for Kindle for Android, so the tools can no longer find
 Apple regularly change the details of their DRM and so the tools in the main tools archive will not work with these ebooks. Apple’s Fairplay DRM scheme can be removed using Requiem if the appropriate version of iTunes can still be installed and used. See the post Apple and ebooks: iBookstore DRM and how to remove it at Apprentice Alf's blog for more details.
 
 ## Why don't the tools work with LCP-encrypted ebooks? / Error message about a "DMCA takedown"
-Support for LCP DRM removal was included in the past, but Readium (the company who developed that particular DRM) has decided to [open a DMCA takedown request](https://github.com/github/dmca/blob/master/2022/01/2022-01-04-readium.md) in January 2022. This means that for legal reasons, this GitHun repository no longer contains the code needed to remove DRM from LCP-encrypted books. For more information please read [this bug report](https://github.com/noDRM/DeDRM_tools/issues/18). 
+Support for LCP DRM removal was included in the past, but Readium (the company who developed that particular DRM) has decided to [open a DMCA takedown request](https://github.com/github/dmca/blob/master/2022/01/2022-01-04-readium.md) in January 2022. This means that for legal reasons, this GitHub repository no longer contains the code needed to remove DRM from LCP-encrypted books. For more information please read [this bug report](https://github.com/noDRM/DeDRM_tools/issues/18). 
 
 ## I’ve got the tools archive and I’ve read all the FAQs but I still can’t install the tools and/or the DRM removal doesn’t work
 * Read the `ReadMe_Overview.txt` file in the top level of the tools archive

Obok_plugin/action.py

@@ -237,7 +237,10 @@ class InterfacePluginAction(InterfaceAction):
 
         :param books_to_add: List of calibre bookmaps (created in get_decrypted_kobo_books)
         '''
-        added = self.db.add_books(books_to_add, add_duplicates=False, run_hooks=False)
+
+        cfg_add_duplicates = (cfg['finding_homes_for_formats'] == 'Add new entry')
+
+        added = self.db.add_books(books_to_add, add_duplicates=cfg_add_duplicates, run_hooks=False)
         if len(added[0]):
             # Record the id(s) that got added
             for id in added[0]:

Obok_plugin/common_utils.py

@@ -265,13 +265,13 @@ class ReadOnlyTableWidgetItem(QTableWidgetItem):
     def __init__(self, text):
         if text is None:
             text = ''
-        QTableWidgetItem.__init__(self, text, QTableWidgetItem.UserType)
+        QTableWidgetItem.__init__(self, text, QTableWidgetItem.ItemType.UserType)
         self.setFlags(Qt.ItemIsSelectable|Qt.ItemIsEnabled)
 
 class RatingTableWidgetItem(QTableWidgetItem):
 
     def __init__(self, rating, is_read_only=False):
-        QTableWidgetItem.__init__(self, '', QTableWidgetItem.UserType)
+        QTableWidgetItem.__init__(self, '', QTableWidgetItem.ItemType.UserType)
         self.setData(Qt.DisplayRole, rating)
         if is_read_only:
             self.setFlags(Qt.ItemIsSelectable|Qt.ItemIsEnabled)
@@ -284,11 +284,11 @@ class DateTableWidgetItem(QTableWidgetItem):
         if date_read is None or date_read == UNDEFINED_DATE and default_to_today:
             date_read = now()
         if is_read_only:
-            QTableWidgetItem.__init__(self, format_date(date_read, fmt), QTableWidgetItem.UserType)
+            QTableWidgetItem.__init__(self, format_date(date_read, fmt), QTableWidgetItem.ItemType.UserType)
             self.setFlags(Qt.ItemIsSelectable|Qt.ItemIsEnabled)
             self.setData(Qt.DisplayRole, QDateTime(date_read))
         else:
-            QTableWidgetItem.__init__(self, '', QTableWidgetItem.UserType)
+            QTableWidgetItem.__init__(self, '', QTableWidgetItem.ItemType.UserType)
             self.setData(Qt.DisplayRole, QDateTime(date_read))
 
 from calibre.gui2.library.delegates import DateDelegate as _DateDelegate

Obok_plugin/config.py

@@ -39,8 +39,13 @@ class ConfigWidget(QWidget):
         self.find_homes = QComboBox()
         self.find_homes.setToolTip(_('<p>Default behavior when duplicates are detected. None of the choices will cause calibre ebooks to be overwritten'))
         layout.addWidget(self.find_homes)
-        self.find_homes.addItems([_('Ask'), _('Always'), _('Never')])
+
+        self.find_homes.addItems([_('Ask'), _('Always'), _('Never'), _('Add new entry')])
+
         index = self.find_homes.findText(plugin_prefs['finding_homes_for_formats'])
+        if index == -1:
+            index = self.find_homes.findText(_(plugin_prefs['finding_homes_for_formats']))
+
         self.find_homes.setCurrentIndex(index)
 
         self.serials_button = QtGui.QPushButton(self)
@@ -69,7 +74,24 @@ class ConfigWidget(QWidget):
 
 
     def save_settings(self):
-        plugin_prefs['finding_homes_for_formats'] = self.find_homes.currentText()
+
+
+        # Make sure the config file string is *always* english.
+        find_homes = None
+        if self.find_homes.currentText() == _('Ask'):
+            find_homes = 'Ask'
+        elif self.find_homes.currentText() == _('Always'):
+            find_homes = 'Always'
+        elif self.find_homes.currentText() == _('Never'):
+            find_homes = 'Never'
+        elif self.find_homes.currentText() == _('Add new entry'):
+            find_homes = 'Add new entry'
+        
+        if find_homes is None:
+            # Fallback
+            find_homes = self.find_homes.currentText()
+
+        plugin_prefs['finding_homes_for_formats'] = find_homes
         plugin_prefs['kobo_serials'] = self.tmpserials
         plugin_prefs['kobo_directory'] = self.kobodirectory
 

Obok_plugin/obok/obok.py

@@ -224,10 +224,17 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,str):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.buffer.write(data)
-        self.stream.buffer.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
 
@@ -305,45 +312,65 @@ class KoboLibrary(object):
                     if sys.getwindowsversion().major > 5:
                         if 'LOCALAPPDATA' in os.environ.keys():
                             # Python 2.x does not return unicode env. Use Python 3.x
-                            self.kobodir = winreg.ExpandEnvironmentStrings("%LOCALAPPDATA%")
+                            if sys.version_info[0] == 2:
+                                self.kobodir = winreg.ExpandEnvironmentStrings(u"%LOCALAPPDATA%")
+                            else: 
+                                self.kobodir = winreg.ExpandEnvironmentStrings("%LOCALAPPDATA%")
                     if (self.kobodir == u""):
                         if 'USERPROFILE' in os.environ.keys():
                             # Python 2.x does not return unicode env. Use Python 3.x
-                            self.kobodir = os.path.join(winreg.ExpandEnvironmentStrings("%USERPROFILE%"), "Local Settings", "Application Data")
+                            if sys.version_info[0] == 2:
+                                self.kobodir = os.path.join(winreg.ExpandEnvironmentStrings(u"%USERPROFILE%"), "Local Settings", "Application Data")
+                            else: 
+                                self.kobodir = os.path.join(winreg.ExpandEnvironmentStrings("%USERPROFILE%"), "Local Settings", "Application Data")
                     self.kobodir = os.path.join(self.kobodir, "Kobo", "Kobo Desktop Edition")
                 elif sys.platform.startswith('darwin'):
                     self.kobodir = os.path.join(os.environ['HOME'], "Library", "Application Support", "Kobo", "Kobo Desktop Edition")
                 elif sys.platform.startswith('linux'):
+                    # Since on Linux, you have to run Kobo Desktop within Wine,
+                    # there is no guarantee where Kobo.sqlite (and the rest of
+                    # the kobo directory) might be.
+                    #
+                    # It should also be noted that Kobo Desktop will store all
+                    # of it files in the current directory where you run it,
+                    # meaning that a user might accidentally create several
+                    # Kobo.sqlite files which are all separate and then be
+                    # confused why Obok can't find any of the new books. Sadly
+                    # there isn't a trivial way to deal with this.
 
-                    #sets ~/.config/calibre as the location to store the kobodir location info file and creates this directory if necessary
-                    kobodir_cache_dir = os.path.join(os.environ['HOME'], ".config", "calibre")
+                    # We cache the kobodir we find in ~/.config/calibre.
+                    kobodir_cache_dir = os.path.join(os.environ['HOME'], ".config", "calibre", "plugins", "obok")
                     if not os.path.isdir(kobodir_cache_dir):
                         os.mkdir(kobodir_cache_dir)
-                    
-                    #appends the name of the file we're storing the kobodir location info to the above path
-                    kobodir_cache_file = str(kobodir_cache_dir) + "/" + "kobo location"
-                    
-                    """if the above file does not exist, recursively searches from the root
-                    of the filesystem until kobodir is found and stores the location of kobodir
-                    in that file so this loop can be skipped in the future"""
-                    original_stdout = sys.stdout
-                    if not os.path.isfile(kobodir_cache_file):
-                        for root, dirs, files in os.walk('/'):
-                            for file in files:
-                                if file == 'Kobo.sqlite':
-                                    kobo_linux_path = str(root)
-                                    with open(kobodir_cache_file, 'w') as f:
-                                        sys.stdout = f
-                                        print(kobo_linux_path, end='')
-                                        sys.stdout = original_stdout
+                    kobodir_cache_file = os.path.join(kobodir_cache_dir, "kobo-location")
 
-                    f = open(kobodir_cache_file, 'r' )
-                    self.kobodir = f.read()
+                    try:
+                        # If the cached version exists and the path does really
+                        # contain Kobo.sqlite, use that.
+                        with open(kobodir_cache_file, "r") as f:
+                            cached_kobodir = f.read().strip()
+                            assert os.path.isfile(os.path.join(cached_kobodir, "Kobo.sqlite"))
+                            self.kobodir = cached_kobodir
+                    except (AssertionError, FileNotFoundError):
+                        # If there was no cached version, search the entire
+                        # filesystem tree for a directory containing
+                        # "Kobo.sqlite".
+                        #
+                        # We first search $HOME to avoid picking another user's
+                        # Kobo.sqlite file, but then fallback to / if there was
+                        # nothing in $HOME.
+                        for candidate_root in (os.environ["HOME"], "/"):
+                            for root, _, files in os.walk(candidate_root):
+                                if "Kobo.sqlite" in files:
+                                    with open(kobodir_cache_file, "w") as f:
+                                        f.write("%s/\n" % (root,))
+                                    self.kobodir = root
+                                    break
 
-            # desktop versions use Kobo.sqlite
+            # Desktop versions use Kobo.sqlite.
             kobodb = os.path.join(self.kobodir, "Kobo.sqlite")
             # check for existence of file
-            if (not(os.path.isfile(kobodb))):
+            if not os.path.isfile(kobodb):
                 # give up here, we haven't found anything useful
                 self.kobodir = u""
                 kobodb  = u""
@@ -417,7 +444,7 @@ class KoboLibrary(object):
         macaddrs = []
         if sys.platform.startswith('win'):
             c = re.compile('\s?(' + '[0-9a-f]{2}[:\-]' * 5 + '[0-9a-f]{2})(\s|$)', re.IGNORECASE)
-            try: 
+            try:
                 output = subprocess.Popen('ipconfig /all', shell=True, stdout=subprocess.PIPE, text=True).stdout
                 for line in output:
                     m = c.search(line)

Obok_plugin/utilities.py

@@ -224,5 +224,5 @@ class ReadOnlyTableWidgetItem(QTableWidgetItem):
     def __init__(self, text):
         if text is None:
             text = ''
-        QTableWidgetItem.__init__(self, text, QTableWidgetItem.UserType)
+        QTableWidgetItem.__init__(self, text, QTableWidgetItem.ItemType.UserType)
         self.setFlags(Qt.ItemIsSelectable|Qt.ItemIsEnabled)

Other_Tools/DRM_Key_Scripts/Adobe_Digital_Editions/adobekey.pyw

@@ -68,12 +68,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 try:
     from calibre.constants import iswindows, isosx

Other_Tools/DRM_Key_Scripts/Barnes_and_Noble_ePubs/ignoblekey.pyw

@@ -39,12 +39,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 try:
     from calibre.constants import iswindows, isosx

Other_Tools/DRM_Key_Scripts/Barnes_and_Noble_ePubs/ignoblekeyfetch.pyw

@@ -45,12 +45,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 try:
     from calibre.constants import iswindows, isosx

Other_Tools/DRM_Key_Scripts/Barnes_and_Noble_ePubs/ignoblekeygen.pyw

@@ -56,12 +56,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 try:
     from calibre.constants import iswindows, isosx

Other_Tools/DRM_Key_Scripts/Kindle_for_Android/androidkindlekey.pyw

@@ -48,12 +48,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 try:
     from calibre.constants import iswindows, isosx

Other_Tools/DRM_Key_Scripts/Kindle_for_Mac_and_PC/kindlekey.pyw

@@ -51,12 +51,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 try:
     from calibre.constants import iswindows, isosx
@@ -292,7 +300,7 @@ if iswindows:
                 numBlocks, numExtraBytes = divmod(len(self.bytesToDecrypt), self.blockSize)
                 if more == None:  # no more calls to decrypt, should have all the data
                     if numExtraBytes  != 0:
-                        raise DecryptNotBlockAlignedError, 'Data not block aligned on decrypt'
+                        raise DecryptNotBlockAlignedError('Data not block aligned on decrypt')
 
                 # hold back some bytes in case last decrypt has zero len
                 if (more != None) and (numExtraBytes == 0) and (numBlocks >0) :
@@ -334,7 +342,7 @@ if iswindows:
             def removePad(self, paddedBinaryString, blockSize):
                 """ Remove padding from a binary string """
                 if not(0<len(paddedBinaryString)):
-                    raise DecryptNotBlockAlignedError, 'Expected More Data'
+                    raise DecryptNotBlockAlignedError('Expected More Data')
                 return paddedBinaryString[:-ord(paddedBinaryString[-1])]
 
         class noPadding(Pad):
@@ -364,8 +372,8 @@ if iswindows:
                 self.blockSize  = blockSize  # blockSize is in bytes
                 self.padding    = padding    # change default to noPadding() to get normal ECB behavior
 
-                assert( keySize%4==0 and NrTable[4].has_key(keySize/4)),'key size must be 16,20,24,29 or 32 bytes'
-                assert( blockSize%4==0 and NrTable.has_key(blockSize/4)), 'block size must be 16,20,24,29 or 32 bytes'
+                assert( keySize%4==0 and keySize/4 in NrTable[4]),'key size must be 16,20,24,29 or 32 bytes'
+                assert( blockSize%4==0 and blockSize/4 in NrTable), 'block size must be 16,20,24,29 or 32 bytes'
 
                 self.Nb = self.blockSize/4          # Nb is number of columns of 32 bit words
                 self.Nk = keySize/4                 # Nk is the key length in 32-bit words
@@ -642,7 +650,7 @@ if iswindows:
             def __init__(self, key = None, padding = padWithPadLen(), keySize=16):
                 """ Initialize AES, keySize is in bytes """
                 if  not (keySize == 16 or keySize == 24 or keySize == 32) :
-                    raise BadKeySizeError, 'Illegal AES key size, must be 16, 24, or 32 bytes'
+                    raise BadKeySizeError('Illegal AES key size, must be 16, 24, or 32 bytes')
 
                 Rijndael.__init__( self, key, padding=padding, keySize=keySize, blockSize=16 )
 
@@ -782,10 +790,11 @@ if iswindows:
             #                             [c_char_p, c_ulong, c_char_p, c_ulong, c_ulong, c_ulong, c_char_p])
             def pbkdf2(self, passwd, salt, iter, keylen):
 
-                def xorstr( a, b ):
+                def xorbytes( a, b ):
                     if len(a) != len(b):
-                        raise Exception("xorstr(): lengths differ")
-                    return ''.join((chr(ord(x)^ord(y)) for x, y in zip(a, b)))
+                        raise Exception("xorbytes(): lengths differ")
+                    return bytes([x ^ y for x, y in zip(a, b)])
+
 
                 def prf( h, data ):
                     hm = h.copy()
@@ -797,13 +806,13 @@ if iswindows:
                     T = U
                     for i in range(2, itercount+1):
                         U = prf( h, U )
-                        T = xorstr( T, U )
+                        T = xorbytes( T, U )
                     return T
 
                 sha = hashlib.sha1
                 digest_size = sha().digest_size
                 # l - number of output blocks to produce
-                l = keylen / digest_size
+                l = keylen // digest_size
                 if keylen % digest_size != 0:
                     l += 1
                 h = hmac.new( passwd, None, sha )

Other_Tools/DRM_Key_Scripts/Kindle_for_iOS/kindleiospidgen.pyw

@@ -32,12 +32,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 iswindows = sys.platform.startswith('win')
 isosx = sys.platform.startswith('darwin')

Other_Tools/Kobo/obok.py

@@ -276,12 +276,20 @@ class SafeUnbuffered:
         if self.encoding == None:
             self.encoding = "utf-8"
     def write(self, data):
-        if isinstance(data,unicode):
+        if isinstance(data,str) or isinstance(data,unicode):
+            # str for Python3, unicode for Python2
             data = data.encode(self.encoding,"replace")
-        self.stream.write(data)
-        self.stream.flush()
+        try:
+            buffer = getattr(self.stream, 'buffer', self.stream)
+            # self.stream.buffer for Python3, self.stream for Python2
+            buffer.write(data)
+            buffer.flush()
+        except:
+            # We can do nothing if a write fails
+            raise
     def __getattr__(self, attr):
         return getattr(self.stream, attr)
+        
 
 
 class KoboLibrary(object):

README.md

@@ -1,9 +1,11 @@
 # DeDRM_tools
 DeDRM tools for ebooks
 
-This is a fork of Apprentice Harper's version of the DeDRM tools. I've added some of the PRs that still haven't been merged, as well as added some more features / bugfixes myself.
+This is a fork of Apprentice Harper's version of the DeDRM tools. Apprentice Harper said that the original version of the plugin [is no longer maintained](https://github.com/apprenticeharper/DeDRM_tools#no-longer-maintained), so I've taken over, merged a bunch of open PRs, and added a ton more features and bugfixes. 
 
-Take a look at [the CHANGELOG](https://github.com/noDRM/DeDRM_tools/blob/master/CHANGELOG.md) to see a list of changes since the last version by Apprentice Harper (v7.2.1). This plugin will start with version v10.0.0 so there won't be conflicting / duplicate version numbers when Apprentice Harper's version is updated again.
+The latest stable (released) version is v10.0.3 which [can be downloaded here](https://github.com/noDRM/DeDRM_tools/releases/tag/v10.0.3).
+
+Take a look at [the CHANGELOG](https://github.com/noDRM/DeDRM_tools/blob/master/CHANGELOG.md) to see a list of changes since the last version by Apprentice Harper (v7.2.1). This plugin will start with version v10.0.0.
 
 The v10.0.0 versions of this plugin should both work with Calibre 5.x (Python 3) as well as Calibre 4.x and lower (Python 2). If you encounter issues with this plugin in Calibre 4.x or lower, please open a bug report.