Release Notes for TrueZIP 6.4

(December 21st, 2006)

Contents

  1. Introduction
  2. List of Updates (Change Log)

Introduction

Though this is a minor version number release, it's a major improvement. The primary focus has been to fortify the code and optimize it: Many bugs were fixed, accompanied by new or enhanced JUnit tests in order to prevent future regressions. Furthermore, performance has been significantly improved in some sectors, especially if a client application happens to deal a lot with false positive archive files, TAR files or RAES encrypted ZIP files. In addition, the Swing based key prompting GUI has now been extensively tested on Linux in order to fix some bugs which only showed on this platform.

Upgrading is highly recommended for everybody!

Legend
New Introduces a new feature.
Fixed Introduces a bug fix of an existing feature.
Enhanced Introduces the enhancement of an existing feature. This update is fully backwards compatible.
Changed Introduces the change of an existing feature. This update may cause backwards incompatibilities .
Deprecated Introduces the deprecation of an existing feature.

List of Updates (Change Log)

Following is an overview of all updates in this release which affect the public API. Please note that internal refactorings are not listed. For a full list of updates, please refer to the CVS repository and diff to the tag TrueZIP-6_3_2.

Cross Package Updates

  1. Enhanced: In addition to the default Swing based key manager class de.schlichtherle.key.passwd.swing.PromptingKeyManager, the classes de.schlichtherle.crypto.io.raes.Type0RaesReadOnlyFile and de.schlichtherle.key.SharedKeyProvider now both enforce a three second suspension penalty if a wrong key was provided. This has been added for protection against exhaustive key searches by malicious key provider implementations or abuses of even friendly implementations. The delay has no recognizable effect on the Swing based key manager because the key provider UI implementations used by this key manager already enforce the same suspension penalty by disabling the OK button for three seconds.
  2. Enhanced: InputArchive and OutputArchive in the package de.schlichtherle.io.archive.spi implementations do not need to be thread safe at all anymore. The synchronization is now completely handled by the archive controller and its companion meta data classes. This fixes some synchronization issues with the TAR driver and potentially the ZIP driver.
  3. Enhanced: Javadoc here and there.

Updates in the Base Package

  1. New: The nzip utility main class has two new commands: "isArchive" prints whether a file system node is a prospective archive file and "length" prints the length of a file system node. These commands may be useful in scripts.
  2. Enhanced: The utility main classes now include version information when printing their usage (finally).

Updates in the Package de.schlichtherle.awt

  1. Enhanced: Added the method invokeAndWaitInterruptibly to the class EventQueue in order to interrupt AWT's Event Dispatch Thread (EDT) too, if the waiting thread was interrupted. The method invokeAndWaitUninterruptibly now logs with the FINE level in case an interrupt was attempted.

Updates in the Package de.schlichtherle.crypto.generators

  1. Enhanced: Seeding algorithm in constructor of class DigestRandom to make creation of instances an order of a magnitude faster - at least on Linux, where the SUN provider's SHA1PRNG implementation uses the slow, but secure /dev/random device. This significantly improves performance when working with lots of RAES encrypted ZIP files on Linux, as the salt for each file is created from a new, self-seeding DigestRandom object.

Updates in the Package de.schlichtherle.io

  1. Fixed: The JVM shutdown hook in the class ArchiveController did not delete the temp files unless the client application called File.umount() after the last modification. This was a regression and used to work in earlier versions.
  2. Fixed: Wrong initialization if "dot-only" paths ("", ".", "./", "./.", ...) were passed as the child parameter to the "parent constructor" File(java.io.File parent, String child, ArchiveDetector detector).
  3. Fixed: The synchronization strategy in the package private classes (In|Out)putArchiveMetaData has been revised in order to to completely exempt archive drivers from the need to be thread safe. This fixes some racing conditions and dead locks which were known to be present with the TAR driver family and potentially the ZIP driver family, too.
  4. Fixed: The method File.delete() sometimes returned true where it should have returned false, in particular when called on archive files which still have open input streams.
  5. Fixed: The method File.cp(File, File) accepted source and destination files which effectively referred to the same file or where the source contained the destination. This resulted in undefined behaviour. Now a ContainsFileException is thrown instead.
  6. Fixed: The method File.cp(File, File) sometimes failed to copy false positives, i.e. a node in the file system which appears to be an archive file, but actually is something else (a regular directory or plain file). In particular, this was true when copying entries within the same false positive directory, i.e. entries in the same regular directory. For the same reason, as opposed to its contract, File.cp() did throw an InputIOException instead of a FileNotFoundException if the input file or any of its parent directories was a false positive.
  7. Fixed/Enhanced: The class DefaultArchiveDetector had some bugs when instantiated with obscure constructor parameters like uppercase file suffixes or null arguments. It has been reimplemented to work with collections and thread locals for enhanced performance and maintainability.
  8. Fixed: When an input stream for an entry in an otherwise unmodified archive file was used which's File instance hasn't been kept, the stream was sometimes arbitrarily closed. This happened because the archive controller was mistakenly garbage collected. The package private classes InputArchiveMetaData and OutputArchiveMetaData have been fixed to hold a strong reference to the archive controller now, so that the archive controller is strongly reachable from any entry stream.
  9. Enhanced: The methods File.listFiles(*) now always return an array of File instances instead of just an array of java.io.File instances entirely populated with File instances. This is in preparation for the migration to Java 5 where these methods will also explicitly declare this return value in order to get rid of those ugly type casts.
  10. New: The classes File(Reader|Writer) have been added as drop-in replacements for the equal named classes in the java.io package.
  11. Enhanced: The ArchiveController class now protects the access to its file system with a state pattern in order to remember false positive archive files. In case of the unit tests for the File class, this results in three times faster performance, primarily because the unit tests are testing false positive archives a lot! The state is reset when File.delete() is called on the false positive archive file or File.update() or File.umount().
  12. Enhanced: If running on JSE 5 or later, TrueZIP automatically takes advantage of the cached thread pools available in the java.util.concurrent package when copying data asynchronously. This provides some performance benefits. Full backwards compatibility to J2SE 1.4.2 is retained.
  13. Enhanced: File.setLenient() now also controls how TrueZIP shall behave on unclosed archive entry streams: If set to false, a hard reference to all archive entry streams is maintained by TrueZIP, causing an ArchiveBusy(Warning)?Exception to be thrown on the next call to File.update() or File.umount() in case of an unclosed archive entry stream. If set to true (the default), then TrueZIP behaves as before and will close archive entry streams when they are garbage collected, reducing the likeliness of an ArchiveBusy(Warning)?Exception to be thrown on the next call to File.update() or File.umount(). This is to support testing client applications for the presence of the "unclosed streams issue".
  14. Enhanced: ArchiveController.mkdir(*) doesn't try to mount the file system anymore if the target file exists already. This avoids to prompt for a key in case the target file is an RAES encrypted ZIP file although the key is not really required at this time and slightly enhances the performance.
  15. Deprecated: File.getDelegate(). This method exists for technical reasons only! In case you want to convert an instance of this class which recognized the leaf of its path as an archive file to a file instance which doesn't recognize this archive file, use the following code instead: ArchiveDetector.NULL.createFile((File) file.getParentFile(), file.getName()).
  16. Fixed: File.createNewFile() failed for entries in a false positive archive file which is actually a native directory.

Updates in the Package de.schlichtherle.io.archive.spi

  1. New: The class TransientIOException may now be used by archive drivers to signal that an IOException occured as a transient event when accessing an archive file and another try to access the same archive file may finally succeed. On the other hand, if the archive controller catches an IOException from an an archive driver when trying to access an archive file which is not a TransientIOException, then the archive controller may consider the archive file to be a false positive and cache the exception until File.umount() or File.update() is called. This is used by the RAES encrypted ZIP file driver family when prompting for passwords has been cancelled by the user and another attempt to prompt the user should be allowed.
  2. Enhanced: The archive driver specification has been relaxed so that InputArchive and OutputArchive implementations do not need to be thread safe anymore. The synchronization is now completely handled by the archive controller and its companion meta data classes. This fixes some synchronization issues with the TAR driver and potentially the ZIP driver, too. Old drivers will work to the specification, obviously.
  3. Enhanced: TransientIOException has been added to the Javadoc of the methods ArchiveDriver.createInputArchive and ArchiveDriver.createOutputArchive. Since this is an IOException, existing archive drivers are not affected.
  4. Changed: The classes InputArchiveBusyException and OutputArchiveBusyException are now subclasses of FileBusyException in order to simplify the cause chain: Previously, a FileBusyException may have had an (In|Out)putArchiveBusyException as its cause.

Updates in the Package de.schlichtherle.io.archive.tar

  1. Fixed: The storeDirectory method in the TarOutputArchive class did not set the entry size to zero before writing the directory entry although it has never written any data to it. This caused interoperability issues with some tools when TAR files have been created on Unix: Most notably, Ant's TarInputStream class failed on the resulting archive files (GNU TAR accepted it).
  2. Enhanced: TarInputArchive now uses a smarter read ahead algorithm to validate TAR files. However, only raw TAR archives benefit from it due to a bug in Sun's J2SE 1.4.2_12 (has been fixed in JSE 1.5.0-b64) which requires a workaround to render it useless. Check the source code for details.
  3. Enhanced: Due to the relaxed archive driver specification, TarOutputArchive doesn't need to be thread safe anymore and so the redundant synchronization has been removed for better performance.
  4. Enhanced: The TarBZip2Driver now uses buffered I/O for the underlying stream for much better performance. It also provides a new constructor argument to set the block size for the BZip2 algorithm - the default is the maximum block size. The new test case for this driver uses this to set the block size to the minimum in order to reduce the exhaustive memory occupation of the BZip2 implementation in the Ant code and make the test work. Warning: This driver is still experimental: Ant's underlying BZip2 implementation is very exhaustive on memory and slow (I'm not sure whether this is an implementation issue or a general issue with the BZip2 specification). Do not use this driver in productive environments for other than small archives! If you need to use it nevertheless, use Ant 1.7.0 RC1 or later. The BZip2 implementation in this version has been much improved and is required to pass TrueZIP's comprehensive unit tests for the File class with this driver.
  5. Enhanced: The TarDriver class now offers a new method for overriding: createInputStream(ReadOnlyFile). When writing a custom driver, you should override this method instead of createInputArchive(Archive, InputStream) in order to support proper closing of all streams in the chain. TarGZipDriver and TarBZip2Driver have been revised appropriately.
  6. Enhanced: Javadoc in almost all classes.

Updates in the Package de.schlichtherle.io.archive.zip

  1. Fixed: The class Zip32OutputArchive produced corrupted file entries if and only if any of the methods copy(All)?(To|From) or cp in the class File in the package de.schlichtherle.io were used and a ZIP file entry in the source used the STORE rather than the DEFLATE method. Note that the methods archiveCopy(All)?(To|From) or cp_p were not affected. Likewise, no data was corrupted if the source didn't contain ZIP file entries or none of these entries used the STORE method.
  2. Enhanced/Fixed: The Checked* driver classes have been changed to use the new feature of the ZipFile class in the package de.schlichtherle.util.zip to read deflated data while concurrently checking the CRC-32 value of the inflated data. The primary effect is to fix a bug which could have potentially corrupted data when copying entry data between archives operated by two ZIP drivers which handle checking differently. Another nice side effect is that these classes now support Direct Data Copying (DDC), which obsoletes the need to recompress data when copying an entry between two archives. This further implies that updating RAES encrypted ZIP files is now faster.
  3. Enhanced: Due to the relaxed archive driver specification, Zip32(In|Out)putArchive don't need to be thread safe anymore and so the redundant synchronization has been removed for better performance. The superclass of these classes are now the newly introduced BasicZip(File|OutputStream) classes in the package de.schlichtherle.util.zip.

Updates in the Package de.schlichtherle.io.archive.zip.raes

  1. Enhanced: The class AbstractZip32RaesDriver now provides an optional constructor parameter to adjust the authentication trigger, which is the largest size of an archive file which's RAES provided Message Authentication Code (MAC) will be checked. This defaults to Long.MAX_VALUE and may be overridden by subclasses (such as SafeZip32RaesDriver) in order to provide a different means to verify the integrity of an encrypted archive file.
  2. New: ParanoidZip32RaesDriver has been introduced: This archive driver always authenticates input archive files using the RAES provided Message Authentication Code (MAC) before the archive can be accessed by client applications. This can take quite a while for large archive files. This driver is now advertised accordingly in the configuration file.
  3. Enhanced: The authentication strategy in SafeZip32RaesDriver has been refined: For input archive files up to 512 KB, the cipher text gets authenticated using the RAES provided Message Authentication Code (MAC) before the archive can be accessed by a client application. For larger input archive files, the MAC is not used, but instead the CRC-32 value of the decrypted and deflated archive entries is checked when the archive entry stream is closed by the client application, resulting in some IOException. For more information why this operation mode is considered safe, please refer to the Javadoc.
  4. Changed: UnsafeZip32RaesDriver has been deprecated and is no longer advertised in the configuration file.

Updates in the Package de.schlichtherle.io.rof

  1. Enhanced: The class ReadOnlyFileInputStream now supports marking if the underlying ReadOnlyFile does.

Updates in the Package de.schlichtherle.io.swing

  1. Fixed/Enhanced: FileComboBoxBrowser now sorts nicely using a Collator for the default locale; case is now always ignored when auto completing; plain java.io.File instances can now be used for the directory, too; a few initialization bugs have been fixed.
  2. Fixed: JFileChooser may have dead locked when used to browse RAES encrypted ZIP files. This has been fixed at the cost of proper repainting when prompting for keys.
  3. Enhanced: JFileChooser now shows correct icons and type descriptions for false positive archive files.

Updates in the Package de.schlichtherle.io.util

  1. Fixed: The utility methods in the class Paths did not always deliver correct results. This negatively affected the way some other classes, most notably the class de.schlichtherle.io.File interpreted paths. In addition, the methods have been moved from Path to Paths again and Path has been deprecated. Backwards compatibility on source code level has been retained, however.
  2. New: Synchronized(In|Out)putStream as synchronized stream decorators. Used as utilities in some other classes.

Updates in the Package de.schlichtherle.key

  1. Enhanced: The contract in PromptingKeyProviderUI has been extended so that an implementation may optionally throw a RuntimeException with an UnknownKeyException as its cause in order to abort the prompting without the PromptingKeyProvider instance to change its state. This is useful when the prompting thread has been interrupted, in which case a subsequent attempt should still proceed instead of being automatically cancelled without actually prompting.
  2. Fixed: Potential synchronization / dead lock problems in the key manager and key provider classes.
  3. Enhanced: The abstract class KeyManager now loads its default implementation by class name in order to remove the compile time dependency on the sub package. In case you are using a class optimizer, don't forget to add all subclasses of KeyManager to the list of classes which are never renamed or discarded.
  4. New: If the JVM is running in headless mode and the API conforms to JSE 6 (the class java.io.Console is available), then the console I/O based key manager implementation in the class de.schlichtherle.key.passwd.console.PromptingKeyManager is used instead of the Swing based key manager implementation in the class de.schlichtherle.key.passwd.swing.PromptingKeyManager.
  5. Changed: The classes Prompting(Aes)?KeyProvider now simply return their base class names as the UI class identifiers instead of their full class name. Client applications should not be affected, however.
  6. Changed: The class SharedKeyProvider has been renamed to AbstractKeyProvider because of its misleading name. A deprecated stub class with the old name is retained as a sub class of the new name in order to maintain backwards compatibility.

New Package de.schlichtherle.key.passwd.console

  1. New: This package has been added to provide a simple key manager implementation which uses console I/O to prompt the user for passwords. This key manager is used by default if the JVM is running in headless mode and the API complies to JSE6 (i.e. the class java.io.Console is available)!  To request it explicitly, set the system property de.schlichtherle.key.KeyManager to de.schlichtherle.key.passwd.console.PromptingKeyManager. This key manager does not support key files and disables prompting if no console is attached to the JVM.

Updates in the Package de.schlichtherle.key.passwd.swing

  1. Fixed: The classes in this package were likely to cause dead locks if the JFileChooser class in the package de.schlichtherle.io.swing was used to browse RAES encrypted ZIP files. This was caused by the implementation of the base class in the javax.swing package and its associated Basic L&F File Loader Threads.
  2. Fixed: The method getParentWindow in the class PromptingKeyManager did not always return the correct window.
  3. Fixed: The event listening in CreateKeyPanel and OpenKeyPanel has been refined so that the Feedback implementation instance is not run every time you switch between password entry mode and key file entry mode. In addition, on Linux, the focus is now properly moved to the input fields upon invalid input. This involved a change in the class AuthenticationPanel, too.
  4. Enhanced: For reliability reasons, CreateKeyPanel now ensures that a key file is read-only.
  5. New: The class HurlingWindowFeedback has been introduced as the successor to QuakingWindowFeedback - the latter is now a sub class of the first. HurlingWindowFeedback has also been improved to use Math.sin(*) to modulate the amplitude of the hurling. Have fun!
  6. Enhanced: The class hierarchy for the Feedback interface and its standard implementations starting with BasicFeedback has been revised to make the standard implementations more reusable. The effect is that the HurlingWindowFeedback implementation can now also be used as the feedback when mismatching passwords or illegal key files for a newly created or updated RAES encrypted ZIP file is entered. In fact, this is the default configuration when using the nzip main class utility. As a result of this, some classes and interfaces have been marked as deprecated.

Updates in the Package de.schlichtherle.swing

  1. Fixed: AbstractComboBoxBrowser now properly initializes the combo box before the first input.
  2. Enhanced: The class EnhancedPanel fires its PanelEvents synchronously again instead of posting them to the Event Queue, thereby reducing event listening problems of some classes in the package de.schlichtherle.key.passwd.swing.

Updates in the Package de.schlichtherle.util.zip

  1. Fixed: Potential racing conditions in ZipFile.getPreambleInputStream() and ZipFile.getPostambleInputStream().
  2. Enhanced: The Zip(File|OutputStream) classes have been refactored so that they now subclass the newly introduced BasicZip(File|OutputStream) classes. Since the relaxed archive driver specification does not require the ZIP driver to be thread safe anymore, the ZIP driver family uses these new subclasses for better performance.
  3. Enhanced: The new method getInputStream(String, boolean, boolean) in the class ZipFile now optionally supports reading of deflated data while checking the CRC-32 value of the inflated data in the close() method of the returned entry input stream. This is used by the checked ZIP driver family in the package de.schlichtherle.io.archive.zip and others in order to support the Direct Data Copying (DDC) feature.