Setting up the Class Path

With its default configuration file, TrueZIP 6 is self contained, so all you need to add to the class path is the file truezip.jar. Note that the file name just includes the major version number in order to express that a later version should be binary compatible to any previous version with the same major version number.

If the configuration gets customized, additional JARs may be required on the run time class path. Please see below how to customize the configuration and refer to the API Overview for the available options and dependencies.

Important: If TrueZIP is going to be used in a multi-class-loader environment (e.g. application servers), it must be added to the boot class path or extension class path. Otherwise, the different class definitions may shadow and bypass each other's associated archive file state, which may even cause loss of data! Please refer to the section "Third Party Access" in the Javadoc for the package de.schlichtherle.io for for full details and workarounds. Simply consider TrueZIP being an extension to the JRE, not part of a web application.

Importing TrueZIP

In order to use TrueZIP, you need to import its packages. Unless otherwise noted, only the high level API in the package de.schlichtherle.io is required for the examples in this tutorial. Because it provides drop-in replacements for some equally named classes in the package java.io, the following boilerplate is required:

import de.schlichtherle.io.*;
import de.schlichtherle.io.File;
import de.schlichtherle.io.FileInputStream;
import de.schlichtherle.io.FileOutputStream;
import java.io.*;
// ...
File file = new File("archive.zip"); // de.schlichtherle.io.File!

Please do not do this instead:

import java.io.*;
// ...
de.schlichtherle.io.File file = new de.schlichtherle.io.File("archive.zip");

This is for the following reasons:

1. Accidentally using java.io.File and de.schlichtherle.io.File instances referring to the same path concurrently will result in erroneous behaviour and may even cause loss of data! Please refer to the section "Third Party Access" in the Javadoc for the package de.schlichtherle.io for full details and workarounds.
2. A de.schlichtherle.io.File subclasses java.io.File and thanks to polymorphism can be used everywhere a java.io.File could be used.
3. Shorter source code is much easier to read.
4. I'm not vain enough to want you write my name all over the place. If you're becoming a fan, please make a donation instead. ;-)

If you still need to use java.io.File, please use this fully qualified class name.

Casting Return Values

Some methods in the File class return new File instances. These methods are guaranteed to return a de.schlichtherle.io.File. However, since TrueZIP 6 is compiled with -source 1.4, the overridden methods are still declared to return a java.io.File and hence a cast may be required:

File file = new File("archive.zip/entry");
File parent = (File) file.getParentFile(); // cast required!

This is going to change in TrueZIP 7, which will require Java SE 5.

Basic Operations

You may already guess how to create a new ZIP file by now:

new File("archive.zip").mkdir();

This works for any archive file suffixes which are configured to be recognized. By default, this is "ear|jar|war|zip".

Note that the call to mkdir() is actually redundant: TrueZIP creates archive files and any missing directories inside on the fly unless setLenient(false) has been called to switch off lenient behavior. Note again that this feature only works for archive files and directories inside archive files: The parent directory of a top level archive file must always exist.

Since TrueZIP 6.5, if the call to mkdir() is omitted, TrueZIP will not create an entry for the directory in the output archive file. This is to mimic the behavior of most archive utilities which do not create archive entries for directories.

For example, to start writing a new file entry in a JAR archive you may simply use:

OutputStream out = new FileOutputStream("my-killer-app.jar/META-INF/LICENSE.TXT");
try {
    // Do I/O here...
} finally {
    out.close(); // ALWAYS close the stream!
}

However, it's highly recommended to use one of the more advanced cat* and copy* methods in the File class instead whenever applicable. These methods provide ease of use, enhanced features, superior performance and require less space in the temp file folder (see below).

Note how the code ensures to close the stream even if an IOException is thrown by calling close() in a finally-block: If the client application does not properly close its streams, TrueZIP may throw a FileBusyException, an ArchiveBusyWarningException or an ArchiveBusyException on certain operations, as documented in the Javadoc. This idiom is not at all specific to TrueZIP: Streams often utilize OS resources such as file descriptors, database or network connections. All OS resources are limited however and sometimes they are even exclusively allocated for a stream, so the stream should always be closed as soon as possible again, especially in long running server applications (relying on finalize() to do this during garbage collection is unsafe). Unfortunately, many Java applications and libraries fail in this respect.

The top level entries in an archive file build its virtual root directory. Just like a regular directory, you can list its contents like follows:

String[] members = new File("my-killer-app.jar").list();

Following the previous example, just new String[] { "META-INF" } would be returned. So just like a regular directory, only the contents of the virtual root directory are listed, not including the contents of its directory members (this method is not recursive).

Note that META-INF is returned although it hasn't been created with mkdir() and hence will not be output to the resulting JAR file. Such a directory is called a ghost directory: A ghost directory behaves like a regular directory with the exception that its last modification time returned by lastModified() is 0L. If the client application sets the last modification time explicitly using setLastModified(long), then the ghost directory reincarnates as a regular directory and will be output to the archive file.

The ArchiveDetector Interface

Whenever a File instance is constructed, an instance of the ArchiveDetector interface is assigned to it which detects archive files solely by scanning the file path - usually by testing for file name suffixes like .zip or the like. Whenever an archive file is recognized, the ArchiveDetector.getArchiveDriver(String) method returns an instance of the ArchiveDriver interface which allows to access it.

ArchiveDetector instances are assigned to File instances in the following way:

1. If an archive detector is explicitly provided as a parameter to the constructor of the File class or any other method which creates File instances (e.g. listFiles(*)), then this archive detector is used.
2. Otherwise, the archive detector returned by File.getDefaultArchiveDetector() is used. This is initially set to the predefined instance ArchiveDetector.DEFAULT . Both the class property and the predefined instance can be customized (see below).

For your convenience, the ArchiveDetector interface provides three predefined instances as constant fields. These are actually instances of the DefaultArchiveDetector class (see next section).

• ArchiveDetector.NULL never recognizes archive files in a path. This can be used as the end of a chain of DefaultArchiveDetector instances or if archive files shall be treated like ordinary files rather than (virtual) directories.
• ArchiveDetector.DEFAULT recognizes the archive file suffixes defined by the key DEFAULT in the configuration file(s). If only TrueZIP's default configuration file is used, then this is set so that no additional JARs are required on the run time class path.
• ArchiveDetector.ALL recognizes all archive file suffixes registered in the global registry by the configuration file(s). This requires additional JARs on the run time class path.

The DefaultArchiveDetector Class

The class DefaultArchiveDetector is the default implementation of the ArchiveDetector interface. Each instance matches file paths against a pattern of archive file suffixes in order to detect prospective archive files and look up their corresponding archive driver in its registry.

When this class is loaded, it uses the current thread's context class loader to enumerate all instances of the relative path META-INF/services/de.schlichtherle.io.registry.properties on the class path. These configuration files are processed in arbitrary order to configure the global registry of archive file suffixes and archive drivers. This allows archive drivers to be "plugged in" by simply providing their own configuration file somewhere on the class path. One such instance is located inside truezip.jar and contains TrueZIP's default configuration (please refer to this file for full details on the syntax). Likewise, the client application may provide its own configuration file somewhere on the class path in order to extend or override the settings configured by TrueZIP and any optional plug-in drivers.

Each instance has a local registry. Constructors are provided which allow an instance to:

1. Filter the set of archive file suffixes in the global registry. For example, "tar|zip" could be accepted by the filter in order to recognize only the TAR and ZIP file formats.
2. Add custom archive file suffixes for supported archive types to the local registry in order to create custom archive types. For example, "myapp" could be added as a custom archive file suffix for the JAR file format.
3. Add custom archive file suffixes and archive drivers to the local registry in order to support new archive types. For example, the suffix "7z" could be associated to a custom archive driver which supports the 7z file format.
4. Put multiple instances in a chain of responsibility: The first instance which holds a mapping for any given archive file suffix in its registry determines the archive driver to be used.

Altogether, this enables to build arbitrary complex configurations with very few lines of Java code or properties in the configuration file(s).

Updating Archive Files

To provide random read/write access to archive files, TrueZIP needs to associate some state for every recognized archive file on the heap and in the folder for temporary files while the client application is operating on the VFS.

TrueZIP performs the mounting and unmounting of the VFS for archive files implicitly and you can safely rely on this. However, sometimes explicit unmounting may be required if...

1. third parties require access to the same path, or...
2. some control is required over the exceptions which may be thrown when unmounting, or...
3. the progress of updating large archive files shall get monitored.

In this context, third parties are:

1. Instances of the class java.io.File which are not instances of the class de.schlichtherle.io.File.
2. Instances of the class de.schlichtherle.io.File which do not recognize the same set of archive files in the path due to the use of a differently working de.schlichtherle.io.ArchiveDetector.
3. Other definitions of the classes in this package which have been loaded by different class loaders.
4. Other system processes.

Explicit unmounting can simply be performed by calling File.umount() like this:

try {
    File.umount(); // with or without parameters
} catch (ArchiveWarningException oops) {
    // Only instances of the class ArchiveWarningException exist in
    // the sequential chain of exceptions. We decide to ignore this.
} catch (ArchiveException ouch) {
    // At least one exception occured which is not just an
    // ArchiveWarningException. This is a severe situation which
    // needs to be handled.
    // Print the sequential chain of exceptions in order of
    // descending priority and ascending appearance.
    //ouch.printStackTrace();
    // Print the sequential chain of exceptions in order of
    // appearance instead.
    ouch.sortAppearance().printStackTrace();
}

However, overly calling umount() can cause serious performance degradation: Unmounting a modified archive file is a linear runtime operation. If the size of the resulting archive file is s bytes, the operation always completes in O(s), even if only a single, small archive entry has been modified within a very large archive file. Unmounting an unmodified or newly created archive file is a constant runtime operation: It always completes in O(1). These magnitudes are independent of whether unmounting was performed explicitly or implicitly. Now if the client application modifies each entry in a loop and accidentally triggers unmounting the archive file on each iteration, then the overall runtime increases to O(s*s)!

Here are some guidelines to find the right balance between performance and control:

1. When the JVM terminates, TrueZIP's JVM shutdown takes care of unmounting and prints the stacktrace of any exceptions on the standard error output. However, calling umount() is recommended in order to handle exceptions explicitly. Furthermore, shutdown hooks shall not take long, but updating a large archive file may take a considerable amount of time doing I/O.
2. Otherwise, in order to achieve best performance, umount() or update() should not get called unless either third party access or explicit exception handling is required.
3. For the same reason, these methods should never get called in a loop which modifies an archive file.
4. umount() is generally preferred over update() for safety reasons.

To get the full story, please refer to the package Javadoc for de.schlichtherle.io.

Recognizing False Positives

If an archive detector hits a file path which probably denotes an archive file, the file is said to be a prospective archive file. On the first read or write access to a prospective archive file, TrueZIP checks its true state in cooperation with the registered archive driver for the respective file suffix. If the true state of the file turns out to be actually a directory or to be incompatible with the archive file format, it's said to be a false positive archive file.

Just like a native file system, the behavior of all read and write operations in TrueZIP depends on the true state of a file. This is an important concept which ensures that TrueZIP cannot get fooled by a false positive archive file. However, a false positive archive file sometimes fools the user. Consider the following example: The ZIP file archive.zip has been created by 7-Zip and uses 7-Zip's popular, but proprietary LZMA compression. Now a user would probably expect new File("archive.zip").list() to return the contents of the virtual root directory in the archive file. However, this method simply returns null because archive.zip is not compatible to the ZIP file format specification and hence is treated like a regular file. For regular files however, the contract of File.list() requires to return null.

Typical Configuration Tasks

Approaches to accomplish these Tasks

This section explains the different approaches to accomplish these two tasks in order of descending preference and ascending priority.

DEFAULT=tar.bz2|tar.gz|zip

DEFAULT=foo|bar
foo=de.schlichtherle.io.archive.zip.JarDriver
bar=de.schlichtherle.io.archive.zip.raes.SafeZipRaesDriver

$ java -Dde.schlichtherle.io.default=tar.bz2|tar.gz|zip -cp truezip.jar ...

File.setDefaultArchiveDetector(new DefaultArchiveDetector("tar.bz2|tar.gz|zip"));

File.setDefaultArchiveDetector(new DefaultArchiveDetector(
      ArchiveDetector.NULL, // delegate
      new String[] {
          "foo", "de.schlichtherle.io.archive.zip.JarDriver",
           "bar", "de.schlichtherle.io.archive.zip.raes.SafeZipRaesDriver",
      }));

ArchiveDetector detector = new DefaultArchiveDetector("tar.bz2|tar.gz|zip");
File file = new File("archive.zip", detector);

ArchiveDetector detector = new DefaultArchiveDetector(
       ArchiveDetector.NULL,
       new Object[] {
          "foo", new de.schlichtherle.io.archive.zip.JarDriver(),
          "bar", new de.schlichtherle.io.archive.zip.raes.SafeZipRaesDriver(),
      }));
File file = new File("archive.foo", detector);

Converting Archive Files to RAES encrypted ZIP Files and vice versa

TrueZIP's JAR contains a utility main class called nzip in the base package which you can use to easily convert any archive file to an RAES encrypted ZIP file and vice versa.

The following example converts an ordinary ZIP file to an RAES encrypted ZIP file (this example assumes a UNIX platform):

$ rm archive.tzp
$ java -cp truezip.jar:lcrypto.jar nzip cp archive.zip archive.tzp

The following call reverses the operation:

$ rm archive.zip
$ java -cp truezip.jar:lcrypto.jar nzip cp archive.tzp archive.zip

The initial removal of the target file is just required to prevent the nzip utility main class to copy the source archive into the destination archive if the latter already exists.

The nzip utility main class provides a lot of other Unix-like commands (cp, ll, rm, mv, rm, ...). For more information, please refer to its inline help which is printed if the class is run without arguments.

Private Key Management

Like any kind of symmetric cipher, the AES cipher used for the RAES file format requires a private key. In order to maintain full transparency for the client application, this archive type specific "detail" is handled by the archive driver in cooperation with the KeyManager class in the package de.schlichtherle.key.

The key manager is a singleton which maps protected resources to their respective key provider: A protected resource can be virtually anything which can be identified by a unique string (the resource ID) and the key provider can be any instance of the KeyProvider interface. This interface is generic enough to be agnostic of the actual source and type of a private key: The source could be a GUI dialog, a text console dialog, a secure database, a secure file, etc. and the key may be an arbitrary object.

public class SimpleAesKeyProvider implements AesKeyProvider {
    public Object getCreateKey() throws UnknownKeyException {
        return "secret".toCharArray(); // returns cloned char array!
    }

    public Object getOpenKey() throws UnknownKeyException {
        return "secret".toCharArray(); // returns cloned char array!
    }

    public void invalidOpenKey() {
        // This method is called whenever a key for an existing protected
        // resource is invalid. A real application should deal with this
        // appropriately, of course.
        throw new UnsupportedOperationException("cannot handle invalid keys!");
    }

    public int getKeyStrength() {
        return KEY_STRENGTH_256;
    }

    public void setKeyStrength(int keyStrength) {
    }
}

// ...

KeyManager.getInstance().setKeyProvider(
        "/home/acme/archive.tzp",
        new SimpleAesKeyProvider());

public class CustomKeyManager extends KeyManager {
    public CustomKeyManager() {
        mapKeyProviderType(AesKeyProvider.class, SimpleAesKeyProvider.class);
    }
}

// ...

System.setProperty("de.schlichtherle.key.KeyManager", "com.acme.CustomKeyManager"));

public class SimpleRaesParameters implements Type0RaesParameters {
    public char[] getCreatePasswd() throws RaesKeyException {
        return "secret".toCharArray(); // returns cloned char array!
    }

    public char[] getOpenPasswd() throws RaesKeyException {
        return "secret".toCharArray(); // returns cloned char array!
    }

    public void invalidOpenPasswd() {
        throw new UnsupportedOperationException("cannot handle invalid passwords!");
    }

    public int getKeyStrength() {
        return KEY_STRENGTH_256;
    }

    public void setKeyStrength(int keyStrength) {
    }
}

public class CustomArchiveDriver extends SafeZipRaesDriver {
    public RaesParameters getRaesParameters(Archive archive) {
        return new SimpleRaesParameters();
    }
}

// ...

File.setDefaultArchiveDetector(new DefaultArchiveDetector(
        ArchiveDetector.DEFAULT, // delegate
        "app", new CustomArchiveDriver()));