it.unimi.dsi.mg4j.io

Bit-level I/O classes.

Package Specification

The standard Java API lacks bit-level I/O classes: to this purpose, MG4J provides {@link it.unimi.dsi.mg4j.io.InputBitStream} and {@link it.unimi.dsi.mg4j.io.OutputBitStream}, which can wrap any standard Java corresponding stream and make it work at the bit level; moreover, they provide support for several useful formats (such as unary, binary, minimal binary, γ, δ and Golomb encoding).

Compression can be achieved using self-delimiting formats supported by the classes above, or also by arithmetic coding, using the classes {@link it.unimi.dsi.mg4j.io.ArithmeticCoder} and {@link it.unimi.dsi.mg4j.io.ArithmeticDecoder}. Note that arithmetic coding is not very efficient in the present implementation, as it does not allow a varying number of symbols.

Bit input and output streams offer also efficient buffering and a way to reposition the bit stream in case the underlying byte stream is a file-based stream or a {@link it.unimi.dsi.fastutil.io.RepositionableStream}.

Conventions

All coding methods work on natural numbers. The encoding of zero is very natural for some techniques, and much less natural for others. To keep methods rationally organised, all methods are able to encode any natural number. If, for instance, you want to write positive numbers in unary encoding and you do not want to waste a bit, you have to decrement them first (i.e., instead of p you must encode p−1).

This class provides an arithmetic coder.

This class provides an arithmetic decoder.

This class is essentially a dirty trick: it borrows some code and precomputed tables from OutputBitStream and exposes two simple methods ( ByteArrayPostingList.setDocumentPointer(int) and ByteArrayPostingList.addPosition(int) ) with obvious semantics.

This class can be used to wrap an input bit stream.

This class can be used to wrap an output bit stream.

This class provides buffering for readers, but it does so with purposes and an internal logic that are radically different from the ones adopted in java.io.BufferedReader .

There is no support for marking.

java.io.ByteArrayOutputStream is nice, but to get its content you must generate each time a new object.

Warning: the lines returned by iterators generated by instances of this class are not cacheable.

Warning: for simplicity, efficiency and lack of usefulness overflowing and unget features have been removed in MG4J 1.1.

Interpolative coding is a sophisticated compression technique that can be applied to increasing sequences of integers.

The intended usage of this class is that of indexing stuff like lists of document identifiers: if the identifiers contain nonalphabetical characters, the default it.unimi.dsi.mg4j.io.FastBufferedReader might do a poor job.

This stream has length 0, and will always return end-of-file on any read attempt.

This class is a singleton.

This stream discards whatever is written into it.

This reader will always return end-of-file on any read attempt.

This class is a singleton.

This class wraps any OutputStream so that you can treat it as bit stream.

Classes implementing this interface must provide a safety-net finaliser—a finaliser that closes the resource and logs that resource should have been closed.

When the implementing class is abstract, concrete subclasses must call super.close() in their own java.io.Closeable.close method to let the abstract class track correctly the resource.

An instance of this class wraps a given input stream (usually a replicable one, such as a java.io.FileInputStream ) and exposes its contents as a number of separated input streams.

The intended implementations of this interface should decorate a given reader (see, for instance, it.unimi.dsi.mg4j.io.FastBufferedReader ). The reader can be changed at any time using WordReader.setReader(Reader) .

This interface is heavily oriented towards reusability and streaming.