Compressed IOStreams
The class
RWCompressedStreamBuffer derives from
std::streambuf, and provides a mechanism to compress data that is serialized to an
std::ostream or deserialized from an
std::istream. RWCompressedStreamBuffer is a class template with a single template parameter that specifies the compressor type to be used. Rogue Wave provides the class
RWSnappyCompressor based on the Google® snappy compression algorithm. Alternatively, you can use your own compressor class, whose only requirement is that it provide the same API as
RWSnappyCompressor.
NOTE >> RWSnappyCompressor and its related typedefs depend on the third-party library
snappy, located in your
<installdir>\3rdparty directory.
The
RWCompressedStreamBuffer-based streams are used to compress and serialize object representations to an associated stream buffer. Objects can be serialized directly to or from these stream classes just as you would with any standard stream:
std::stringstream string_stream;
RWCompressedIOStream<RWSnapyCompressor> compressed_stream(string_stream);
RWCString str (a,10000);
compressed_stream << str;
The
RWCompressedStreamBuffer-based streams can also be used with the Essential Tools Module virtual streams classes. For example, here is a method to write an
RWCollectable to an
-std::ofstream in a portable format:
void write_collectable_to_file (std::ofstream& os, const RWCollectable& obj)
{
RWeostream eos (os);
eos << obj;
}
The above function writes the provided object to a
std::ofstream by way of an
RWeostream. To add compression, you would simply do this:
void write_collectable_to_file (std::ofstream& os, const RWCollectable& obj)
{
RWCompressedOStream<RWSnappyCompressor> compress (os);
RWeostream eos (compress);
eos << obj;
}
The
RWCompressedOStream simply takes the portable object representation generated by the
RWeostream, compresses it using an
RWSnappyCompressor, and then forwards the compressed data along to the
std::ofstream for storage on disk, just as in the previous example.
Here is another example for writing and reading compressed data from an
RWSocketPortal:
RWSocketPortal socket_portal (socket);
RWPortalStreambuf portal_stream_buffer (socket_portal);
RWCompressedIOStream compressed_stream (&portal_stream_buffer);
RWCString data (‘a’,10000);
compressed_stream << data;
compressed_stream >> data;
After creating a portal, and a stream for reading from and writing to the portal, the above code creates an
RWCompressedIOStream. The final lines stream the data to and from the portal.
Buffer Size
The constructors for each of the compressed stream classes take a buffer size parameter. If not provided, the default size is 2048U. Data sent to the stream buffer is buffered in a block of the provided size until there is no room for additional data. Once the buffer is filled, it is compressed and forwarded along to the underlying stream buffer. A larger buffer tends to improve the compression ratio, but makes large blocks of contiguous memory unavailable to the rest of your application. A smaller value mitigates the memory problem, but the compression ratio suffers. This is something you need to adjust in the context of your application to decide on the value that produces the best performance.
Typedefs
The following convenience typedefs are provided:
• RWSnappyStreamBuffer
• RWSnappyIOStream
• RWSnappyIStream
• RWSnappyOStream
Use these to create IOStream objects using the default Snappy compressor. For example:
RWSnappyIOStream compressed_stream;
Custom Compressor Interface
In general, the interface for a custom compressor must match that of
RWSnappyCompressor. More specifically:
• The interface must include the methods compress(), decompress(), and max_compressed_length(). The method uncompressed_length() is not required, but may be helpful.
• The above methods must return a negative value to indicate failure, and 0 to indicate success.