C++ stl stringstream direct buffer access


Answers

std::stringstream doesn't (necessarily) store its buffer contiguously but can allocate chunks as it is gradually filled. If you then want all of its data in a contiguous region of memory then you will need to copy it and that is what str() does for you.

Of course, if you want to use or write a class with a different storage strategy then you can, but you don't then need to use std::stringstream at all.

Question

this should be pretty common yet I find it fascinating that I couldn't find any straight forward solution.

Basically I read in a file over the network into a stringstream. This is the declaration:

std::stringstream membuf(std::ios::in | std::ios::out | std::ios::binary);

Now I have some C library that wants direct access to the read chunk of the memory. How do I get that? Read only access is OK. After the C function is done, I dispose of the memorystream, no need for it.

str() copies the buffer, which seems unnecessary and doubles the memory.

Am I missing something obvious? Maybe a different stl class would work better.

Edit: Apparently, stringstream is not guaranteed to be stored continuously. What is?

if I use vector<char> how do I get byte buffer?




As for the second bullet

Given that it seems easy, but not trivial to implement this, is there any variant available via boost or other sources, that package this functionality?

There is Boost.Iostreams and it even contains an example of how to implement an (o)stream Sink with a string.

I came up with a little test implementation to measure it:

#include <string>
#include <boost/iostreams/stream.hpp>
#include <libs/iostreams/example/container_device.hpp> // container_sink

namespace io = boost::iostreams;
namespace ex = boost::iostreams::example;
typedef ex::container_sink<std::wstring> wstring_sink;
struct my_boost_ostr : public io::stream<wstring_sink> {
    typedef io::stream<wstring_sink> BaseT;
    std::wstring result;
    my_boost_ostr() : BaseT(result)
    { }

    // Note: This is non-const for flush.
    // Suboptimal, but OK for this test.
    const wchar_t* c_str() {
        flush();
        return result.c_str();
    }
};

In the tests I did, using this with it's c_str()helper ran slightly faster than a normal ostringstream with it's copying str().c_str() version.

I do not include measuring code. Performance in this area is very brittle, make sure to measure your use case yourself! (For example, the constructor overhead of a string stream is non-negligible.)




This seems like premature optimization to me. How much work is being done in the processing. Assuming a modernish desktop/server, and not an embedded system, copying a few MB of data during intialization is fairly cheap, especially compared to reading the file off of disk in the first place. I would stick with what you have, measure the system when it is complete, and the decide if the potential performance gains would be worth it. Of course if memory is tight, this is in an inner loop, or a program that gets called often (like once a second), that changes the balance.




boost::iostreams have a few ready made sinks for this. There's array_sink if you have some sort of upper limit and can allocate the chunk upfront, such a sink won't grow dynamically but on the other hand that can be a positive as well. There's also back_inserter_device, which is more generic and works straight up with std::vector for example. An example using back_inserter_device:

#include <string>
#include <iostream>
#include "boost/iostreams/stream_buffer.hpp"
#include "boost/iostreams/device/back_inserter.hpp"
int main() 
{
    std::string destination;
    destination.reserve( 1024 ); 
    boost::iostreams::stream_buffer< boost::iostreams::back_insert_device< std::string > > outBuff( ( destination ) );
    std::streambuf* cur = std::cout.rdbuf( &outBuff );
    std::cout << "Hello!" << std::endl;
    // If we used array_sink we'd need to use tellp here to retrieve how much we've actually written, and don't forgot to flush if you don't end with an endl!
    std::cout.rdbuf( cur );
    std::cout << destination;
}



How to read file content into istringstream?

std::ifstream has a method rdbuf(), that returns a pointer to a filebuf. You can then "push" this filebuf into your stringstream:

int main()
{
    std::ifstream file( "myFile" );

    if ( file )
    {
        std::stringstream buffer;

        buffer << file.rdbuf();

        file.close();

        // operations on the buffer...
    }
}

EDIT: As Martin York remarks in the comments, this might not be the fastest solution since the stringstream's operator<< will read the filebuf character by character. You might want to check his answer, where he uses the ifstream's read method as you used to do, and then set the stringstream buffer to point to the previously allocated memory.






Tags