Best way to store data locally in .NET (C#)


Answers

It really depends on what you're storing. If you're talking about structured data, then either XML or a very lightweight SQL RDBMS like SQLite or SQL Server Compact Edition will work well for you. The SQL solution becomes especially compelling if the data moves beyond a trivial size.

If you're storing large pieces of relatively unstructured data (binary objects like images, for example) then obviously neither a database nor XML solution are appropriate, but given your question I'm guessing it's more of the former than the latter.

Question

I'm writing an application that takes user data and stores it locally for use later. The application will be started and stopped fairly often, and I'd like to make it save/load the data on application start/end.

It'd be fairly straightforward if I used flat files, as the data doesn't really need to be secured (it'll only be stored on this PC). The options I believe are thus:

  • Flat files
  • XML
  • SQL DB

Flat files require a bit more effort to maintain (no built in classes like with XML), however I haven't used XML before, and SQL seems like overkill for this relatively easy task.

Are there any other avenues worth exploring? If not, which of these is the best solution?


Edit: To add a little more data to the problem, basically the only thing I'd like to store is a Dictionary that looks like this

Dictionary<string, List<Account>> 

where Account is another custom type.

Would I serialize the dict as the xmlroot, and then the Account type as attributes?


Update 2:

So it's possible to serialize a dictionary. What makes it complicated is that the value for this dict is a generic itself, which is a list of complex data structures of type Account. Each Account is fairly simple, it's just a bunch of properties.

It is my understanding that the goal here is to try and end up with this:

<Username1>
    <Account1>
        <Data1>data1</Data1>
        <Data2>data2</Data2>
    </Account1>
</Username1>
<Username2>
    <Account1>
        <Data1>data1</Data1>
        <Data2>data2</Data2>
    </Account1>
    <Account2>
        <Data1>data1</Data1>
        <Data2>data2</Data2>
    </Account2>
 </Username2>

As you can see the heirachy is

  • Username (string of dict) >
  • Account (each account in the List) >
  • Account data (ie class properties).

Obtaining this layout from a Dictionary<Username, List<Account>> is the tricky bit, and the essence of this question.

There are plenty of 'how to' responses here on serialisation, which is my fault since I didn't make it clearer early on, but now I'm looking for a definite solution.




How to decide where to store per-user state? Registry? AppData? Isolated Storage?

If you have a small number of key/value pairs and the values aren't big the registry is great - and you don't care about xcopy deployment - then use the registry (I know this isn't exact, but it's usually obvious when working with the registry becomes a pain).

If you want xcopy deployment the data must be in the same folder as the program obviously - but the program can be somewhere under the AppData folder, it doesn't have to be under "program files".

Use isolated storage only when you need it or have to use it - for example ClickOnce.

Otherwise use AppData\Roaming, use Local or LocalLow only if you have a good reason.

EDIT: Here is the difference between Roaming, Local and LocalLow:

Windows has a little known feature called "roaming profiles", the general idea is that in a corporate environment with this feature enabled any user can use any computer.

When a user logs in his private settings are downloaded from the server and when he logs out his settings are uploaded back to the server (the actual process is more complicated, obviously).

Files in the User's "Roaming" folder in Vista or "Application Data" in XP move around with the user - so any settings and data should be stored there.

Files under "Local" and "LocalLow" in vista and "Local Settings" in XP do not, so it's a good place for temp files, things that are tied to the specific computer or data that can be recalculated.

In Vista, as part of the new security features we all know and love, you can have programs running in "low integrity mode" (for example IE in protected mode), those programs are running with reduced privileges and can't access files in the user's profile - except for files under the "LocalLow" folder.

So, in conclusion, files stored in "LocalLow" are inherently insecure and files in "Local"/"Local Settings" are likely to be unavailable in some large companies - so unless you have good reason and know exactly what you are doing go with "Roaming"/"Application Data".




Have you considered something like db4o? The licensing might restrict you but it would seem to fit the bill otherwise.




Versioning friendly, extendible binary file format

Have you considered using SQL Server Compact Edition?

  1. It has plenty of .NET support
  2. The versioning of the schema and the ability for new versions of your application handling old schemas would be entirely in your control. Versioning of SQL Server Compact should be somewhat seemless beyond your application using features in a newer version that did not exist in the older version.
  3. You have the most of the SQL syntax available to you for querying.
  4. Obviously from the name, this version of SQL Server was designed for embedded systems which can include applications that want to avoid installation of SQL Express or the full blown version of SQL Server.

Now, this would have the same issues as SQLite in that the data structure, from what you have told us, could get complicated, but that will be true even if you roll you own binary format.

Btw, it occurs to me that you haven't clarified what exactly is meant by "sizeable". If "sizeable" means close to or more than 4 GB, obviously SQL Compact will not work nor will a host of other database file formats.

EDIT I notice that you have added SQL Compact Edition to your list of "too heavyweight" list after my post. SQL Compact requires only 5MB of RAM and 2MB of disk storage depending on the size of the database. So, the problem cannot be that is heavyweight. Now, as to the second point of claiming the data structure would be pretty complicated. If that is true, I suspect it will be true of any relational database product and rolling your own binary format will be even more complicated. Given that, you might look at non-relational database products such as mongodb.







If you haven't done it yet it would be best to start with XML file input/output before getting into anything too advanced.

Normally you would read and write to files by using the following method to get the path:

string appDataPath = Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData);

So if you want to store your data in a folder called "Cookbook" and a file called "recipes.xml" you could do the following:

string dataPath = Path.Combine(appDataPath, "Cookbook");
string recipesFileFullPath = Path.Combine(dataPath, "recipes.xml");

This gives you a path like C:\Users\John\AppData\Local\Cookbook\recipes.xml or something similar which you can pass to file input and output functions.

Then you can get started with the System.IO namespace classes like File and FileStream to learn how to properly open and read/write to files.

Then the next higher level step is to pass these file streams to something used to read and write XML to objects, such as Linq to XML (the XDocument class) which is the preferred approach. Or the older XmlSerializer.

Edit:

Here's some sample code to create an object and save it to an XML file:

public class RecipeBook
{
    public List<Recipe> Recipes { get; set; }

    public RecipeBook()
    {
        Recipes = new List<Recipe>();
    }
}

public class Recipe
{
    public DateTime LastModified { get; set; }
    public DateTime Created { get; set; }
    public string Instructions { get; set; }
}

public void SomeFunction()
{
    RecipeBook recipeBook = new RecipeBook();

    var myRecipe = new Recipe()
    {
        Created = DateTime.Now,
        LastModified = DateTime.Now,
        Instructions = "This is how you make a cake."
    };

    recipeBook.Recipes.Add(myRecipe);

    var doc = new XDocument();
    using (var writer = doc.CreateWriter())
    {
        var serializer = new XmlSerializer(typeof(RecipeBook));

        serializer.Serialize(writer, recipeBook);
    }

    doc.Save(recipesFileFullPath);
}

You would just need to break this code out into a structure that works for you. For example, if you were making a Windows Forms application then the RecipeBook would be a private member variable of your main form. In the constructor you could construct the recipesFileFullPath string and store it as a private member variable too. On the Form.Loaded event you could check if the XML file already exists and if so load it. If not you would create a new RecipeBook class that's empty. You would also probably only serialize and save when the user clicks a save button or when the Form.Closing event is raised.

EDIT:

To deserialize and read from a file you can do the following:

var serializer = new XmlSerializer(typeof(RecipeBook));
using (var fs = new FileStream(recipesFileFullPath))
{
    RecipeBook book = (RecipeBook)serializer.Deserialize(fs);
}






Tags