Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ Core » Optional serialization techniques
Optional serialization techniques [message #31354] Thu, 24 February 2011 16:27 Go to next message
Mindtraveller is currently offline  Mindtraveller
Messages: 917
Registered: August 2007
Location: Russia, Moscow rgn.
Experienced Contributor

I have a version 1.0 of my application which serializes a VectorMap of some object into the file with StoreToFile. We of course know that if VectorMap object is being changed, the whole de-serialization is failed.
So here is my problem: I develop 1.1 version with objects which are slightly different. And actually what I want is that 1.1 version reads everything from config file ignoring the fact that objects can't be de-serialized completely (I just add more members since 1.0).
I don't want to make object members 'dynamic' (using VectorMap<String,Value> instead of plain members).
Yes, and I really don't want to use XML as speed is the most important in this case. And there is no problem just adding new members since new version (not removing old or replacing them).

Can you please suggest the most effective way of doing it?
Effective means the most quickly working while not rewriting all of U++ serialization code Cool

[Updated on: Thu, 24 February 2011 18:29]

Report message to a moderator

Re: Optional serialization techniques [message #31357 is a reply to message #31354] Fri, 25 February 2011 09:46 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Mindtraveller wrote on Thu, 24 February 2011 10:27

I have a version 1.0 of my application which serializes a VectorMap of some object into the file with StoreToFile. We of course know that if VectorMap object is being changed, the whole de-serialization is failed.
So here is my problem: I develop 1.1 version with objects which are slightly different. And actually what I want is that 1.1 version reads everything from config file ignoring the fact that objects can't be de-serialized completely (I just add more members since 1.0).
I don't want to make object members 'dynamic' (using VectorMap<String,Value> instead of plain members).
Yes, and I really don't want to use XML as speed is the most important in this case. And there is no problem just adding new members since new version (not removing old or replacing them).

Can you please suggest the most effective way of doing it?
Effective means the most quickly working while not rewriting all of U++ serialization code Cool


I usually do:

void Foo::Serialize(Stream& s)
{
   int version = 0;
   s / version;
   s % x % y;
}


... and later I add 'z' to Foo:

void Foo::Serialize(Stream& s)
{
   int version = 1;
   s / version;
   s % x % y;
   if(version >= 1) {
      s % z;
   }
}


That said, it does not solve the problem all the time and generally, I would NOT recommend using binary serialization for permanent storage of important files. It is fine for configs (where if you loose one, it is not that bad) or for transfering data (e.g. over network).

Do not use it for documents Smile

Mirek
Re: Optional serialization techniques [message #31358 is a reply to message #31357] Fri, 25 February 2011 10:14 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
I prefer XML serialization that is perfect for development and debugging. For end users that do not have to see the data, I just encrypt the XML file.

Best regards
IƱaki
Re: Optional serialization techniques [message #31363 is a reply to message #31354] Fri, 25 February 2011 11:16 Go to previous messageGo to next message
chickenk is currently offline  chickenk
Messages: 169
Registered: May 2007
Location: Grenoble, France
Experienced Member
Maybe what you search for is some kind of migration procedure so that your 1.0 serialized objects could be automatically migrated to 1.1 format before loading.

The migration layer is still to be done, but that would mean no code change in your config loader, just make it deserialize 1.1 objects. the additional code would be isolated.

If you want to keep your original 1.0 objects, you can save the 1.1 migrated file in the same place with a filename change so that both are available then. And when you detect a 1.0 version, you try to find or generate the 1.1 version.

that would allow an automatic upgrade of saved files for your users. Then, the migration code would simply include the default values for a specific migration.

A simple migration class would only add the required changes between 2 specific version. Then you can chain the migrations if several versions appeared.

For example, a user has 1.0 format and upgrades it software where the 1.2 version of the data format is used. Before loading, the 1.0 version is detected, and then:

1.0 --> migrate_100_to_110 --> 1.1 --> migrate_110_to_120 --> 1.2

This migration system is used by some existing software such as Ruby on Rails (which gave me this idea for you).

Not sure it can apply, but that could be something to dig into, and maybe a small bazaar package in sight? Wink

[Updated on: Fri, 25 February 2011 12:03]

Report message to a moderator

Re: Optional serialization techniques [message #31368 is a reply to message #31357] Fri, 25 February 2011 15:34 Go to previous messageGo to next message
Mindtraveller is currently offline  Mindtraveller
Messages: 917
Registered: August 2007
Location: Russia, Moscow rgn.
Experienced Contributor

Thanks everyone for the answers. I guess the problem is solved.

mirek wrote on Fri, 25 February 2011 11:46

That said, it does not solve the problem all the time and generally, I would NOT recommend using binary serialization for permanent storage of important files. It is fine for configs (where if you loose one, it is not that bad) or for transfering data (e.g. over network).
Do not use it for documents Smile

BTW, why? Looks like not a bad storage solution.
Re: Optional serialization techniques [message #31371 is a reply to message #31368] Fri, 25 February 2011 19:42 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Mindtraveller wrote on Fri, 25 February 2011 09:34

Thanks everyone for the answers. I guess the problem is solved.

mirek wrote on Fri, 25 February 2011 11:46

That said, it does not solve the problem all the time and generally, I would NOT recommend using binary serialization for permanent storage of important files. It is fine for configs (where if you loose one, it is not that bad) or for transfering data (e.g. over network).
Do not use it for documents Smile

BTW, why? Looks like not a bad storage solution.


I guess the main problem is that it is so simple to serialize things that it is too easy to accidentally break the format...

And, sometimes, the problem also is that it is impossible or too diffuclt to keep backward compatibility. Simple example would be forgetting to put 'version' somewhere...

Mirek
Re: Optional serialization techniques [message #31378 is a reply to message #31371] Sat, 26 February 2011 07:49 Go to previous messageGo to next message
tojocky is currently offline  tojocky
Messages: 607
Registered: April 2008
Location: UK
Contributor

mirek wrote on Fri, 25 February 2011 20:42



I guess the main problem is that it is so simple to serialize things that it is too easy to accidentally break the format...

And, sometimes, the problem also is that it is impossible or too diffuclt to keep backward compatibility. Simple example would be forgetting to put 'version' somewhere...

Mirek


Mirek, What about to migrate to a XML serialization? The things will be easy!
Your realization is very nice, but debugging and keeping backward compatibility is a little hard.

[Updated on: Sat, 26 February 2011 08:10]

Report message to a moderator

Re: Optional serialization techniques [message #31459 is a reply to message #31378] Fri, 04 March 2011 10:01 Go to previous message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
tojocky wrote on Sat, 26 February 2011 01:49

mirek wrote on Fri, 25 February 2011 20:42



I guess the main problem is that it is so simple to serialize things that it is too easy to accidentally break the format...

And, sometimes, the problem also is that it is impossible or too diffuclt to keep backward compatibility. Simple example would be forgetting to put 'version' somewhere...

Mirek


Mirek, What about to migrate to a XML serialization? The things will be easy!



Uh, why? For what it is used, binary serialization is fine.

Quote:


Your realization is very nice, but debugging and keeping backward compatibility is a little hard.



I guess all this says is that you should not use it where backward compatibility is important.

Mirek
Previous Topic: do upp have an equivalent of "eval()" in Javascript
Next Topic: Core/Path.cpp: the limitation of GetCurrentDirectory for POSIX
Goto Forum:
  


Current Time: Thu Mar 28 21:33:50 CET 2024

Total time taken to generate the page: 0.01854 seconds