Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ Core » GZDecompress bug
GZDecompress bug [message #22669] Tue, 04 August 2009 05:49 Go to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
GZDecompress has a bug when size of an archive is bigger than size of an archived file.

I attached a test case. Sorry, I couldn't figure out myself how to fix that.


Regards,
Novo
Re: GZDecompress bug [message #22679 is a reply to message #22669] Tue, 04 August 2009 20:05 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Novo wrote on Mon, 03 August 2009 23:49

GZDecompress has a bug when size of an archive is bigger than size of an archived file.

I attached a test case. Sorry, I couldn't figure out myself how to fix that.



I am quite confused with testcase. Everything seems to be OK, except that in the case with .gz with longer name (not used in testcase), the size of memory stream is not long enough..

Mirek
Re: GZDecompress bug [message #22682 is a reply to message #22679] Wed, 05 August 2009 05:40 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
luzr wrote on Tue, 04 August 2009 14:05

Novo wrote on Mon, 03 August 2009 23:49

GZDecompress has a bug when size of an archive is bigger than size of an archived file.

I attached a test case. Sorry, I couldn't figure out myself how to fix that.



I am quite confused with testcase. Everything seems to be OK, except that in the case with .gz with longer name (not used in testcase), the size of memory stream is not long enough..

Mirek


If I understand correctly memory stream size is supposed to be equal to the size of an archived file (because I'm extracting this file into the memory stream). At least this test works fine with archives where size of an archived file is bigger than size of an archive itself.

Another .gz file, which is not used in the testcase, is just another example. If you want to use it, then idx_file_size should be set to 80.

Sorry, I forgot to describe wrong behavior. If you extract archived file using another application (I usually use Total Commander for such operations), and using test_gunzip and compare contents of extracted files you will see the difference.

Actually, the "a.a.a" file will contain the "eeFreeFreeFreeFreeFreeFree" suffix.

I tested that on Windows Vista 32 bit.

I ran a similar application on 114 files and only these two were extracted not correctly.


Regards,
Novo
Re: GZDecompress bug [message #22685 is a reply to message #22682] Wed, 05 August 2009 08:23 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Novo wrote on Tue, 04 August 2009 23:40


Sorry, I forgot to describe wrong behavior. If you extract archived file using another application (I usually use Total Commander for such operations), and using test_gunzip and compare contents of extracted files you will see the difference.

Actually, the "a.a.a" file will contain the "eeFreeFreeFreeFreeFreeFree" suffix.



OK, thanks. I was in impression that the size is wrong...

Mirek
Re: GZDecompress bug [message #22687 is a reply to message #22685] Wed, 05 August 2009 08:49 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
int GZDecompress(Stream& out, Stream& in, int size, Gate2<int, int> progress)


Here the 'size' parameter is a number of bytes to be read from in.

The size of .gz is 75 bytes, but you put there 48.

#include <Core/Core.h>

using namespace Upp;

CONSOLE_APP_MAIN
{
	FileOut out("u:/gztest/output.bin");
	FileIn in;

	if (!in.Open("u:/gztest/MedicalEnRu_abrv.idx.gz"))
		return;
	
	Buffer<char> index_data;
	MemStream index_stream;
	const int idx_file_size = 48;

	// Preallocate memory and create a memory stream ...
	index_data.Alloc(idx_file_size);
	index_stream.Create(~index_data, idx_file_size);

	GZDecompress(index_stream, in, in.GetLeft());
	index_stream.Seek(0);
			
	CopyStream(out, index_stream);
}


This works.

BTW, why MemStream? You can use 'out' as output directly without CopyStream.

Mirek
Re: GZDecompress bug [message #22691 is a reply to message #22687] Wed, 05 August 2009 17:17 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
luzr wrote on Wed, 05 August 2009 02:49

int GZDecompress(Stream& out, Stream& in, int size, Gate2<int, int> progress)


Here the 'size' parameter is a number of bytes to be read from in.

The size of .gz is 75 bytes, but you put there 48.



GZDecompress(index_stream, in, in.GetSize());


Thanks! That helped. I was confused by the name “size”. IMO there was no need to provide size of the input stream because it can be easily retrieved from the input stream. The only useful size I could imagine in this situation was a minimal size of the output stream (to avoid multiple reallocations).

A few lines of documentation would help me a lot in this situation.


Quote:


BTW, why MemStream? You can use 'out' as output directly without CopyStream.

Mirek


This is just a testing application. In a real application I keep uncompressed data in memory.

Another alternative to MemStream would be StringStream, but it wouldn’t let me to reserve initial buffer (what seems to be a good idea), and I’d like to avoid multiple reallocations.

Thanks a lot for your help!


Regards,
Novo
Re: GZDecompress bug [message #22692 is a reply to message #22691] Wed, 05 August 2009 17:26 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Novo wrote on Wed, 05 August 2009 11:17


Thanks! That helped. I was confused by the name “size”. IMO there was no need to provide size of the input stream because it can be easily retrieved from the input stream.


The idea is that in some cases, you might want to store more .gz parts in single stream... In that case you need to know when to stop decompressing Wink (Of course, this is perhaps only useful for ZDecompress (without header), but was kept for header variant).

Mirek
Re: GZDecompress bug [message #22693 is a reply to message #22692] Wed, 05 August 2009 19:16 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
luzr wrote on Wed, 05 August 2009 11:26

Novo wrote on Wed, 05 August 2009 11:17


Thanks! That helped. I was confused by the name “size”. IMO there was no need to provide size of the input stream because it can be easily retrieved from the input stream.


The idea is that in some cases, you might want to store more .gz parts in single stream... In that case you need to know when to stop decompressing Wink (Of course, this is perhaps only useful for ZDecompress (without header), but was kept for header variant).

Mirek



Now I understand that. Wink But something like more explanatory argument’s name or even a single line of documentation would make my way to understanding much shorter.

"size" can mean anything.


Regards,
Novo
Re: GZDecompress bug [message #22694 is a reply to message #22693] Wed, 05 August 2009 20:21 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Sorry about that. I hope to add docs soon.

Just a little note (for future ?), if I would like to give size to
out, I would use this signature:

int GZDecompress(Stream& out, int size, Stream& in, Gate2<int, int> progress)


I believe we are quite consistent with that...

Mirek
Re: GZDecompress bug [message #22761 is a reply to message #22694] Wed, 12 August 2009 04:34 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
luzr wrote on Wed, 05 August 2009 14:21

Sorry about that. I hope to add docs soon.

Just a little note (for future ?), if I would like to give size to
out, I would use this signature:

int GZDecompress(Stream& out, int size, Stream& in, Gate2<int, int> progress)


I believe we are quite consistent with that...

Mirek



IMO the "out_size" name would be more informative.
At least you would know that it is related to "out".

I figured out how to reserve memory with StringStream.
StringStream strm;
String s;

s.Reserve(512);
strm.Open(s);


It looks like that should work, thought I'd prefer syntax below.
StringStream strm(String().Reserve(512));



Regards,
Novo
Re: GZDecompress bug [message #22764 is a reply to message #22761] Wed, 12 August 2009 09:25 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Well, it is perhaps a good idea to add

void StringStream::Reserve(int n)

which I did Smile

Mirek
Re: GZDecompress bug [message #22778 is a reply to message #22764] Fri, 14 August 2009 06:07 Go to previous message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
luzr wrote on Wed, 12 August 2009 03:25

Well, it is perhaps a good idea to add

void StringStream::Reserve(int n)

which I did Smile

Mirek


Thanks a lot! That will let me get rid of MemStream in my code.


Regards,
Novo
Previous Topic: Environment variables code page
Next Topic: ConvertInt > templatable Convert<T>
Goto Forum:
  


Current Time: Mon Apr 29 10:38:03 CEST 2024

Total time taken to generate the page: 0.02893 seconds