Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ MT-multithreading and servers » WebSSL memory leak problem.
WebSSL memory leak problem. [message #26619] Sun, 16 May 2010 17:42 Go to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hi,

I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.

I've also tried to use valgrind to inspect this problem, but due to some errors, all my attempts were unsuccessful:

Valgrind..
==4657== --xml=yes has been specified, but there is no XML output
==4657== destination.  You must specify an XML output destination
==4657== using --xml-fd=, --xml-file= or --xml=socket=.
valgrind: Bad option '--xml=yes, but no XML destination specified'; aborting.
valgrind: Use --help for more information.
Parsing valgrind output..
Error parsing valgrind output


Any ideas?

Regards.


p.s. : I am using Kubuntu Lucid Lynx.


Re: WebSSL memory leak problem. [message #26620 is a reply to message #26619] Sun, 16 May 2010 18:59 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Oblivion wrote on Sun, 16 May 2010 17:42


I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.


Hi Oblivion,

Glad to hear the PPA works Smile

I can't help you with the memory leak, but I know how to fix the valgrind. The command line syntax was apparently changed lately. Quick fix (in ide/Valgrind.cpp):
void Ide::Valgrind()
{
	if(!IsValgrind())
		return;
	static String ValgrindLogFile;
	if(IsNull(ValgrindLogFile)) {
		StringStream ss;
		CreateHostRunDir()->Execute("valgrind --help", ss);
		String txt = ss;
		if(txt.Find("--log-file-exactly") > 0)
		   ValgrindLogFile = "--log-file-exactly=";
		else
		   ValgrindLogFile = "--log-file=";
		if(txt.Find("--xml-file") > 0)		// Added
		   ValgrindLogFile = "--xml-file=";	// Added
	}
...
But this requires recompiling theide... If you don't want to do that, I would recommend you to use nightly PPA. I believe someone will commit this or better fix in one or two days.

Best regards,
Honza

Re: WebSSL memory leak problem. [message #26622 is a reply to message #26620] Sun, 16 May 2010 23:38 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
dolik.rce wrote on Sun, 16 May 2010 12:59

Oblivion wrote on Sun, 16 May 2010 17:42


I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.


Hi Oblivion,

Glad to hear the PPA works Smile

I can't help you with the memory leak, but I know how to fix the valgrind. The command line syntax was apparently changed lately. Quick fix (in ide/Valgrind.cpp):
void Ide::Valgrind()
{
	if(!IsValgrind())
		return;
	static String ValgrindLogFile;
	if(IsNull(ValgrindLogFile)) {
		StringStream ss;
		CreateHostRunDir()->Execute("valgrind --help", ss);
		String txt = ss;
		if(txt.Find("--log-file-exactly") > 0)
		   ValgrindLogFile = "--log-file-exactly=";
		else
		   ValgrindLogFile = "--log-file=";
		if(txt.Find("--xml-file") > 0)		// Added
		   ValgrindLogFile = "--xml-file=";	// Added
	}
...




Patch applied...

Mirek
Re: WebSSL memory leak problem. [message #26629 is a reply to message #26619] Mon, 17 May 2010 14:11 Go to previous messageGo to next message
phirox is currently offline  phirox
Messages: 49
Registered: December 2007
Member
I've posted about the WebSSL memory leak before. In short, it is this line:
CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

If you comment it out there shouldn't be a memory leak warning anymore, even though it is actually still there.

For a permanent fix, all the correct openssl unload functions should be called. I will check it out myself later, but for those in a hurry; see http://www.openssl.org/support/faq.html#PROG13 for more information.

Edit: After reviewing the code, Ultimate++ actually unloads exactly according to documentation. The memory leak is 'known' and unfixed by the OpenSSL library itself, the above workaround is still the best patch IMHO.

[Updated on: Tue, 18 May 2010 14:42]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26635 is a reply to message #26620] Mon, 17 May 2010 17:10 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
dolik.rce wrote on Sun, 16 May 2010 19:59

Oblivion wrote on Sun, 16 May 2010 17:42


I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.


Hi Oblivion,

Glad to hear the PPA works Smile

I can't help you with the memory leak, but I know how to fix the valgrind. The command line syntax was apparently changed lately. Quick fix (in ide/Valgrind.cpp):
void Ide::Valgrind()
{
	if(!IsValgrind())
		return;
	static String ValgrindLogFile;
	if(IsNull(ValgrindLogFile)) {
		StringStream ss;
		CreateHostRunDir()->Execute("valgrind --help", ss);
		String txt = ss;
		if(txt.Find("--log-file-exactly") > 0)
		   ValgrindLogFile = "--log-file-exactly=";
		else
		   ValgrindLogFile = "--log-file=";
		if(txt.Find("--xml-file") > 0)		// Added
		   ValgrindLogFile = "--xml-file=";	// Added
	}
...
But this requires recompiling theide... If you don't want to do that, I would recommend you to use nightly PPA. I believe someone will commit this or better fix in one or two days.

Best regards,
Honza




Hello Honza,

I've switched to the nightly PPA, probably it'll be better that way.

By the way, I forgot to thank you for your efforts on putting the upp in a PPA Smile

Nice Job!


Quote:


I've posted about the WebSSL memory leak before. In short, it is this line:

CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

If you comment it out there shouldn't be a memory leak warning anymore, even though it is actually still there.

For a permanent fix, all the correct openssl unload functions should be called. I will check it out myself later, but for those in a hurry; see http://www.openssl.org/support/faq.html#PROG13 for more information.



Thanks for the information phirox, I'll check it asap.

Regards


Re: WebSSL memory leak problem. [message #26636 is a reply to message #26635] Mon, 17 May 2010 17:36 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Oblivion wrote on Mon, 17 May 2010 17:10

I've switched to the nightly PPA, probably it'll be better that way.

By the way, I forgot to thank you for your efforts on putting the upp in a PPA Smile

Nice Job!

Thank you, I'm glad to hear someone really uses it Smile

And as is said on the download page: Because there is no backporting of bugfixes, the most stable code is the latest SVN Smile (Which is just a few revisions ahead of nightly builds)

Honza
Re: WebSSL memory leak problem. [message #26637 is a reply to message #26629] Mon, 17 May 2010 18:00 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
phirox wrote on Mon, 17 May 2010 14:11

I've posted about the WebSSL memory leak before. In short, it is this line:
CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

If you comment it out there shouldn't be a memory leak warning anymore, even though it is actually still there.

For a permanent fix, all the correct openssl unload functions should be called. I will check it out myself later, but for those in a hurry; see http://www.openssl.org/support/faq.html#PROG13 for more information.

Hello phirox

This is a really old U++ bug ! Confused .

You discovered it in 2008 and it is not solved yet.

I found the same bug some months ago as I posted here: http://www.ultimatepp.org/forum/index.php?t=msg&goto=257 72&

It is an OpenSSL cleanup problem.

Other way to solve it would be to do this in Web/SSL/util.cpp:
INITBLOCK {
	Socket::Init();
	CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);
	SSL_load_error_strings();
	//SSL_library_init();		//SOLUTION TO MEMORY LEAK !!
}




Best regards
Iñaki
Re: WebSSL memory leak problem. [message #26638 is a reply to message #26619] Mon, 17 May 2010 19:00 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Well, I'm not really no way expert in this, but I guess (based on the openssl FAQ link above) that adding
EXITBLOCK {
    CRYPTO_cleanup_all_ex_data();
}
should solve it as well. Or maybe using the safer functions ENGINE_cleanup() and CONF_modules_unload(), I didn't read the documentation nor did I test this.

BTW: Funny how many solution can exist to fix a single bug Smile

Honza

UPDATE: Apparently I was wrong... This EXITBLOCK already exists, sorry for useless post Embarassed

[Updated on: Mon, 17 May 2010 19:09]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26651 is a reply to message #26637] Tue, 18 May 2010 07:41 Go to previous messageGo to next message
Sc0rch is currently offline  Sc0rch
Messages: 99
Registered: February 2008
Location: Russia, Rubtsovsk
Member

koldo wrote on Mon, 17 May 2010 23:00

phirox wrote on Mon, 17 May 2010 14:11

I've posted about the WebSSL memory leak before. In short, it is this line:
CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

If you comment it out there shouldn't be a memory leak warning anymore, even though it is actually still there.

For a permanent fix, all the correct openssl unload functions should be called. I will check it out myself later, but for those in a hurry; see http://www.openssl.org/support/faq.html#PROG13 for more information.

Hello phirox

This is a really old U++ bug ! Confused .

You discovered it in 2008 and it is not solved yet.

I found the same bug some months ago as I posted here: http://www.ultimatepp.org/forum/index.php?t=msg&goto=257 72&

It is an OpenSSL cleanup problem.

Other way to solve it would be to do this in Web/SSL/util.cpp:
INITBLOCK {
	Socket::Init();
	CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);
	SSL_load_error_strings();
	//SSL_library_init();		//SOLUTION TO MEMORY LEAK !!
}





That's not work. Without "SSL_library_init();", SSL context cannot be created (SSL Context Error in result). I can't run the secured server without this line(

ENGINE_cleanup() - also not helps.

Another variants?

[Updated on: Tue, 18 May 2010 07:42]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26652 is a reply to message #26619] Tue, 18 May 2010 07:46 Go to previous messageGo to next message
Sc0rch is currently offline  Sc0rch
Messages: 99
Registered: February 2008
Location: Russia, Rubtsovsk
Member

I noticed that in CONSOLE MT I have no memory leaks with Web/SSL at all. There are memory leaks with GUI MT.
hmmm =)

[Updated on: Tue, 18 May 2010 07:50]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26656 is a reply to message #26652] Tue, 18 May 2010 10:48 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Sc0rch wrote on Tue, 18 May 2010 07:46

I noticed that in CONSOLE MT I have no memory leaks with Web/SSL at all. There are memory leaks with GUI MT.
hmmm =)


Hello Sc0rch

I dissent from your statement Smile.

This program gets memory leaks...

#include <Core/Core.h>
using namespace Upp;
CONSOLE_APP_MAIN
{ 
}


The only packages it has are Core and Web/SSL.

This OpenSSL memory leak has been reported in forums for years (at least from 2003) and nobody seems to give an answer. There are only a few bytes lost and it is reported this leak does not increase with different calls to init and free functions.

The only fix for now is to comment SSL_library_init() line if it is not really required.

Confused



Best regards
Iñaki
Re: WebSSL memory leak problem. [message #26696 is a reply to message #26619] Wed, 19 May 2010 18:07 Go to previous messageGo to next message
Sc0rch is currently offline  Sc0rch
Messages: 99
Registered: February 2008
Location: Russia, Rubtsovsk
Member

Sorry, it was my fault. I forgot to add waiting loop after thread initialization Sad
I agree with phirox: the best option to deal with SSL leaks is to comment "CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);". No more notifications. My server with SSL connections works stable (hard tested).
Re: WebSSL memory leak problem. [message #26756 is a reply to message #26619] Wed, 26 May 2010 12:00 Go to previous messageGo to next message
rylek is currently offline  rylek
Messages: 79
Registered: November 2005
Member
Hello there!

I'm afraid that commenting out CRYPTO_set_mem_functions doesn't remove the memory leaks, it just hides them because OpenSSL then uses standard C allocator instead of the one in U++. In a similar vein, getting rid of SSL_library_init() is in my opinion a bad idea either. Some time ago when debugging a https client using the Web/SSL package I also started to receive some memory leaks, but after recompiling an updated version of OpenSSL they disappeared.

In order to be able to debug this kind of leaks, a long time ago I added a simple memory allocation logging feature to Web/SSL (just set LOG_UPP_SSL_MALLOC to 1). The last time I was tracing the leaks, they seemed related to certain global tables allocated during the SSL library initialization (which might explain why removing the initialization call seemingly removed the problem). However, as I said, after recompiling the OpenSSL they disappeared altogether so I honestly didn't continue to pursue the problem.

Please also note that serious havoc would result from linking your application to OpenSSL compiled with a different build style (MT vs ST, shared vs static libraries). When recompiling OpenSSL it is good to note which /M flag the make sends to the compiler (at least this holds when compiling under 32-bit MSVC 7.1, which is what I did). In any case if you fail to get rid of the leaks, I can at least try to post somewhere the compiled OpenSSL libraries (libeay / ssleay) which do work for me so that you can try them out. I hope the Web/SSL package is not discontinued, it's just that getting rid of memory leaks in code you didn't write is sometimes quite complicated. Which of course doesn't mean we should ignore them.

Regards

Tomas
Re: WebSSL memory leak problem. [message #26757 is a reply to message #26756] Wed, 26 May 2010 12:19 Go to previous message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Hello Tomas

Quote:

I can at least try to post somewhere the compiled OpenSSL libraries (libeay / ssleay) which do work for me so that you can try them out.


It could be great if you could put these libraries somewhere or you could tell us where we can find them or how we can build them the right way.

I have tried to do it but I get memory leaks compiling with different modes.

If they work with MSC and MinGW it would be great too.


Best regards
Iñaki
Previous Topic: CoWork tidy up
Next Topic: SSL patch
Goto Forum:
  


Current Time: Thu Mar 28 23:55:01 CET 2024

Total time taken to generate the page: 0.03165 seconds