U++ forum: Welcome to the forum

Search on this site

Search in forums

Home » U++ Library support » U++ MT-multithreading and servers » WebSSL memory leak problem.

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

WebSSL memory leak problem. [message #26619]

Sun, 16 May 2010 17:42

Oblivion
Messages: 1202
Registered: August 2007

Senior Contributor

Hi,

I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.

I've also tried to use valgrind to inspect this problem, but due to some errors, all my attempts were unsuccessful:

Valgrind..
==4657== --xml=yes has been specified, but there is no XML output
==4657== destination.  You must specify an XML output destination
==4657== using --xml-fd=, --xml-file= or --xml=socket=.
valgrind: Bad option '--xml=yes, but no XML destination specified'; aborting.
valgrind: Use --help for more information.
Parsing valgrind output..
Error parsing valgrind output

Any ideas?

Regards.

p.s. : I am using Kubuntu Lucid Lynx.

Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat

Report message to a moderator

Re: WebSSL memory leak problem. [message #26620 is a reply to message #26619]

Sun, 16 May 2010 18:59

dolik.rce
Messages: 1791
Registered: August 2008
Location: Czech Republic

Ultimate Contributor

Oblivion wrote on Sun, 16 May 2010 17:42

I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.

Hi Oblivion,

Glad to hear the PPA works Smile

I can't help you with the memory leak, but I know how to fix the valgrind. The command line syntax was apparently changed lately. Quick fix (in ide/Valgrind.cpp):

void Ide::Valgrind()
{
	if(!IsValgrind())
		return;
	static String ValgrindLogFile;
	if(IsNull(ValgrindLogFile)) {
		StringStream ss;
		CreateHostRunDir()->Execute("valgrind --help", ss);
		String txt = ss;
		if(txt.Find("--log-file-exactly") > 0)
		   ValgrindLogFile = "--log-file-exactly=";
		else
		   ValgrindLogFile = "--log-file=";
		if(txt.Find("--xml-file") > 0)		// Added
		   ValgrindLogFile = "--xml-file=";	// Added
	}
...

But this requires recompiling theide... If you don't want to do that, I would recommend you to use nightly PPA. I believe someone will commit this or better fix in one or two days.

Best regards,
Honza

Report message to a moderator

Re: WebSSL memory leak problem. [message #26622 is a reply to message #26620]

Sun, 16 May 2010 23:38

mirek
Messages: 14255
Registered: November 2005

Ultimate Member

dolik.rce wrote on Sun, 16 May 2010 12:59

Oblivion wrote on Sun, 16 May 2010 17:42

I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.

Hi Oblivion,

Glad to hear the PPA works Smile

I can't help you with the memory leak, but I know how to fix the valgrind. The command line syntax was apparently changed lately. Quick fix (in ide/Valgrind.cpp):

void Ide::Valgrind()
{
	if(!IsValgrind())
		return;
	static String ValgrindLogFile;
	if(IsNull(ValgrindLogFile)) {
		StringStream ss;
		CreateHostRunDir()->Execute("valgrind --help", ss);
		String txt = ss;
		if(txt.Find("--log-file-exactly") > 0)
		   ValgrindLogFile = "--log-file-exactly=";
		else
		   ValgrindLogFile = "--log-file=";
		if(txt.Find("--xml-file") > 0)		// Added
		   ValgrindLogFile = "--xml-file=";	// Added
	}
...

Patch applied...

Mirek

Report message to a moderator

Re: WebSSL memory leak problem. [message #26629 is a reply to message #26619]

Mon, 17 May 2010 14:11

phirox
Messages: 49
Registered: December 2007

Member

I've posted about the WebSSL memory leak before. In short, it is this line:

CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

If you comment it out there shouldn't be a memory leak warning anymore, even though it is actually still there.

For a permanent fix, all the correct openssl unload functions should be called. I will check it out myself later, but for those in a hurry; see http://www.openssl.org/support/faq.html#PROG13 for more information.

Edit: After reviewing the code, Ultimate++ actually unloads exactly according to documentation. The memory leak is 'known' and unfixed by the OpenSSL library itself, the above workaround is still the best patch IMHO.

[Updated on: Tue, 18 May 2010 14:42]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26635 is a reply to message #26620]

Mon, 17 May 2010 17:10

Oblivion
Messages: 1202
Registered: August 2007

Senior Contributor

dolik.rce wrote on Sun, 16 May 2010 19:59

Oblivion wrote on Sun, 16 May 2010 17:42

I've successfully installed the latest stable release (2361-lucid-i386) of upp via ubuntu "stable" ppa yesterday. But I'm getting heap leaks with the WebSSL package all the time.

Hi Oblivion,

Glad to hear the PPA works Smile

I can't help you with the memory leak, but I know how to fix the valgrind. The command line syntax was apparently changed lately. Quick fix (in ide/Valgrind.cpp):

void Ide::Valgrind()
{
	if(!IsValgrind())
		return;
	static String ValgrindLogFile;
	if(IsNull(ValgrindLogFile)) {
		StringStream ss;
		CreateHostRunDir()->Execute("valgrind --help", ss);
		String txt = ss;
		if(txt.Find("--log-file-exactly") > 0)
		   ValgrindLogFile = "--log-file-exactly=";
		else
		   ValgrindLogFile = "--log-file=";
		if(txt.Find("--xml-file") > 0)		// Added
		   ValgrindLogFile = "--xml-file=";	// Added
	}
...

Hello Honza,

I've switched to the nightly PPA, probably it'll be better that way.

By the way, I forgot to thank you for your efforts on putting the upp in a PPA Smile

Nice Job!

Quote:

I've posted about the WebSSL memory leak before. In short, it is this line:

CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

If you comment it out there shouldn't be a memory leak warning anymore, even though it is actually still there.

For a permanent fix, all the correct openssl unload functions should be called. I will check it out myself later, but for those in a hurry; see http://www.openssl.org/support/faq.html#PROG13 for more information.

Thanks for the information phirox, I'll check it asap.

Regards

Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat

Report message to a moderator

Re: WebSSL memory leak problem. [message #26636 is a reply to message #26635]

Mon, 17 May 2010 17:36

dolik.rce
Messages: 1791
Registered: August 2008
Location: Czech Republic

Ultimate Contributor

Oblivion wrote on Mon, 17 May 2010 17:10

I've switched to the nightly PPA, probably it'll be better that way.

By the way, I forgot to thank you for your efforts on putting the upp in a PPA Smile

Nice Job!

Thank you, I'm glad to hear someone really uses it Smile

And as is said on the download page: Because there is no backporting of bugfixes, the most stable code is the latest SVN Smile

(Which is just a few revisions ahead of nightly builds)

Honza

Report message to a moderator

Re: WebSSL memory leak problem. [message #26637 is a reply to message #26629]

Mon, 17 May 2010 18:00

koldo
Messages: 3432
Registered: August 2008

Senior Veteran

phirox wrote on Mon, 17 May 2010 14:11

I've posted about the WebSSL memory leak before. In short, it is this line:

CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

Hello phirox

This is a really old U++ bug ! Confused

.

You discovered it in 2008 and it is not solved yet.

I found the same bug some months ago as I posted here: http://www.ultimatepp.org/forum/index.php?t=msg&goto=257 72&

It is an OpenSSL cleanup problem.

Other way to solve it would be to do this in Web/SSL/util.cpp:

INITBLOCK {
	Socket::Init();
	CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);
	SSL_load_error_strings();
	//SSL_library_init();		//SOLUTION TO MEMORY LEAK !!
}

Best regards
Iñaki

Report message to a moderator

Re: WebSSL memory leak problem. [message #26638 is a reply to message #26619]

Mon, 17 May 2010 19:00

dolik.rce
Messages: 1791
Registered: August 2008
Location: Czech Republic

Ultimate Contributor

Well, I'm not really no way expert in this, but I guess (based on the openssl FAQ link above) that adding

EXITBLOCK {
    CRYPTO_cleanup_all_ex_data();
}

should solve it as well. Or maybe using the safer functions ENGINE_cleanup() and CONF_modules_unload(), I didn't read the documentation nor did I test this.

BTW: Funny how many solution can exist to fix a single bug Smile

Honza

UPDATE: Apparently I was wrong... This EXITBLOCK already exists, sorry for useless post Embarassed

[Updated on: Mon, 17 May 2010 19:09]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26651 is a reply to message #26637]

Tue, 18 May 2010 07:41

Sc0rch
Messages: 99
Registered: February 2008
Location: Russia, Rubtsovsk

Member

koldo wrote on Mon, 17 May 2010 23:00

phirox wrote on Mon, 17 May 2010 14:11

I've posted about the WebSSL memory leak before. In short, it is this line:

CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);

Hello phirox

This is a really old U++ bug ! Confused

INITBLOCK {
	Socket::Init();
	CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);
	SSL_load_error_strings();
	//SSL_library_init();		//SOLUTION TO MEMORY LEAK !!
}

That's not work. Without "SSL_library_init();", SSL context cannot be created (SSL Context Error in result). I can't run the secured server without this line(

ENGINE_cleanup() - also not helps.

Another variants?

[Updated on: Tue, 18 May 2010 07:42]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26652 is a reply to message #26619]

Tue, 18 May 2010 07:46

Sc0rch
Messages: 99
Registered: February 2008
Location: Russia, Rubtsovsk

Member

I noticed that in CONSOLE MT I have no memory leaks with Web/SSL at all. There are memory leaks with GUI MT.
hmmm =)

[Updated on: Tue, 18 May 2010 07:50]

Report message to a moderator

Re: WebSSL memory leak problem. [message #26656 is a reply to message #26652]

Tue, 18 May 2010 10:48

koldo
Messages: 3432
Registered: August 2008

Senior Veteran

Sc0rch wrote on Tue, 18 May 2010 07:46

I noticed that in CONSOLE MT I have no memory leaks with Web/SSL at all. There are memory leaks with GUI MT.
hmmm =)

Hello Sc0rch

I dissent from your statement Smile

.

This program gets memory leaks...

#include <Core/Core.h>
using namespace Upp;
CONSOLE_APP_MAIN
{ 
}

The only packages it has are Core and Web/SSL.

This OpenSSL memory leak has been reported in forums for years (at least from 2003) and nobody seems to give an answer. There are only a few bytes lost and it is reported this leak does not increase with different calls to init and free functions.

The only fix for now is to comment SSL_library_init() line if it is not really required.

Confused

Best regards
Iñaki

Report message to a moderator

Re: WebSSL memory leak problem. [message #26696 is a reply to message #26619]

Wed, 19 May 2010 18:07

Sc0rch
Messages: 99
Registered: February 2008
Location: Russia, Rubtsovsk

Member

Sorry, it was my fault. I forgot to add waiting loop after thread initialization Sad

I agree with phirox: the best option to deal with SSL leaks is to comment "CRYPTO_set_mem_functions(SSLAlloc, SSLRealloc, SSLFree);". No more notifications. My server with SSL connections works stable (hard tested).

Report message to a moderator

Re: WebSSL memory leak problem. [message #26756 is a reply to message #26619]

Wed, 26 May 2010 12:00

rylek
Messages: 79
Registered: November 2005

Member

Hello there!

I'm afraid that commenting out CRYPTO_set_mem_functions doesn't remove the memory leaks, it just hides them because OpenSSL then uses standard C allocator instead of the one in U++. In a similar vein, getting rid of SSL_library_init() is in my opinion a bad idea either. Some time ago when debugging a https client using the Web/SSL package I also started to receive some memory leaks, but after recompiling an updated version of OpenSSL they disappeared.

In order to be able to debug this kind of leaks, a long time ago I added a simple memory allocation logging feature to Web/SSL (just set LOG_UPP_SSL_MALLOC to 1). The last time I was tracing the leaks, they seemed related to certain global tables allocated during the SSL library initialization (which might explain why removing the initialization call seemingly removed the problem). However, as I said, after recompiling the OpenSSL they disappeared altogether so I honestly didn't continue to pursue the problem.

Please also note that serious havoc would result from linking your application to OpenSSL compiled with a different build style (MT vs ST, shared vs static libraries). When recompiling OpenSSL it is good to note which /M flag the make sends to the compiler (at least this holds when compiling under 32-bit MSVC 7.1, which is what I did). In any case if you fail to get rid of the leaks, I can at least try to post somewhere the compiled OpenSSL libraries (libeay / ssleay) which do work for me so that you can try them out. I hope the Web/SSL package is not discontinued, it's just that getting rid of memory leaks in code you didn't write is sometimes quite complicated. Which of course doesn't mean we should ignore them.

Regards

Tomas

Report message to a moderator

Re: WebSSL memory leak problem. [message #26757 is a reply to message #26756]

Wed, 26 May 2010 12:19

koldo
Messages: 3432
Registered: August 2008

Senior Veteran

Hello Tomas

Quote:

I can at least try to post somewhere the compiled OpenSSL libraries (libeay / ssleay) which do work for me so that you can try them out.

It could be great if you could put these libraries somewhere or you could tell us where we can find them or how we can build them the right way.

I have tried to do it but I get memory leaks compiling with different modes.

If they work with MSC and MinGW it would be great too.

Best regards
Iñaki

Report message to a moderator

Previous Topic:	CoWork tidy up
Next Topic:	SSL patch

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Apr 26 14:23:39 CEST 2025

Total time taken to generate the page: 0.01018 seconds