Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ MT-multithreading and servers » Core multithread dangers
Core multithread dangers [message #835] Fri, 03 February 2006 23:09 Go to next message
hojtsy is currently offline  hojtsy
Messages: 241
Registered: January 2006
Location: Budapest, Hungary
Experienced Member
I was told before that Core & Web packages are thread-safe. But some methods of Core are not thread-safe, and something like
CriticalSection::LockMain should be added to them to fix this. Here is a list of them.
static TimingInspector& s_zero()
TimingInspector::TimingInspector(const char *_name)
static void sInit()
String& sHomeDir()
String  ConfigFile(const char *file)
byte *Alloc4KBRaw()
void *MemoryAllocPermanent(size_t size)
void *MemoryAlloc(size_t size)
void MemoryProbe(const char *name, dword flags)
const LanguageInfo& GetLanguageInfo(int lcode)
static Array<LngModule>& sMod()
CriticalSection& MainCriticalSection()
PageInfo *NewPage(int magnitude)
static CriticalSection& sHeapLock()
static CriticalSection& sHeapLock2()
static inline void sInitTables() 
void MemoryInitDiagnostics()
RHITCOUNT(n)
static int sMappingGranularity_()
VectorMap<String, VectorMap<String, VectorMap<String, Topic > > >& TopicBase() 
String GetIniKey(const char *name) 

These other functions of Core need bigger rewrite to be threadsafe:
sDumpPtr(void *ptr)
sDump(dword w)
const char *Dump(PageInfo *page)
const char *MemoryCounters()

These functions of the Web package are not thread-safe:
void HttpServer::ReadPostData(Socket& socket, HttpQuery& query)
void SSLInit()
static const dword *GetCRCTable()
RefPtr<HttpQuery::Data> HttpQuery::Empty()
dword WINAPI HttpExtensionProc(EXTENSION_CONTROL_BLOCK *ecb)

The problem is always with the local static variables which are not protected from parallel initialization on two threads.
Re: Core multithread dangers [message #836 is a reply to message #835] Fri, 03 February 2006 23:51 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
You are right in some cases, but look better. Many of those are in fact serialized elsewhere (that is esp. true for heap - that one IS thread safe to my knowledge).

OTOH, you have missed some that might cause a trouble in future (e.g. when working on GuiMT, I have fount that Ptr/Pte are not really thread safe - something to fix ASAP).

Also, note that StaticCriticalSection is zero-initilized POD, so it does not perform any further form of initilization (nice simple solution to "how to serialize initialization of serializer").

Anyway, thanks. I must admit that multithreading is generally less tested / fixed than the rest of library. Any warnings/suggestions here are welcome.

Mirek
Re: Core multithread dangers [message #841 is a reply to message #836] Sat, 04 February 2006 12:14 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Well, thining about issue and going through the Core...

I have found many places where MT is broken - most of them will not affect normal apps, but anyway...

I already have MT on ToDo list, but it turns out that careful audit of sources will be needed. I hope to get it done while working on new Draw.

I am also considering another MT option for GUI - perhaps it is not that bad idea to provide single GUI critical section that would got locked while processing input events - other threads of MT GUI app would simply lock this section before calling methods of any widget.

Mirek
Re: Core multithread dangers [message #843 is a reply to message #836] Sat, 04 February 2006 14:53 Go to previous messageGo to next message
hojtsy is currently offline  hojtsy
Messages: 241
Registered: January 2006
Location: Budapest, Hungary
Experienced Member
luzr wrote on Fri, 03 February 2006 17:51

Also, note that StaticCriticalSection is zero-initilized POD, so it does not perform any further form of initilization (nice simple solution to "how to serialize initialization of serializer").
Yes, that is a very good idea. But I needed almost an hour Shocked to figure out how that works before you posted it - it was quite tricky. Even a short code comment would have saved me lots of time.

I think this new macro is not threadsafe:
#define ONCELOCK \
for(static volatile bool b = true; b;) \
	for(static CriticalSection section; b;) \
		for(CriticalSection::Lock lock(section); b; b = false)
The problem is that the b flag is not protected from repeated default-initialization to true.

Re: Core multithread dangers [message #845 is a reply to message #843] Sat, 04 February 2006 18:20 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Yes, my fault, thanks. Fixed.

Mirek
Re: Core multithread dangers [message #868 is a reply to message #845] Mon, 06 February 2006 11:46 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Hojtsy, I have one MT trouble I would like to share....

It is about Ptr implementation. While current version is MT safe to my knowledge, it is inferior to previous one as it allocates shared data for the lifetime of Pte target.

Original version was able to delete shared data whenever no more pointer were pointing to it, something like

release prec:
if(--prec->n == 0) {
// MT!
prec->ptr->prec = NULL;
delete prec;
}

However, I do not see a way how to implement it without adding a lock to Pte, which is even worse. The trouble is that at the MT! point, Pte can be destructed and prec is not valid anymore....

(Just for record, ~Pte() { if(prec) prec->ptr = NULL; })

Hm, thinking about it, maybe single _global_ lock would do?

Mirek
Re: Core multithread dangers [message #874 is a reply to message #868] Mon, 06 February 2006 14:18 Go to previous messageGo to next message
hojtsy is currently offline  hojtsy
Messages: 241
Registered: January 2006
Location: Budapest, Hungary
Experienced Member
Mirek, I did not fully understand your post. I will try to sync and see the modified implementation of Ptr, but I only have uvs set up on my home computer, so it will take some time. But my question is: how would a global lock be better then a per-object lock? Are you trying to optimize the memory consumption of the lock objects?
Re: Core multithread dangers [message #875 is a reply to message #874] Mon, 06 February 2006 14:24 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Yes, optimize memory. Per-object lock would add enourmous data to Pte (more that would be saved by avoiding dynamic storage).

Then, "new/delete" itself have global lock and it would be called per each Pte construction/destruction without Pte global lock, so global lock for Pte does not add too much overhead IMHO. And locked areas are extremly slow (a couple of assembly instructions), which makes me believe that concurrent access will not happen very often (it is much likely to happen during new/delete).

I have tried to implement global lock version - now there are both in Core, former version commented out. Sync uvs2.

Mirek
Re: Core multithread dangers [message #878 is a reply to message #875] Mon, 06 February 2006 14:37 Go to previous messageGo to next message
hojtsy is currently offline  hojtsy
Messages: 241
Registered: January 2006
Location: Budapest, Hungary
Experienced Member
OK, I will check it. In the meanwhile let me indicate a bug, which may have been corrected already. I am trying to compile an upp multithreaded application on Linux. TheIde binary is from the 511 release, the library sources are from the latest 20060129 snapshot. Single threaded appliactions are compiling and running OK. But when I compile anything with the MT flag (including single threaded applications, such as the Bombs example) the application just freezes while starting. With some debugging we discovered that it seems to be waiting forever for some mutex. Is it possible that recursive locking happens for the same lock?
Re: Core multithread dangers [message #879 is a reply to message #878] Mon, 06 February 2006 14:52 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Definitely.

Frankly, you are perhaps the first one trying to do MT in Linux Smile

My guess is that it is either the issue of Single or INIT_LOCK.

Mirek
Re: Core multithread dangers [message #880 is a reply to message #878] Mon, 06 February 2006 19:19 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Well, indeed. Some that I missed in my education: linux mutext is not reentrant.

I have spent nice productive day fixing this issue. At least GuiMT example now seems to work.

Mirek
Re: Core multithread dangers [message #885 is a reply to message #880] Tue, 07 February 2006 09:59 Go to previous messageGo to next message
hojtsy is currently offline  hojtsy
Messages: 241
Registered: January 2006
Location: Budapest, Hungary
Experienced Member
luzr wrote on Mon, 06 February 2006 13:19

I have spent nice productive day fixing this issue. At least GuiMT example now seems to work.
You mean it works on Linux? Because previously it didn't even work on windows for me - but that is fixed now.
Re: Core multithread dangers [message #886 is a reply to message #885] Tue, 07 February 2006 11:09 Go to previous messageGo to next message
fudadmin is currently offline  fudadmin
Messages: 1321
Registered: November 2005
Location: Kaunas, Lithuania
Ultimate Contributor
Administrator
Smile But I've got this error:
Linking...
LINK : fatal error LNK1104: cannot open file 'LIBCMTD.lib'


this article says something about missing LIB variables for Visual Studio .NET
http://www.celoxica.com/support/view_article.asp?ArticleID=5 27

As I remember, when installing Ultimate++ said: "don't need to setup environment variables for MS toolkit".
Any quick fix?
Re: Core multithread dangers [message #888 is a reply to message #886] Tue, 07 February 2006 11:41 Go to previous messageGo to next message
unodgs is currently offline  unodgs
Messages: 1366
Registered: November 2005
Location: Poland
Ultimate Contributor

I'm not sure but in VisualC++ toolkit mulithreaded libs are missing... (try to get full visual studio..)
Re: Core multithread dangers [message #889 is a reply to message #888] Tue, 07 February 2006 12:49 Go to previous messageGo to next message
fudadmin is currently offline  fudadmin
Messages: 1321
Registered: November 2005
Location: Kaunas, Lithuania
Ultimate Contributor
Administrator
unodgs wrote on Tue, 07 February 2006 05:41

I'm not sure but in VisualC++ toolkit mulithreaded libs are missing... (try to get full visual studio..)


But I've got that library... Shocked
Re: Core multithread dangers [message #896 is a reply to message #889] Tue, 07 February 2006 14:42 Go to previous message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
The simple way is to download Visual Studio Express 2005. It is free (till the end of year I guess). And you even do not need to install the studio, just commandline compiler Smile

Mirek
Next Topic: SlaveProcess should be moved from Web to Core
Goto Forum:
  


Current Time: Fri Mar 29 06:42:25 CET 2024

Total time taken to generate the page: 0.01591 seconds