Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ MT-multithreading and servers » Use same variable in different threads
Use same variable in different threads [message #30116] Thu, 09 December 2010 18:26 Go to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Hello all

What is the best way to read and write a variable in different threads?

I have seen atomic, volatile, RWMutex. However I do not know the way to declare, read and write a shared variable.


Best regards
Iñaki
Re: Use same variable in different threads [message #30117 is a reply to message #30116] Thu, 09 December 2010 18:57 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Hi Koldo,

From my little knowledge (close to yours probably Smile ):

Atomic is just typedefed int, which can be manipulated by U++ functions Atomic{Read,Write,Inc,Dec,XAdd}(). They are implemented using atomic operations, so it should be guaranteed that the return value is correct even if other thread changes value of the variable while processing the function call.

Volatile tells compiler that the variable can be changed from other threads, so that compiler knows that the generated code must check for its value everytime, instead using for example value stored in cpu cache, because other thread might have changed since it was stored there.

Mutex is a mechanism that allow you to lock some code so that if other code gets to the same section it has to wait for the first one to leave. This is utilized by INTERLOCKED macro in U++.

RWMutex is a variant of mutex that allows you to mark "read code" which can be executed by any number of threads at once and "write code" that can be executed only by one thread at a time and no thread can run in write mode until all reading threads are finished.

Hope that at least some of it helps... It definitelly helped me to sort it out in my head Smile

Best regards,
Honza
Re: Use same variable in different threads [message #30121 is a reply to message #30117] Thu, 09 December 2010 22:04 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Hello Honza

Thank you for your explanation.

The question now is, how to program it?


Best regards
Iñaki
Re: Use same variable in different threads [message #30122 is a reply to message #30121] Fri, 10 December 2010 00:26 Go to previous messageGo to next message
Didier is currently offline  Didier
Messages: 680
Registered: November 2008
Location: France
Contributor
Hi Koldo,

I would add one thing to Honza's explanation: 'Memory bariers'

Actually this is not a thing that is used directly by people but it is necessary to understand all that is going on.

Volatile allows you to access a shared variable not protected by mutex and you will be sure it always has the wright value (the cache on the processor might be different for each thread, so in extreme cases you can get de synchronised values of one variable from time to time ..... ==> bugs impossible to find).
Of course if the variable is larger than the processor bus ... the R/W process will not be atomic so you need a mutex !

Memory barriers are points where the cpu cache is flushed so all the synchronisation problems just disappear
In mutexes, such barriers are used to avoid such problems, and therefore when using a mutex lock/unlock procedure, you protect the resource (and it's memory) but you also flush the cash at each mutex lock/unlock.
So .... when using a mutex there is no need to use volatile it might even slow down you're app.

Finally, to answer you're question:
* if the variable (and it's context !) is just an int or smaller it can be accessed atomically ==> use this method, it's the fastest one (volatile will work in most cases but to be sure use the dedicated functions)
* otherwise you have to use the good old mutex, no may out !
- mutex.lock()
- do you're thing ..R..W....etc
- mutex.unlock()


NB: the volatile/synchronisation problems mostly has effects on programs compiled in O3.

Hope this helps you !

[Updated on: Fri, 10 December 2010 00:29]

Report message to a moderator

Re: Use same variable in different threads [message #30123 is a reply to message #30122] Fri, 10 December 2010 06:49 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Quote:


The question now is, how to program it?

That is always the question Smile Unfortunatelly the answer is not simple and it depends a lot on what you are trying to achieve...

As Didier pointed out, for simple int compatible variables Atomic* functions are best. For something rarely used (e.g. flag to stop all threads) I ussually use volatile. And fore manipulating larger objects, you should definitely use mutex. The basic approach for mutex might be to lock it everytime before you access the variable that is shared between threads and unlock after you are done with it. I think it is best to have one mutex for each object that is accessed from various threads.

In my experience, the best solution is to avoid inter-thread communication whenever possible Smile Of course, than is definitely not an universal solution...

Honza
Re: Use same variable in different threads [message #30127 is a reply to message #30123] Fri, 10 December 2010 10:01 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Thank you all for your answers Smile

- So, if the var is just an int or smaller, I would do:

// Declaration
Atomic myRunProcess = true;

...

// Thread process. Permitted
while(AtomicRead(myRunProcess)) {
... do all
}


// Thread process. Not permitted ????
while(myRunProcess) {
... do all
}

- If the var is bigger, I would use mutex but, how?
For example, imagine there is a struct to share:

struct mySharedData {
    int a;
    String b;
    double c;
};


What do I have to do to share this struct between different threads and main thread ?


Best regards
Iñaki
Re: Use same variable in different threads [message #30130 is a reply to message #30127] Fri, 10 December 2010 11:34 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Here is a little sample code for you, demonstrating the Mutex and Atomic. It is stupidly written (on purpose, of course Wink ) to show how bad it can go if you don't use the locking. See for yourself by uncommenting the "#define NOMUTEX".

In this example it is easy to see what is wrong with the code and it might be possible to fix it so it works better even without locking, but in many cases the errors are more subtle and harder to find. Also remember that if you access the shared variable from multiple places in you code, you should use the mutex around each of them.

Here is the code:
#include <Core/Core.h>
using namespace UPP;

struct mySharedData {
	int a,b;
	String c;
};

Mutex m;
mySharedData data;
Atomic threadnum;

//uncomment this to see how it ends up when not using mutex
//#define NOMUTEX

void ThreadFunction(){
	int thisthread=AtomicInc(threadnum); //thisthread serves as an unique identifier of thread, just for the sake of the examples clarity
	for(int i = 0; i < 10; i++){
		Thread::Sleep(Random(10)); // Pretend some work...
#ifndef NOMUTEX
		m.Enter();	    // Enter the section that accesses the shared data
#endif
		data.a++;
		data.c<<data.a<<": ";
		Thread::Sleep(20); //Let's pretend that some slow operation happens here (for example file access)
		data.b=data.a;
		data.c<<data.b<<" ["<<thisthread<<"]\n";
#ifndef NOMUTEX
		m.Leave();    // we don't need the exclusive access to data anymore, release the mutex
#endif
	}
}

CONSOLE_APP_MAIN {
	//initialize variables (no threads yet, so it is save to make it without mutex);
	threadnum=0;
	data.a=0;
	data.b=0;
	data.c="";
	
	// start the threads
	Thread::Start(callback(ThreadFunction));
	Thread::Start(callback(ThreadFunction));
	Thread::Start(callback(ThreadFunction));
	
	while(Thread::GetCount()); // the main thread should wait for other to finish, so we can see the results 
		Thread::Sleep(100);
	
	Cout()<<data.c<<"\n";
}


Honza
Re: Use same variable in different threads [message #30132 is a reply to message #30130] Fri, 10 December 2010 13:19 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Hello Honza

So:

- Atomic vars has to be handled using AtomicXXX() functions

- Variables handled with Mutex are declared normally, but all use of them (read or write) has to be between an Enter() and a Leave()


Best regards
Iñaki
Re: Use same variable in different threads [message #30133 is a reply to message #30132] Fri, 10 December 2010 14:08 Go to previous messageGo to next message
gprentice is currently offline  gprentice
Messages: 260
Registered: November 2005
Location: New Zealand
Experienced Member
No expert here either but when you're sharing data between threads, I think you need both volatile and synchronization.

A mutex ensures that the compiler/linker can't optimise code across the mutex entry, that the cache is flushed and that only one thread can be executing the code the mutex protects (i.e. it's synchronized).

http://msdn.microsoft.com/en-us/library/ms686355(v=VS.85).aspx

volatile ensures that the compiler/linker can't optimise your code and use a cached value. With Microsoft, optimisation can occur at link time. On some platforms you can get away without volatile if you call a global function that the compiler can't see - the compiler has to assume that global function might modify the variable you're sharing so is forced to re-read the variable from memory, but that is't safe with Microsoft.

There's also thread local storage - see thread__

Regarding atomic - on Win32, 32 bits are atomic and on Win64, 64 bits are atomic. On Win32, the atomixXXX functions use the Interlocked... functions that allow you to read/write without being interrupted by another thread etc, and also provide a memory barrier.

Hence I think you need
volatile mySharedData data;
volatile Atomic threadnum;
Anyway, I don't think that using volatile would be wrong, even if it's not always necessary.

Graeme

[Updated on: Fri, 10 December 2010 14:09]

Report message to a moderator

Re: Use same variable in different threads [message #30134 is a reply to message #30132] Fri, 10 December 2010 14:16 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

koldo wrote on Fri, 10 December 2010 13:19

Hello Honza

So:

- Atomic vars has to be handled using AtomicXXX() functions

- Variables handled with Mutex are declared normally, but all use of them (read or write) has to be between an Enter() and a Leave()

Yes. I would just add that Atomic can be handled as int if you know that no clash can happen (i.e. when only one thread can access it at given moment). Similar with mutex, you can skip the locking if you are sure that there is only one thread executing that code. But those are details and being more cautious than necessary does never hurt Wink

Graeme is completely right about the volatile I think. I completely forgot to mark the variables Embarassed

Honza
Re: Use same variable in different threads [message #30135 is a reply to message #30134] Fri, 10 December 2010 14:34 Go to previous messageGo to next message
gprentice is currently offline  gprentice
Messages: 260
Registered: November 2005
Location: New Zealand
Experienced Member
Actually, on second thoughts I think the code as written is fine without volatile because the mutex forces the shared data to be updated in memory. Possibly an atomic variable that is read or written outside of a mutex region is more likely to need volatile.

Edit : except that doesn't ensure the hardware doesn't use a cached value ... so now I just noticed AtomicRead and AtomicWrite in U++ - so I take it all back - you probably don't need volatile at all Smile

Graeme

[Updated on: Fri, 10 December 2010 14:40]

Report message to a moderator

Re: Use same variable in different threads [message #30137 is a reply to message #30135] Fri, 10 December 2010 18:50 Go to previous messageGo to next message
Didier is currently offline  Didier
Messages: 680
Registered: November 2008
Location: France
Contributor
Hi all,

Quote:


Actually, on second thoughts I think the code as written is fine without volatile because the mutex forces the shared data to be updated in memory. Possibly an atomic variable that is read or written outside of a mutex region is more likely to need volatile.



Yes this is the case, and is what I explained in my earlier post.

Maybe what you need Koldo, in the example you gave is something like this:
#include <CtrlLib/CtrlLib.h>

using namespace Upp;
class AtomicVar
{
	private:
		Atomic val;
	public:
		AtomicVar() {};
		
		AtomicVar( AtomicVar& p) { AtomicWrite(val, p); }
		template <class T>
		AtomicVar& operator=(const T& p) { AtomicWrite(val, p); }

		operator int() {return AtomicRead(val); }
	
};

GUI_APP_MAIN
{
	
	AtomicVar v;
	
	v=5;
	
	String str = "Value = ";
	
	str << (int)v;
	
	LOG(str);
	
	
	
}


All the accesses to 'v' are atomic, so in you're case all you have to do is replace
Atomic myRunProcess = true;

with
AtomicVar myRunProcess = true;


and just use myRunProcess normaly, without worrying about Atomic considerations !!


Re: Use same variable in different threads [message #30138 is a reply to message #30137] Sat, 11 December 2010 01:37 Go to previous messageGo to next message
gprentice is currently offline  gprentice
Messages: 260
Registered: November 2005
Location: New Zealand
Experienced Member
Didier wrote on Sat, 11 December 2010 06:50

Hi all,

Quote:


Actually, on second thoughts I think the code as written is fine without volatile because the mutex forces the shared data to be updated in memory. Possibly an atomic variable that is read or written outside of a mutex region is more likely to need volatile.



Yes this is the case, and is what I explained in my earlier post.



Just to be clear, apparently there are situations when volatile is useful
http://www.drdobbs.com/high-performance-computing/212701484

but in general, you need a memory barrier when accessing shared data.

After googling for a while, I haven't found any clear explanation of how a mutex solves the caching and memory visibility issues. I'm guessing that both acquire mutex and release mutex flush the entire memory cache for all CPUs/caches and prevent hardware re-ordering across the memory barrier - which deals with the hardware problem. For dealing with compiler/linker optimisation, I'm guessing that the call to the "external" acquire/release mutex function forces the compiler not to cache values across that call because it can't tell what memory that function call is going to access. Explanations of how a mutex works don't seem to mention these issues.


That AtomicVar class looks like a good idea. Should the copy constructor be
AtomicVar( const AtomicVar& p) { AtomicWrite(val, AtomicRead(p.val)); }

Graeme
Re: Use same variable in different threads [message #30147 is a reply to message #30138] Sat, 11 December 2010 23:13 Go to previous messageGo to next message
Didier is currently offline  Didier
Messages: 680
Registered: November 2008
Location: France
Contributor
Hi Graeme,

I'm very far from beeing an expert and I faced the same problems as you when I had to write some tough MT code: no clear documentation on the subject. Confused
Thank's for the link, very interresting Razz


That AtomicVar class looks like a good idea. Should the copy constructor be
AtomicVar( const AtomicVar& p) { AtomicWrite(val, AtomicRead(p.val)); }


Yes, it was what I coded at first but it didn't compile because AtomicRead() does not accept const parameter.

Anyway this sample class needs some polishing and all the classic operators should also get overloaded.

[Updated on: Sat, 11 December 2010 23:30]

Report message to a moderator

Re: Use same variable in different threads [message #30148 is a reply to message #30147] Sun, 12 December 2010 01:57 Go to previous messageGo to next message
gprentice is currently offline  gprentice
Messages: 260
Registered: November 2005
Location: New Zealand
Experienced Member
Hi Didier

This compiles for me with MSC10 and GCC something. I had to change operator= to get it to compile. It's ok for CV qualification to increase so it *should* compile.
BTW - it's a little bit confusing that on Win32, Atomic is declared as long but the Atomic functions have int parameters. Possibly operator int() below could be operator Atomic(), to be more platform independent.

 class AtomicVar
{
	private:
		Atomic val;
	public:
		AtomicVar() {};
		
		AtomicVar( const AtomicVar& p) { AtomicWrite(val, AtomicRead(p.val)); }
		template <class T>
		AtomicVar& operator=(const T& p) { AtomicWrite(val, p); return *this; }

		operator int() {return AtomicRead(val); }
	
};


Graeme

[Updated on: Sun, 12 December 2010 01:57]

Report message to a moderator

Re: Use same variable in different threads [message #30154 is a reply to message #30138] Sun, 12 December 2010 09:43 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
gprentice wrote on Fri, 10 December 2010 19:37


After googling for a while, I haven't found any clear explanation of how a mutex solves the caching and memory visibility issues.



Because mutex acts as memory barrier... Smile

If you read _very_ carefully the POSIX threads docs, it can be (sort of) derived from the information there. Well, to be more specific, the mutex implementation has to act as memory barrier to support POSIX mutex specification...

Mirek
Re: Use same variable in different threads [message #30167 is a reply to message #30154] Sun, 12 December 2010 23:16 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Hello all

Nice answers Smile !

For now if I understand well I will use:

- AtomicVar class. Nice!

- Mutex as here:

struct mySharedData {
	int a, b;
	String c;
	Mutex m;
};

mySharedData data;

void ThreadFunction(){
	for(int i = 0; i < 10; i++){
		Thread::Sleep(Random(10)); // Pretend some work...
		data.m.Enter();	    // Enter the section that accesses the shared data
		data.a = i;
		data.c << data.a << "\n";
		Thread::Sleep(20); //Let's pretend that some slow operation happens here (for example file access)
		data.b = data.a;
		data.m.Leave();    // we don't need the exclusive access to data anymore, release the mutex
	}
}




Best regards
Iñaki
Re: Use same variable in different threads [message #30172 is a reply to message #30167] Mon, 13 December 2010 11:35 Go to previous messageGo to next message
tojocky is currently offline  tojocky
Messages: 607
Registered: April 2008
Location: UK
Contributor

koldo wrote on Mon, 13 December 2010 00:16

Hello all

Nice answers Smile !

For now if I understand well I will use:

- AtomicVar class. Nice!

- Mutex as here:

struct mySharedData {
	int a, b;
	String c;
	Mutex m;
};

mySharedData data;

void ThreadFunction(){
	for(int i = 0; i < 10; i++){
		Thread::Sleep(Random(10)); // Pretend some work...
		data.m.Enter();	    // Enter the section that accesses the shared data
		data.a = i;
		data.c << data.a << "\n";
		Thread::Sleep(20); //Let's pretend that some slow operation happens here (for example file access)
		data.b = data.a;
		data.m.Leave();    // we don't need the exclusive access to data anymore, release the mutex
	}
}





But what about read control?
Example:
read data.a
write data.a and data.b
read data.b (b will be modified)

In this case I think:
1. the write look need to wait until the all reads was finished. and every read locks need to wait until the write lock was finished.
2. Or reads and writes will use only one lock (data.m). But in this case will not possible multiple parallel reads.

Later I will try to provide a simple example.

[Updated on: Mon, 13 December 2010 20:21]

Report message to a moderator

Re: Use same variable in different threads [message #30173 is a reply to message #30167] Mon, 13 December 2010 11:46 Go to previous messageGo to next message
gprentice is currently offline  gprentice
Messages: 260
Registered: November 2005
Location: New Zealand
Experienced Member
I'm guessing that if you have one thread writing to shared data and another thread reading the shared data (no writing), both threads have to acquire the mutex before accessing the data - even a thread that's only reading. There are ways of organizing things so that multiple threads can have read access to the data at the same time, instead of all blocking each other.

http://en.wikipedia.org/wiki/Readers-writers_problem

Graeme

Edit - oops, I didn't see tojocky's post before I wrote mine but it looks like we're on the same subject.

[Updated on: Mon, 13 December 2010 11:48]

Report message to a moderator

Re: Use same variable in different threads [message #30181 is a reply to message #30173] Mon, 13 December 2010 22:48 Go to previous messageGo to previous message
Didier is currently offline  Didier
Messages: 680
Registered: November 2008
Location: France
Contributor
Hi Koldo,

I would add just one last thing:
DEADLOCKs ==> what MT programs dread the most.

In practice, there is a case in C++ where deadlocks can appear without notice : when an exception occurs.

Imagine an exception occurs in the middle of you're "slow operation" ==> the mutex doesn't get released ==> a deadlock will come up soon enough Confused

To avoid this I use, what I call a ScopedLock class:

template<class MUTEX>
class ScopedLock
{
private:
	MUTEX& mutex;

private:
	// The following constructor/operators are expilictly FORBIDDEN
        // because they have no meaning
	ScopedLock(void) {};
	ScopedLock(const ScopedLock& ) {};
	ScopedLock& operator=(ScopedLock& ) { return *this; };

public:
	inline 	ScopedLock(MUTEX& mut)
	: mutex(mut)
	{
		mutex.lock();
	}

	inline ~ScopedLock(void)
	{
		mutex.unLock();
	}
};


The point is to create a 'ScopedLock' object when interring a protected zone of code, and when the scope ends ==> the unlock is automatically done IN ALL POSSIBLE CASES !! even exceptions: The compiler handles all for you Cool

so you're code would become:
void ThreadFunction(){
	for(int i = 0; i < 10; i++){
		Thread::Sleep(Random(10)); // Pretend some work...
		{
			ScopedLock(data.m);// Enter the section that accesses the shared data
			data.a = i;
			data.c << data.a << "\n";
			Thread::Sleep(20); //Let's pretend that some slow operation happens here (for example file access)
			data.b = data.a;
		} // implicit release
	}
}


You don't have any more "mutex leaks" possible
Previous Topic: OpenMP
Next Topic: Different native pthread.h implementations
Goto Forum:
  


Current Time: Fri Mar 29 08:39:00 CET 2024

Total time taken to generate the page: 0.01321 seconds