Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » The age of multicore has just started for U++....
The age of multicore has just started for U++.... [message #8929] Sun, 08 April 2007 12:00 Go to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
...well, perhaps only a little step, but I have just made experimental U++ website generation package, using the new "CoWork" class (in the next dev release) to use both of my E4300 cores to do the job, to became the "real thing" (removing "experimental" status). Means I will now use to do the "real job".

Reduces U++ website generation from 23s to 14s.

Mirek
Re: The age of multicore has just started for U++.... [message #8932 is a reply to message #8929] Sun, 08 April 2007 16:33 Go to previous messageGo to next message
exolon is currently offline  exolon
Messages: 62
Registered: July 2006
Location: 53'21N 6'18W
Member
What does this Cowork class do, and how?
Re: The age of multicore has just started for U++.... [message #8933 is a reply to message #8932] Sun, 08 April 2007 17:14 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
exolon wrote on Sun, 08 April 2007 10:33

What does this Cowork class do, and how?


Well, detailed description will come with new Core, but essentially it is a thread pool. A tool to simply paralellize loops.

E.g. this is some original loop from the uppweb:

VectorMap<String, String> reflink;

struct ScanTopicIterator : RichText::Iterator {
	String         link;

	virtual bool operator()(int pos, const RichPara& para)
	{
		if(!IsNull(para.format.label)) {
			reflink.Add(para.format.label, link);
		}
		return false;
	}
};

void GatherRefLinks(const char *upp)
{
	Progress pi;
	pi.AlignText(ALIGN_LEFT);
	for(FindFile pff(AppendFileName(upp, "*.*")); pff; pff.Next()) {
		if(pff.IsFolder()) {
			pi.Step();
			String package = pff.GetName();
			String pdir = AppendFileName(upp, package);
			TopicLink tl;
			tl.package = package;
			for(FindFile ff(AppendFileName(pdir, "*.tpp")); ff; ff.Next()) {
				if(ff.IsFolder()) {
					String group = GetFileTitle(ff.GetName());
					tl.group = group;
					String dir = AppendFileName(pdir, ff.GetName());
					for(FindFile ft(AppendFileName(dir, "*.tpp")); ft; ft.Next()) {
						if(ft.IsFile()) {
							String path = AppendFileName(dir, ft.GetName());
							tl.topic = GetFileTitle(ft.GetName());
							String link = TopicLinkString(tl);
							pi.SetText("Indexing topic " + tl.topic);
							ScanTopicIterator sti;
							sti.link = link;
							ParseQTF(ReadTopic(LoadFile(path))).Iterate(sti);
						}
					}
				}
			}
		}
	}
}



parallel version with CoWork:

StaticCriticalSection     reflink_lock;
VectorMap<String, String> reflink;

struct ScanTopicIterator : RichText::Iterator {
	String         link;

	virtual bool operator()(int pos, const RichPara& para)
	{
		if(!IsNull(para.format.label)) {
			INTERLOCKED_(reflink_lock)
				reflink.Add(para.format.label, link);
		}
		return false;
	}
};


static void sDoFile(const char *path, const char *link)
{
	ScanTopicIterator sti;
	sti.link = link;
	ParseQTF(ReadTopic(LoadFile(path))).Iterate(sti);
}

void GatherRefLinks(const char *upp)
{
	CoWork work;
	Progress pi;
	pi.AlignText(ALIGN_LEFT);
	for(FindFile pff(AppendFileName(upp, "*.*")); pff; pff.Next()) {
		if(pff.IsFolder()) {
			pi.Step();
			String package = pff.GetName();
			String pdir = AppendFileName(upp, package);
			TopicLink tl;
			tl.package = package;
			for(FindFile ff(AppendFileName(pdir, "*.tpp")); ff; ff.Next()) {
				if(ff.IsFolder()) {
					String group = GetFileTitle(ff.GetName());
					tl.group = group;
					String dir = AppendFileName(pdir, ff.GetName());
					for(FindFile ft(AppendFileName(dir, "*.tpp")); ft; ft.Next()) {
						if(ft.IsFile()) {
							String path = AppendFileName(dir, ft.GetName());
							tl.topic = GetFileTitle(ft.GetName());
							String link = TopicLinkString(tl);
							pi.SetText("Indexing topic " + tl.topic);
							work & callback2(sDoFile, path, link);
						}
					}
				}
			}
		}
	}
}


The advantage is that you do not have start/join/manage threads, in fact, CoWork starts a couple of threads ("thread pool") first time it is used and never quits them until application exists.

Mirek
Re: The age of multicore has just started for U++.... [message #8938 is a reply to message #8929] Sun, 08 April 2007 22:50 Go to previous messageGo to next message
exolon is currently offline  exolon
Messages: 62
Registered: July 2006
Location: 53'21N 6'18W
Member
Nice... so it looks like an asynchronous function call using threads?
Re: The age of multicore has just started for U++.... [message #8975 is a reply to message #8933] Wed, 11 April 2007 05:23 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
Sorry. I do not want to be boring, but isn't easier to use specialized libraries for this purpose? Something like ACE (http://www.cs.wustl.edu/~schmidt/ACE-overview.html).

Those guys are 15 years ahead of you.


Regards,
Novo
Re: The age of multicore has just started for U++.... [message #8986 is a reply to message #8975] Wed, 11 April 2007 10:35 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Novo wrote on Tue, 10 April 2007 23:23

Sorry. I do not want to be boring, but isn't easier to use specialized libraries for this purpose? Something like ACE (http://www.cs.wustl.edu/~schmidt/ACE-overview.html).

Those guys are 15 years ahead of you.


Maybe. How does the above rewrite look like in ACE? (I was unable to find anything about thread pool in docs).

Moreover, the thread pool patter is well known. However, the aim of CoWork is to provide interface/implementation that exploits the rest of U++ library, like callbacks (perfect fit here btw).

Mirek
Re: The age of multicore has just started for U++.... [message #9004 is a reply to message #8975] Wed, 11 April 2007 20:09 Go to previous messageGo to next message
zsolt is currently offline  zsolt
Messages: 697
Registered: December 2005
Location: Budapest, Hungary
Contributor
Novo wrote on Wed, 11 April 2007 05:23

Sorry. I do not want to be boring, but isn't easier to use specialized libraries for this purpose? Something like ACE (http://www.cs.wustl.edu/~schmidt/ACE-overview.html).

Those guys are 15 years ahead of you.

I tried ACE some years ago, but it is a very oldschool stuff, compared to U++, I think.
Re: The age of multicore has just started for U++.... [message #9023 is a reply to message #9004] Fri, 13 April 2007 02:41 Go to previous messageGo to next message
okigan is currently offline  okigan
Messages: 16
Registered: March 2006
Promising Member
ACE is very heavy dependency...

Why the peculiar choice of & operator overloading?

-Okigan
Re: The age of multicore has just started for U++.... [message #9026 is a reply to message #9023] Fri, 13 April 2007 08:23 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
okigan wrote on Thu, 12 April 2007 20:41

ACE is very heavy dependency...

Why the peculiar choice of & operator overloading?

-Okigan


Please, consider this one "highly experimental" (and questionable), but IMO it resembles ending linux console line with '&'....

Of course, simple method named "Do" would do to Smile

Mirek
Re: The age of multicore has just started for U++.... [message #11071 is a reply to message #8929] Sat, 18 August 2007 07:36 Go to previous messageGo to next message
kenhty is currently offline  kenhty
Messages: 1
Registered: August 2007
Junior Member
Anyone consider using TBB ( http://osstbb.intel.com/ )
Re: The age of multicore has just started for U++.... [message #11073 is a reply to message #11071] Sat, 18 August 2007 10:05 Go to previous message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
kenhty wrote on Sat, 18 August 2007 01:36

Anyone consider using TBB ( http://osstbb.intel.com/ )


I have noticed its existence.

I have looked at the example of parallel_for and at the first glance, it looks as inferior solution to me... A lot of code to write, compared to CoWork.... (I might be wrong... I will perhaps try to reimplement the example given by Intel with CoWork).
Previous Topic: Anybody knows how to get number of CPU cores?
Next Topic: svn
Goto Forum:
  


Current Time: Fri Apr 26 16:37:53 CEST 2024

Total time taken to generate the page: 0.03945 seconds