|
|
|
Home » Community » Coffee corner » The power of Makefile
 The power of Makefile [message #24947] |
Fri, 05 February 2010 15:45  |
|
|
Hello everybody!
I just have to share this bit of kowledge I've just gained with all of you... Until yesterday, I thought that make is just another simple unix tool. But after reading Managing projects with GNU make, I changed my mind.
Actually make is more like a programming language. It provides a lot of features and possibilities I would ever think of. Like defining functions, built-in rules, pattern expansion and many more.
Just to show you (at least to those of you who don't know make really good): Following Makefile is just a few lines, but it can analyze U++ package dependencies. For now, it is restricted to one directory, so it works only for uppsrc or in directory with exported project. Also, it ignores flags (just includes everything in "uses") . Both of this could be probably fixed with few more lines. Another problem might be it's speed. For ide it takes about 12s on my laptop, but for average-sized packages, it's less than second (also it's definitely not optimal, some steps are duplicated). Ok, here is the code:
PKG=ide
DEP=$(PKG)
define get-upp
$(if $(findstring /,$1),$1/$(notdir $1).upp,$1/$1.upp)
endef
define get-uses
$(strip $(shell cat $(call get-upp,$1) | tr ",\n;" " \n" | grep "uses" | sed 's/(.*)//'))
endef
define get-deps
$(sort $1 $(foreach d,$(sort $(subst \,/,$(filter-out uses%,$(call get-uses,$1)))),$(call get-deps,$d)))
endef
.PHONY: all
all:
@echo "To build $(PKG) following packages are needed:"
@echo "$(call get-deps,$(PKG))"
If you read all the way down here, thanks for your patience 
Best regards,
Honza
|
|
|
|
|
|
|
|
| Re: The power of Makefile [message #24983 is a reply to message #24950] |
Sat, 06 February 2010 20:39  |
|
|
| masu wrote on Fri, 05 February 2010 16:32 |
| luzr wrote on Fri, 05 February 2010 16:04 |
Does it work with BSD-make too?
|
Unfortunately this does not work with BSD make.
It does not support function definitions and it also does not have build in functions like 'findstring'.
Matthias
|
It should work with gmake. From what I read, it is fairly standard on BSD. I think some ports even require gmake to build properly...
Bye,
Honza
|
|
|
|
|
|
|
|
| Re: The power of Makefile [message #24999 is a reply to message #24991] |
Sat, 06 February 2010 23:34 |
|
|
| masu wrote on Sat, 06 February 2010 22:55 |
I tried your Makefile with gmake on FreeBSD and it works fine  .
Matthias
|
Hi Matthias,
Thanks for testing! I was just playing with this idea, so I was not concerned about compatibility. After looking into BSD make docs, I believe I could do the same in BSD style too. It would just contain more shell commands. And it wouldn't be compatible with GNU make BSD make uses different syntax for conditional inclusions. So if we want to use more of the makes power, we should stick to GNU version as it is available pretty much anywhere, from Linux through BSD and mac to cygwin.
Now, does someone actually think (like me ) that this is fascinating? Having universal Makefile for any project is tempting idea, even though I'm not sure if it would be really useful.
Honza
|
|
|
|
|
|
| Re: The power of Makefile [message #25220 is a reply to message #25001] |
Sun, 14 February 2010 21:08 |
masu
Messages: 378
Registered: February 2006
|
Senior Member |
|
|
Hi,
Like I mentioned here:
http://www.ultimatepp.org/forum/index.php?t=msg&th=4968& amp; amp;start=0&
I took up Honza's tests with GNU make and I would like to extend the idea to have a Makefile for building arbitrary U++ packages.
The version I have at the moment is supporting packages that are located including all its prerequisite packages. It does not support building packages distributed over several assemblies, but for now the Makefile should be put into uppsrc assembly and can then be used to build TheIDE.
I have tested it with FreeBSD and Ubuntu. I have also tried it in OpenBSD, but there seem to be other problems not related to the make process. At least all parameters (i.e. source files, link libraries) were extracted correctly on that system, too.
Here is a checklist I would like to work on:
1. Add support for building from other directories (to only provide on Makefile in root directoy for source releases)
2. Add support for setting object files' output directory
3. Add support for package building over different assemblies
And of course any error fixing in the meantime if erros occur during testing.
Edit: I forgot to mention, that per default (if you just type make) TheIDE is build with GTK bindings and shared linking.
Tip: You van also build with make -jx where x is the number of parallel processes run by make. I used parameter -j2 on my Dual Core system.
Matthias
-
Attachment: Makefile.tbz
(Size: 1.89KB, Downloaded 419 times)
[Updated on: Sun, 14 February 2010 21:14] Report message to a moderator |
|
|
|
 Re: The power of Makefile [message #25223 is a reply to message #25220] |
Sun, 14 February 2010 22:21 |
|
|
Hi Matthias,
You've got my deep respect! I didn't expect anyone even trying my ideas, let alone doing some work. But you almost finished it and actually much faster than I expected that it would take. Very impressive!
I'm just testing the Makefile and trying to figure out how it works. I have problems to even find the parts that I wrote But it seems to be working perfectly. Theide compiled perfectly on my machine, so I guess there should be no problems with other packages. And by the way, thanks for the hint with -j parameter for parallel builds, I didn't know it is that easy.
Now about the Makefile. First of all, I love the way you made it verbose. Reporting every step is really user friendly. I would even suggest (optional) hiding of the commands, so only the commands would be visible. About your TODO list: Numbers 1 and 2 should be easy. Just adding a variable with path on proper places. I'll try to figure it out. Number 3 is trickier, but most important. Currently, only uppsrc packages can be built, since other standard assemblies span across two directories. Hopefully we will find a way how to search through both.
Oh, and I almost forgot, another great think about this, that was not possible with the exported Makefiles: if you put it in directory with all sources, you can build any number of (main) packages to producing several binaries, while effectively caching the .o files. Similar way as theide does it. I think this is the biggest feature we have got here. I guess we could even add one more cycle and build multiple binaries at once with something like make "PKG=SocketClient SocketServer". Something like this is quite difficult to do in theide...
I would be very happy if this Makefile could be once distributed with U++ as an alternative to current exported Makefiles. I'm aware that it will never supersede it, because of it's complexity and also because the exported files are BSDmake compatible. This one just looks better to me. Even if just because it compresses 37000 lines into 150 It is really elegant.
Best regards,
Honza
|
|
|
|
|
|
| Re: The power of Makefile [message #25253 is a reply to message #25252] |
Mon, 15 February 2010 16:50 |
|
|
| masu wrote on Mon, 15 February 2010 16:24 |
Hi Honza,
first, thanks for your appreciation .
A problem I have is to find a way to define variables when calling make which can be extended then during the make flow.
I thought VAR+=x should work, but it does not work for env variables defined as parameter to make, like this:
make DEFS=-DflagNOGTK
In this case the variable DEFS is completely overwritten by -DflagNOGTK. But what I want is to add this flag to already defined flags inside the Makefile.
Do you have an idea?
Matthias
|
Hi,
This is usually solved using another variable: FIXEDFLAGS=...
FLAGS=$(FIXEDFLAGS) $(DEFS)
Simple, stupid, but works
Honza
|
|
|
|
|
|
| Re: The power of Makefile [message #25841 is a reply to message #24947] |
Mon, 15 March 2010 08:57 |
|
|
Hello all!
After a long time I finally got to back to the Makefile... and I finally finished everything I wanted to.
So the file I present you here is rewritten from scratch. It is faster and smarter. New features are:
- Compiles multiple packages at once
- Reads and uses most of the options from .upp in the same way as theide
- Verbosity switches
- Simulation mode
- Help target
- Commented code (a bit)
- Several bugs fixed
- Little ASCII Art bonus
The last thing I miss is blitz
BTW: There are even some options that can not be set in theide Or at least I couldn't find how... E.g. optimize file/package for size.
To get started, just copy the file into upp directory. I advise you to start with typing Both show you a help text, that contains a description and default values for variables that can controll the build process.
If you want to try something more elaborated, you can test something like this: make "PKG=ide usvn `ls examples reference tutorial |sed 's/[^ ]*://g' |tr '\n' '�'`" USEMAINCFG=y BINDIR=bin/ BINEXT= JOBS=3
I tested the makefile by building more than 200 applications with various flags and randomly running many of them, without any encountering any errors in the makefile. So I believe that it should work in most cases. If anyone tries this, please let me know about your experiences Also, if you have any questions, feel free to ask (on forum or IRC).
Best regards,
Honza
-
Attachment: Makefile
(Size: 19.07KB, Downloaded 367 times)
|
|
|
|
|
|
|
|
|
|
| Re: The power of Makefile [message #25846 is a reply to message #25842] |
Mon, 15 March 2010 13:02 |
|
|
| chickenk wrote on Mon, 15 March 2010 11:38 |
Hi Honza,
I tried it, it is absolutely great. Thank you very much for this effort, much appreciated.
Just for record, I added a very small rule 'showflags' (patch attached) that displays the flags for the main package as they are guessed. Example:
$ make PKG=ide showflags
ide_FLAGS: GCC LINUX MAIN
$ make PKG=ide USEMAINCFG=y showflags
ide_FLAGS: GCC GUI LINUX MAIN
Can be useful to know these flags before actually trying to compile.
Another way to do that without the patch:
$ make PKG=ide USEMAINCFG=y -p | grep '^ide_FLAGS'
ide_FLAGS := GCC GUI LINUX MAIN
Just my 2 cents. Keep up this good work, will surely help u++ to expand.
|
Thank you, chickenk!
If you think this is helpful, I will add it. Usability is my main goal Also in future I plan (optional) interactive mode, where user could choose which set of flags from mainconfig to use and some other tricks...
Regards,
Honza
|
|
|
|
| Re: The power of Makefile [message #25847 is a reply to message #25843] |
Mon, 15 March 2010 13:12 |
|
|
| koldo wrote on Mon, 15 March 2010 12:13 |
Hello Honza
It looks great. What should they be the next steps to integrate this and Launchpad into U++ infrastructure ?
|
Hi Koldo!
This has a little common with Launchpad. I'm not sure why the nightly builds doesn't work yet, probably Mirek is busy... But yes, I'm thinking about using this instead of generated makefiles, it is actually as easy as overwriting one file and changing few lines of script. I'll have a look at it soon.
About integration into U++: This is easy too. Just include the file in builds and mention it in documentation. Additionally someone might integrate it into theide. Probably in similar style as current makefiles, just copying this makefile into export directory and write correct defaults in it.
Regards,
Honza
|
|
|
|
| Re: The power of Makefile [message #25892 is a reply to message #24947] |
Thu, 18 March 2010 00:18 |
|
|
Hello all!
New version is here! Changelog follows:
- Fixed bug when running via -f option (thanks Matthias)
- Modified output to make Chickenk more happy with SIMULATE mode output
- Added export target
- Expanded help
- Various little bugs fixed
The most important change is the export target. It parses the makefile with given options and saves it to a file given in EXPORT variable. The resulting file is faster (especially if you work with FAST=n), simpler, smaller, BSD-make compatibleand doesn't require any commandline parameters, since they are all hardcoded in it. You can sure magine now what can it be used for The only downside is that it is a bit messy inside, due to the nature of the exporting process and I don't think that it could be fixed (at least not easily).
Example: cd ~/uppsvn
make -f ~/Makefile PKG=ide "FLAGS=GUI SSE2 MT SPEED" FAST=n BIN=bin EXPORT=Makefile.ide
#later, as many times you want, possibly on different machine(s) (within same directory tree)
make -f Makefile.ide
Best regards,
Honza
-
Attachment: Makefile
(Size: 21.35KB, Downloaded 368 times)
|
|
|
|
Goto Forum:
Current Time: Sun Apr 27 18:36:09 CEST 2025
Total time taken to generate the page: 0.02814 seconds
|
|
|
Members
Pages
Help
Register
Login
Home
!
