Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » MediaPlayer is out
MediaPlayer is out [message #30314] Fri, 24 December 2010 08:57 Go to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Hello all

Finally it is there Smile. Please follow installing instructions. They are very simple but have to be strictly followed.

It includes MediaPlayer control and a demo. It has been more tested in Windows. A problem in Linux is that ffmpeg Ubuntu packages are very old, and all my efforts to compile from sources have failed.

It uses SDL (SDLCtrl) and ffmpeg:
- SDL gives fast YUV video and audio playing.
- ffmpeg gives audio and video decoding and encoding.

So in summary for playing:
- ffmpeg opens the file (.avi, .mp3, ...)
- ffmpeg reads the codified data and converts it in uncompressed images and audio chunks.
- SDL displays the images and plays the audio fluently.

It uses at least 5 threads... plus the main:
- basic data chunks (called packets) retrieval
- audio decoding
- video decoding
- subtitles decoding
- audio playing

Please try it and give all the feedback. Be as clear as possible, as the threads interaction is not easy.

The next immediate steps from now are:
- Solve problems Rolling Eyes
- Add RGB video playing to do video processing Smile
- Reduce number of threads... specially video decoding thread has to be removed (it has been the source of many problems)

Other improvements are:
- Add video and audio encoding Smile Smile (to generate new files, .avi. mp3, ...)
- Remove SDL dependencies...:
--- Mirek: To see how to do fast image rendering
--- Honza and Mirek: To play audio with PortAudio


If anybody has any special need, just s/he has to describe it Smile.




Best regards
Iñaki
Re: MediaPlayer is out [message #30317 is a reply to message #30314] Fri, 24 December 2010 13:03 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Hi Koldo,

Wonderful Christmas gift! Smile

I had a quick look at the audio playback part and I think it will be possible (and probably not very hard) to make a switchable PortAudio backend. The architecture is very similar and you already did all the hard work. I think that it will need just a few #defines... I'll have a look at it during the holidays.

Good work Thumbs Up

Honza

PS: I'm still thinking do we really need the SDL part? If the audio can be played using PortAudio, couldn't we also display the raw image data using U++ and get rid of the dependency? Smile
Re: MediaPlayer is out [message #30319 is a reply to message #30314] Fri, 24 December 2010 14:25 Go to previous messageGo to next message
jerson is currently offline  jerson
Messages: 202
Registered: June 2010
Location: Bombay, India
Experienced Member

I am a bit uninitiated, so I do not know how to.

I got the sdl and ffmpeg libraries shown in the links. Do I also need to download the nightly build to get access to the media player etc?

My current build is 2791.

Merry Christmas
Re: MediaPlayer is out [message #30321 is a reply to message #30319] Fri, 24 December 2010 15:25 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

jerson wrote on Fri, 24 December 2010 14:25

I am a bit uninitiated, so I do not know how to.

I got the sdl and ffmpeg libraries shown in the links. Do I also need to download the nightly build to get access to the media player etc?

Hi Jerson,

You need fresh U++ sources, version 2924 or newer. Last nights build is just fine. There is a package MediaPlayer in bazaar. If you have all the libraries, it should compile just fine. Just give it a try with some audio or video files Wink

Honza
Re: MediaPlayer is out [message #30323 is a reply to message #30321] Fri, 24 December 2010 17:40 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Quote:

PS: I'm still thinking do we really need the SDL part?
Yes Honza. One goal is to remove SDL dependencies.

Jerson: The Honza explanation is perfect.


Best regards
Iñaki
Re: MediaPlayer is out [message #30326 is a reply to message #30314] Fri, 24 December 2010 21:37 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
koldo wrote on Fri, 24 December 2010 02:57


--- Mirek: To see how to do fast image rendering



Would be SetSurface helpful here?
Re: MediaPlayer is out [message #30335 is a reply to message #30326] Sat, 25 December 2010 10:25 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
mirek wrote on Fri, 24 December 2010 21:37

koldo wrote on Fri, 24 December 2010 02:57


--- Mirek: To see how to do fast image rendering



Would be SetSurface helpful here?

Hello Mirek

Perhaps it is enough. I will add it as a possibility to compare performance. A drawback is that it requires conversion from YUV to RGB. I will dig in it.

Anyway, do you have a sample?


Best regards
Iñaki
Re: MediaPlayer is out [message #30337 is a reply to message #30335] Sat, 25 December 2010 10:55 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Sorry, no sample, but I guess the function singature tells it all Smile

void SetSurface(SystemDraw& w, int x, int y, int cx, int cy, const RGBA *pixels)

YUV->RGB might be a problem. I would even say that a bit of SSE2 code inlines will be necessarry... (Then there are also issues of rescaling... which I am afraid in SW is not fast enough).

Mirek
Re: MediaPlayer is out [message #30343 is a reply to message #30337] Sat, 25 December 2010 11:35 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Reference/SetSurface Rolling Eyes . Sorry, I have got a flu...

Best regards
Iñaki
Re: MediaPlayer is out [message #30345 is a reply to message #30343] Sat, 25 December 2010 20:23 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Hello Honza

Quote:

I had a quick look at the audio playback part and I think it will be possible (and probably not very hard) to make a switchable PortAudio backend. The architecture is very similar and you already did all the hard work. I think that it will need just a few #defines... I'll have a look at it during the holidays.

I would not do it switchable Smile. I would just replace SDL audio code with yours using PortAudio. The only problem is MSC.


Best regards
Iñaki
Re: MediaPlayer is out [message #30347 is a reply to message #30345] Sat, 25 December 2010 20:27 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Hello Mirek

Quote:

YUV->RGB might be a problem. I would even say that a bit of SSE2 code inlines will be necessary...


Yes, it is possible (I do not say easy Smile) to use hardware acceleration.

An approach would be to use Direct2D or perhaps OpenGL.


Best regards
Iñaki
Re: MediaPlayer is out [message #30348 is a reply to message #30347] Sat, 25 December 2010 20:37 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
An explanation:

The advantages of YUV are two:
- Video is usually encoded in YUV so, to show it, convert it to RGB is an additional step.
- An image in YUV is smaller than in RGB.


Best regards
Iñaki
Re: MediaPlayer is out [message #30353 is a reply to message #30347] Sun, 26 December 2010 01:04 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
koldo wrote on Sat, 25 December 2010 14:27

Hello Mirek

Quote:

YUV->RGB might be a problem. I would even say that a bit of SSE2 code inlines will be necessary...




SSE2 is not that hard, really, using intrinsics (which are MSC/GCC compatible).
Re: MediaPlayer is out [message #30488 is a reply to message #30314] Tue, 04 January 2011 19:13 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Hello guys,

I learned quite a bit about ffmpeg reading Koldos sources and the documentation on the web (altough it is lacking many important things - almost like U++ docs Smile ).

The result of my work in this field is a conclusion that audio/video playback is really possible using only U++ ffmpeg and portaudio. In the attachment you will find a proof of concept video player. It is very simple compared to the one Koldo published, but it is a learning tool for me and as such it is adequate. Its main characteristics are: oversimplified, single threaded, blocking (no UI interaction while playing).

The video display is done using SetSurface, which proved to be fast enough for this purpose (good work, Mirek! Smile ). Scaling and YUV->RGBA conversion is done in single step using sws_scale from libsws that comes with ffmpeg. It should use hardware acceleration where available, but on my system everything was done in software. The rescaling uses bilinear algorithm. I tested with bicubic as well but it became very slow for higher resolutions. The audio output is done in the simplest possible way using blocking SoundStream from my Sound package published in bazaar. The blocking nature of the sound API is also misused to provide audio-video syncing. For mute videos there is a quick hack to make Paint() block until it is the right time to output the image.

If you decide to test, I recommend you to try with shorter movies. Once you start it, it runs until the end (or SIGKILL). No seeking, no pause, no interaction while playing. This should be actually easy to fix using threads and some pieces of code from Koldo.

I tested this on my Intel Atom netbook, i.e. with no graphics card to speak of. I got fluent playback for most of the movies on my hard-drive, but didn't test any HD clips. Actually the slowest part is the resizing, so if the HD resolution matches with your monitor resolution, it should play fine Smile For me there were some sound buffer underflows when playing better quality movies on fullscreen (1024x600), but playing them in native resolution was fine.

I left few RTIMING macros in the code so you can make your idea about how fast each part of processing is Wink Generally speaking audio decoding is very fast, SetSurface scales linearly with size but the scaling can do weird thing depending on the size and on original resolution.

So to sum it up: ffmpeg+portaudio+U++ is a combination sufficient to do most of the common video processing and SDL dependency will most definitely be dropped soon. Which is IMHO great news Cool

Best regards,
Honza
Re: MediaPlayer is out [message #30489 is a reply to message #30314] Tue, 04 January 2011 19:13 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

EDIT: Deleted accidental double post...

[Updated on: Tue, 04 January 2011 19:36]

Report message to a moderator

Re: MediaPlayer is out [message #30495 is a reply to message #30489] Wed, 05 January 2011 10:06 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Hello Honza

This post is about MediaPlayer. If you want to do other things please open a new post.

I do not understand you: I informed many months ago about this project and its advance. I released it a couple of weeks ago and now, voilà, you release a package to do almost the same.

One week ago we chatted in ICQ about it and I opened the package to you, to do a join effort. Now you break your word and do it by yourself without any previous comment. For me it is dishonest, it is a backstab.

It is true anybody can take, lets say, all U++ project and open tomorrow a new project in Sourceforge, a new web page, change a few things including the name and logo, and begin a new project. It is legal. The license permits it. It is called fork. But if there is no previous provocation and if the original project has been fair and all project contributors have had opportunities proportional to their effort, to open a fork is not fair.

For sure I am of the old school. But if U++ project acts this way I do not want to be here.


Best regards
Iñaki
Re: MediaPlayer is out [message #30496 is a reply to message #30495] Wed, 05 January 2011 10:14 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Hello all

Now MediaPlayer control can manage video image as RGB. It means that programmer can easily process the image. It is explained here.

To show this MediaPlayer packages includes demos to set a logo over the live video, to detect motion in image and to set to grayscale a moving rectangle in the image.

It is easy to handle through callback function WhenFrame(SDLSurface &surface), called every time a video frame is displayed. And the examples are very simple.

Just a sample:

index.php?t=getfile&id=3017&private=0
  • Attachment: BW.PNG
    (Size: 67.96KB, Downloaded 689 times)


Best regards
Iñaki
Re: MediaPlayer is out [message #30499 is a reply to message #30495] Wed, 05 January 2011 11:28 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

koldo wrote on Wed, 05 January 2011 10:06

Hello Honza

This post is about MediaPlayer. If you want to do other things please open a new post.

I do not understand you: I informed many months ago about this project and its advance. I released it a couple of weeks ago and now, voilà, you release a package to do almost the same.

One week ago we chatted in ICQ about it and I opened the package to you, to do a join effort. Now you break your word and do it by yourself without any previous comment. For me it is dishonest, it is a backstab.

It is true anybody can take, lets say, all U++ project and open tomorrow a new project in Sourceforge, a new web page, change a few things including the name and logo, and begin a new project. It is legal. The license permits it. It is called fork. But if there is no previous provocation and if the original project has been fair and all project contributors have had opportunities proportional to their effort, to open a fork is not fair.

For sure I am of the old school. But if U++ project acts this way I do not want to be here.


Hi Koldo,

Please don't get me wrong. The package I posted here definitely does not focus on the same things as the MediaPlayer and I did not have any intentions to steal your credit.

The purpose why I did this was to learn better the ffmpeg internals to be able to help you with the portaudio sound, just as we talked about. I was also interested in the possibilities of SetSurface, as it would allow to drop SDL completely, which we agreed to be a worthy goal. So that is a reason why I didn't concentrate only on audio but also put in the video part. Working directly on your sources was just too difficult for me to begin with, so I decided to create the simplest possible implementation to learn how the things work.

I wouldn't call this fork. Yes, I was partly inspired by parts of your code, but the main part was based on an ffmpeg example source (about saving video frames to picture files). The whole thing is a stub, mere learning tool or demo, not suitable for use for any other purpose. I was hoping that you would look at it and reuse (or just get inspired by) the important parts, that is the audio output and sws_scale&SetSurface usage.

I believe this is just misunderstanding and that you will understand my reasons and intentions now.

Best regards,
Honza
Re: MediaPlayer is out [message #30527 is a reply to message #30496] Thu, 06 January 2011 16:20 Go to previous messageGo to next message
Zbych is currently offline  Zbych
Messages: 325
Registered: July 2009
Senior Member
Hello Koldo,

I've just tried your MP on ubuntu (10.10) and ffmpeg from svn. There were few compilation/linking errors I had to correct:
- multiple definitions of av_free_packet, av_get_codec_tag_string, avsubtitle_free. I think that those functions should stay in their original location (libavcodec/utils.c etc.)
- alsa is enabled by default in ffmpeg, so maybe asound library should be added to link libraries in media package?
- by default libavcodec/audioconvert.h and libavutil/colorspace.h are not copied to include directory (/usr/local/include). Are you sure you can use them outside ffmpeg?

MP crushes when movie is playing and I try to resize the window. If I resize window and then pres play, movie is played, but it stops after few seconds.
Re: MediaPlayer is out [message #30635 is a reply to message #30527] Thu, 13 January 2011 09:20 Go to previous messageGo to previous message
koldo is currently offline  koldo
Messages: 3356
Registered: August 2008
Senior Veteran
Zbych wrote on Thu, 06 January 2011 16:20

Hello Koldo,

I've just tried your MP on ubuntu (10.10) and ffmpeg from svn. There were few compilation/linking errors I had to correct:
- multiple definitions of av_free_packet, av_get_codec_tag_string, avsubtitle_free. I think that those functions should stay in their original location (libavcodec/utils.c etc.)
- alsa is enabled by default in ffmpeg, so maybe asound library should be added to link libraries in media package?
- by default libavcodec/audioconvert.h and libavutil/colorspace.h are not copied to include directory (/usr/local/include). Are you sure you can use them outside ffmpeg?

MP crushes when movie is playing and I try to resize the window. If I resize window and then pres play, movie is played, but it stops after few seconds.


Hello Zbych

Thank you, I will check it all.


Best regards
Iñaki
Previous Topic: Sound device access system based on PortAudio
Next Topic: U++ development: Two philosophies
Goto Forum:
  


Current Time: Fri Apr 19 19:16:22 CEST 2024

Total time taken to generate the page: 0.02756 seconds