We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpSound,  Music Libraries › Synchronizing generative video + audio
Pages: 1 2 3 4 
Synchronizing generative video + audio (Read 29944 times)
Re: Synchronizing generative video + audio
Reply #30 - Aug 29th, 2007, 12:42am
afpAccessDenied is a quicktime/macos error. it means that it can't get to the file for some reason, which could be things like 1) the file isn't there 2) you don't have permissions to read it 3) it's in use by another process, etc..
Re: Synchronizing generative video + audio
Reply #31 - Oct 7th, 2007, 5:30pm


 I need some advice on compression and codecs for an audio-reactive audio-visual project that I made with processing. I use Mr. Bollinger's code to write the movie file while analyzing the audio file off-line. Animation codec seems to be the best choice to get the highest quality video file. However, the data rates are so high, normally, I can't even play it properly on my mac. The thing is, I need to get this animation on a DVD, which is restricted to 9 Mbps, and I need to find the right compression type and data rates to do this.

 I tried (using moviemaker) cinepak, sorenson, h263/264 to get small files but the quality is considerably reduced. Animation and raw types are great, but I need to compress them anyway in order to make a DVD-video disc.

 So what would you suggest? I couldn't get the animation codec compressed with acceptable quality and the others don't work very well. What is the procedure that you would follow in order to do what I want to do? Say, would you write the animation file in a lossless codec in the first place, and then compress it (with which compression method?) to put it on a DVD, or would you save it in a lossy codec while writing it in the first place from processing? Also, is it possible to limit the data rate when writing the file with moviemaker?



Re: Synchronizing generative video + audio
Reply #32 - Oct 8th, 2007, 12:00am
I posted the "moviemaker" version of the code simply as a convenience for those who use that approach.  I myself don't use it, and prefer to write out individual uncompressed frames (tif, tga, png, bmp, whatever) and then do ALL compression as a post-processing (pun intended! Cheesy) step.

As to "HOW" you do that, it will depend on what software you have available.  And as to what compression format you want to use for compressed files, again, depends on target.  Just as a point of reference, I've had great success using the DivX codec when the target is computer playback or online playback transcoding (like youtube, vimeo, google, etc).

Your conversion path to get the best DVD output may vary - it is SO dependent on the authoring software you use (and I'm a Windows guy, so can't address Mac questions). Generally though, (for most software), you want to feed it the highest quality uncompressed video and have IT do the transcoding for you as part of the authoring process.  Check the manuals, or other references for your authoring software, as to what's the best input material format.

Hope that helps. Smiley
Re: Synchronizing generative video + audio
Reply #33 - Jan 9th, 2008, 9:33pm
does anyone know how to get the levels in ESS for each frame?

in the example on the first page you get the spectrum for each frame through

myFFT.getSpectrum(myChannel.samples, (int)(frameNumber * myChannel.sampleRate / framesPerSecond));

is there a similar way for the levels? I couldnt figure it out on my own.
Re: Synchronizing generative video + audio
Reply #34 - Jan 10th, 2008, 10:45am

It seems that you can do that with getLevel function.
From the sonia reference "returns the peak value for the specified channel."

Have a look at http://sonia.pitaru.com/ in the reference page.

Re: Synchronizing generative video + audio
Reply #35 - Jan 10th, 2008, 12:25pm
first of all i am using ess not sonia.

Ess also has a getLevel function but you can only get the levels if you actually play the sound. Since there is no level array (as there is for the spectrum), i dont really know how to handle it.
Re: Synchronizing generative video + audio
Reply #36 - Jan 10th, 2008, 1:11pm
Sorry for the bad answer was reading too quickly...
Sorry for that.

(brain overflow).
Re: Synchronizing generative video + audio
Reply #37 - Jan 10th, 2008, 4:15pm
okay anyways, I was wondering if there is also a way to grab the levels for each frame in ESS without actually playing the sound.(for rendering porpuses)

You can do it with the spectrum through this function:

myFFT.getSpectrum(myChannel.samples, (int)(frameNumber * myChannel.sampleRate / framesPerSecond));  

is there something similar in ESS for the levels?
Re: Synchronizing generative video + audio
Reply #38 - May 10th, 2008, 11:35pm
davbol wrote on May 9th, 2007, 6:50pm:
Thanks Chris.  Timestamp -- clever approach.

However, I've just re-RTFM and realized that (for my purposes anyway) I'm making this wayyy more difficult than it has to be.  With ESS, the FFT's getSpectrum() method will take a float array (um, samples) and an offset (um, timecode) - how did I miss that!.  IOW, offline analysis is already built in, you don't even need to play the audio at all.  Demo usage for others who might've missed it also:

Now just assemble the frames, overlay the audio, and viola! perfect sync.  Smiley


i'm working on fft and have problem with saving sequencial spectrum images. Btw, i've tried davbol's code. It really helps but i don't know what's P3D (in page 1) here i extract:

void setup() {

can anyone pls explain

Re: Synchronizing generative video + audio
Reply #39 - May 11th, 2008, 1:21am
P3D is processing software 3d rendering engine.
you can choose from p3d or opengl
Re: Synchronizing generative video + audio
Reply #40 - May 11th, 2008, 4:09pm
thanks crmx.
if you dont mind i have more questions about "try-catch" methods. After reading from many threads, i've come acroos that "try-catch" are only needed if the application works on the internet?
or i misunderstood?

Re: Synchronizing generative video + audio
Reply #41 - Jun 12th, 2008, 11:55am
lol, u're quite on the wrong track there

try-catch is also called error handling and is used to CATCH errors, so that the application does not quit when errors occur.

for example, a division by zero will throw an error and quit your application:

int test = 10 / 0;

to avoid that the application quits you have to TRY the code first and CATCH the errors:

 int test = 10 / 0;
}catch(Exception e){
 println("look! i caught folling error: " + e);  

however, this is the wrong thread to ask this.
just as it is the wrong thread to answer this.  (;
Re: Synchronizing generative video + audio
Reply #42 - Sep 27th, 2008, 1:45am
Does anybody know if this process would be simplified by taking the draw() Thread out of the picture? For instance, Let's say you tell the PApplet "noLoop()" and then do everything in one call of draw(), treating it like a main() function. Does that make any sense?
Re: Synchronizing generative video + audio
Reply #43 - Sep 27th, 2008, 10:49pm
I have done a little experimenting to find that the program that davebol suggested only works for a framerate of 25 fps. Still can't figure out why.
Re: Synchronizing generative video + audio
Reply #44 - Oct 30th, 2008, 4:11pm
You're likely using the MovieMaker as output?  I've recently found that MM actually creates a 30.3 fps when 30.0 is requested, so sync would fail when merging audio in post.  (bug submitted) I myself don't typically use MM, so never really bothered to check it too closely.  Yes, 25fps works fine with MM, as will 20fps, but not 30fps. (it's a divisor issue i believe)

That general code method for audio sync will work fine at 30.0fps as long as you instead render to still images and assemble to 30.0fps video in post, just avoid MM at 30fps for now.

Pages: 1 2 3 4