Question about animation and sound..

I have trouble mathcing speech sound to animated character portraits, not sure if it's me or the computer messing things up. This is what I do:

I think one of new features Simon mentioned is that you can now overlay videos onto scenes, but I think it's only currently possible with Lua script & I don't currently have a clue how it works. Simon said he will upload some scripts/examples when he has the time to sort some out.

As for your question about timing. Yes I believe it's possible that the time can be affected by the machine running the game as the system loop time is inconsistent. It doesn't loop every x (absolute) ms, but every random x ms instead, so it's possible each new frame isn't being shown bang on 67ms.

Burf, that's sad. 

Burf, that's sad. 

But hopefully a feature that can be implemented as soos as Simon has a script sample. I do not have a clue on coding

I know most people just loop some simple mouth movements in these cases, but I want a bit better lipsync and that's why it has to match up.

Thanks for the reply, I will keep my eyes open for the new feature

Based on your first post, I'm guessing what you are currently doing is creating a unique animation for each dialog? Sounds like a lot of work to me.

You could opt for simpler Hanna Barbera style talk animation. It won't look as nice as using millions of frames to generate all the different kinds of mouth movement, but for 2D I don't think it really matters that much. Basically you create about 6 or 7 or 10 (I forget the actual amount) of mouth shapes & then you use timings to try & pick the closest mouth shape to the word being spoken. You also force idle animation during pauses between words/paragraphs or Oh/Ahh mouth shapes if one of those kind of words were spoken & the time until the next word is not very long.

http://www.angryanimator.com/word/2010/11/26/tutorial-3-dialog/

There's another method too, which I call faux lip-syncing. It's kind of like Hanna Barbera method except you have it play the talk animation (set to random frame) then you force idle or oh/ah animation during silent parts of the currently playing speech audio file. It wasn't perfect, but it looked nicer than having character talk animation playing the entire duration of the speech audio file.

Anyway you go about it though will require a lot of hard work & patience.

P.S: Daedalic lip sync their games, but they use some kind of application to generate phoneme data then they create Lua tables & use a script/loop or something & probably text ID's to force which talk animation frames should be displayed & when - probably not perfect either due to the time thing we both mentioned, but I just figured I would mention it.

I know I have tried the Hanna Barbera style and it would have worked fine on more stylized or pixelized graphics, but looked very paper doll-ish on 3d characters.

I wonder if some lip sync feature could be added to support lip syncing of 3D character models, then again, that wouldn't cover facial expressions. I like the subtle details you are talking about & I think they work well for close-up talking scenes, but general talk animations of the regular sized character in scenes I don't think matters all that much.

Maybe you could get away with the video thing when Simon posts a script/example for it... though I only imagine that would be useful for when character is in a set position when they are meant to talk otherwise it would be hard to line things up correctly.

I agree. It would be perfect for these "dialogue portraits" though, and maybe Tvs and such in the background.