Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

FaceFXAnimSet

Semi-technical area to discuss content modding research and discoveries. Technical information necessary for coding tools should be posted in Technical Research on the Coders board.

Re: FaceFXAnimSet

Postby Deager » 04 Jun 2016, 18:28

giftfish wrote:This would indeed be useful. Either for entire lines or parts of lines. If we could make that happen, it might be a decent enough solution to actually having the ability to autogen new lipsync a la FaceFX.


Well, and to even have the list not collapse would make true lip syncing attempts more feasible. I'm not saying it's easy but I now have a pretty good handle on this I think. This is probably the final version of this line which used to be my least favorite line and now it should be fine because I just shortened what the actors said.



And to do a sum up of high level stuff here.

Unknown 1 is definitely used by Unknown 2

Every ascending number sequence in Unknown 2 is for an object in Unknown 1, m_EE, m_Jaw+, E_GESTURE_whatever...

Unknown 3 is still somewhat that, it relates, but I don't know exactly how

Unknown 2 slots below:
slot 1; time
slot 2; position
slots 3 and 4; technically not proven but I'm almost sure it's either movement along another axis or because they always come in pairs, something with making the movement more emphatic from what I've seen in testing

Gestures still count and must be dealt with elsewhere

I'm sure giftfish and others knew that already about gestures but in this example, by using the Camera Tool, I was checking out Interp 1212 and saw that 452 did Gesture stuff. Sure enough, that was was was causing the head rolls to the left and made Broshep look stupid at the end of his statement by cranking his head left. I just moved that last track time out a few seconds so it never plays.

Then the more subtle movement, like how Broshep was "nodding" made more sense and sure enough, leaving the keyframe in but making it zero for Emphaseis_Head_Pitch solved the nod.

I then had just a touch of work for Femshep since her mouth wasn't quite closing in time, and actually never does in the original animation, so I just had to make sure values from 3.9+ to 4.4 were zeroed out as well to get her to a neutral state. Granted, it's technically a quick cut and I think the mouth may be a touch off but I don't think it's something people should notice.

And all that for a line that, arguably, most people don't pick because I would wager more than 50% of people go Paragon with the Citadel DLC. But the exercise was worth it.

giftfish, you probably have the most questions, if any, of how all this works. Does anybody else? A tutorial is...doable, but very time consuming. If it's only one or two people working on this, I'd almost rather help out directly to show you how it works. Or maybe I'll actually do a quick video where I talk and highlight points. Anybody want that?

I also need to check out how to make a request on github because I might be willing to start working on lipsyncing if the lip wouldn't collapse because converting floats to hex and then making sure I'm in the right place in the file and double checking...ugh. Granted, I am not a programmer so I don't know how difficult that would be. Obviously a full on editor with bezier handles would be amazing but I can't imagine how difficult that must be to put a GUI on something like this! Especially since I still haven't fully identified everything in Unknown 2 and 3.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby giftfish » 04 Jun 2016, 21:21

@Deager -- Well, I have no context for the scene, but the lipsync looks perfect. Certainly as good as BW's -- which are almost never perfect, themselves.

I would be hugely interested in a tutorial. If making a YT vid is easiest, then I'd say go for it. You can still make a thread here and just embed it, and it would be easy to embed on the wiki, as well.

From what you say above, it seems in this particular example you were just needing to cut off the sync early? Not match it to new words. Is that correct?
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: FaceFXAnimSet

Postby Deager » 04 Jun 2016, 21:26

giftfish wrote:From what you say above, it seems in this particular example you were just needing to cut off the sync early? Not match it to new words. Is that correct?


Right. This was a very easy edit. I just had to stop the mouth from moving and stop odd animations from happening to shorten the line. So the lip sync in this case was already correct and I still think trying to lip sync from scratch will be brutal but it is technically possible; especially if the number of syllables is really close.

I may slap up a YT video or maybe start a tutorial which will be massively edited. I'll try to get it all in order for main topics. There's a lot going on, kind of at once, although maybe if I break it down it won't be so bad.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby giftfish » 04 Jun 2016, 21:36

@Deager -- Forgot to mention that *sometimes* even larger head/neck moves are indeed part of the FXA. One I know off the top of my head that is, is Ashley's "Yeah" neck snap gesture when having the final conversation in Huerta. Her rubberneck there looks awful, but it's not controlled by a gesture track, which means that FXA is the only option.

Much of the time, these sorts of gestures are controlled in the matinee, though, so always good to give it a quick check, first.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: FaceFXAnimSet

Postby Deager » 04 Jun 2016, 23:08

Yeah, they keep it fun. I was going to record a youtube video but right now my mic is screwed up so no go. I actually have started looking into actually editing a lip sync to match a new sound file. Keeping it easy, I'm looking at a simple one from ME3 that originally was, "Well, nobody's perfect," and the new line in the new file is "Nobody's perfect."

Lots of issues but doable. First off, to avoid massive hex editing...which is an option I suppose if the FXA stuff was moved to the end of the file so that hex could be added at will, this is what you need.

1. Unknown List 1 must have the same number of objects.

2. The Names list found under the Header tab must have the expressions you need. I'm guessing they'll usually be there.

3. The amount of items in Unknown List 2 probably have to be similar. In my case the original had 137 and the line I'm going to attempt has 99. But, the original line had one more syllable.

4. Try not to change the line.

Here's where I'm going to get into trouble because by removing the "Well," I now have to figure out all the timing changes for the shortened line since it's shortened from the front, not the end. I have all the animation movements ready to go but I still have to struggle with timing.

5. You have to be insane to try this.

So, for the mouth movement alone, which is all I want as I don't want to screw around with head gestures since those I want to keep with the existing scene, I have to track 14 mouth things. I also, because the timing changed, need to confirm that the number of "keys" is enough for each mouth setting. It can be more, as I can always add in dummy times with no keys but if it's less, I'm screwed. And I'm sure there are other issues I'm not even seeing yet.

I'm going to try the process with this short Kasumi line but my gut tells me not many are going to be up for this and I don't even know if I can pull it off. And...if its a line from ME2 or ME1 then there's no real reference. I'm sure if someone got really good they could start looking at words for any character and get an idea of the general settings for the mouth position but we're talking a ton of time to figure that out and test it for every line.

For the team...I'll let you know how it goes. After that I can maybe get a better mic and do a quick recording to show how this all works. Right now assume that only shortening lines is possible. Do not think lip syncs are feasible right now. It's kind of like I could probably find a needle in a haystack but is it really worth the effort to find that needle?
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby giftfish » 04 Jun 2016, 23:24

@Deager -- Yeah, it sounds too labor intensive to be feasible right now. We're going to need a better solution in the long term :/

Thanks for all the time you've spent looking into this :)
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: FaceFXAnimSet

Postby Deager » 05 Jun 2016, 00:33

giftfish wrote:@Deager -- Yeah, it sounds too labor intensive to be feasible right now. We're going to need a better solution in the long term :/


Granted, I'm still learning, but my first failed attempt took 90 minutes on the line, "Nobody's perfect." However, I have a feeling that once I know the right way, a line that short would take 90 minutes. So...depending on how important the line is, how the close the camera is, will a player always hear it or it that about 50% will see it...there are potentially some uses for this. Of course, I still have to get it to work to even see if it's possible.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby Deager » 05 Jun 2016, 01:35

I had one other method to try and no go. Plus, I already knew that the responses from the same character, in this case it's Kasumi, are different for male and female Sheps but also, the exact same line has a different number for all three unknowns.

As a side note, it may be possible that while the list goes in ascending order and resets for new...things, that we may not be able to just willy-nilly change the length of those. If m_EE has 9 lines then it seems that it must retain 9 lines. If this is correct, it's further evidence that lip syncs are just not going to happen.

Still, when you need to shorten a line but hold a camera shot for some reason, in my case it'll be for the Shep lines, it's handy to know how this works.

If I come up with any bright ideas I'll let you all know. At least we have some understanding of what is possible and I'm actually still pretty excited since I'm going through CEM and starting from a different base than I did in July of 2013. Already I'm getting lots of improvement my snagging animations from other, unused lines to make lip syncs better.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby CreeperLava » 05 Jun 2016, 07:58

If this was to be automated, it would probably be a good idea to use something like an automatoc lip sync, to at least avoid that trouble. I'm pretty sure you can find some of these on the internet.
User avatar
CreeperLava
User
 
Posts: 844
Joined: 07 Feb 2015, 21:52
Has thanked: 119 time
Have thanks: 83 time

Re: FaceFXAnimSet

Postby giftfish » 05 Jun 2016, 13:15

CreeperLava wrote:If this was to be automated, it would probably be a good idea to use something like an automatoc lip sync, to at least avoid that trouble. I'm pretty sure you can find some of these on the internet.

Nope, and that's precisely what FaceFX does. It autogenerates both lipsync and facial expressions. Then the animators come along and tweak things. It's included in UDK, though, so if we *could* get the necessary game data to export and import on both ends properly, then we'd be mostly in business.

Outside of the obvious difficulties in technical implementation, the other challenging aspect of that would be if you wanted to keep all the other facial expressions *except* for the lipsync. I don't know if it would be possible to do that or not in that scenario.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

PreviousNext

Return to Modders' Research

Who is online

Users browsing this forum: No registered users and 0 guests

suspicion-preferred