Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

[Tutorial] Using FaceFXAnimSet Editor

Made by modders for modders. Step-by-step instructions on how to achieve specific modding goals with ME3Explorer. We *strongly* encourage you to post new tutorials on the wiki.

[Tutorial] Using FaceFXAnimSet Editor

Postby Deager » 13 Jun 2016, 22:38

How to use FaceFXAnimSet Editor to (somewhat) control lip movement. This is what I know so far. I will change it if and when I learn more.

First, definitely grab at least me3explorer version 2.0.7. Huge improvement to the tool which allows changed to be made in a list without it collapsing on you every time. Check the upper right hand corner of me3explorer. At the time of this tutorial, 2.0.8 is already out.
http://me3explorer.freeforums.org/new-releases-f26.html
Spoiler:
Image


This will be a bit of overkill for the skill level of people likely to venture into this but I'll try to be cover most things.
Open up FaceFXAnimSet Editor from Developer Tools -> Dialogue -> FaceFXAnimSet Editor
Spoiler:
Image


We're working with the language files to make these edits. In this example, I want the English version of a Citadel DLC file, namely, BioD_CitApt_Thane_LOC_INT.pcc. Go to File -> Open and open that file if you want to follow along.
Spoiler:
Image


Now, here we have a list of fun things. We really only need to work with 2 or 4 things. If you want to change a line Shepard says, you'll work with objects 1172 and 1173. Respectively, citapt_kolyat_m_D.FXA_citapt_kolyat_m_Player_F and citapt_kolyat_m_D.FXA_citapt_kolyat_m_Player_M. If you want to work with Kolyat's lines, you guessed it, objects 1170 and 1171, citapt_kolyat_m_D.FXA_citapt_kolyat_m_Owner_F and citapt_kolyat_m_D.FXA_citapt_kolyat_m_Owner_M. Likewise, objects 1175 and 1176 for Thane. In this example, while 1177 and 1178 would be for anything Shep might say during the logs, nothing is said so virtually no data is there. I'm not sure why it's even listed but it is. Probably some UDK engine thing I completely don't understand.
Spoiler:
Image


OK, let's look at what's going on a little bit from the little I know so far. Under the Header tab -> Names, you'll see a list of things that look like animation type stuff. m_* would seem to be mouth movement. Number 18 is Orientation_Head_Pitch so it probably does as expected, and so on. We'll come back to this list in a bit.
Spoiler:
Image


On to where the magic really happens. Data tab -> Entries.
Spoiler:
Image


Which when expanded gives a long list. I'm working with shortening the line for Shepards for line 782905, which is item 21 on the list. Expanding only #21 shows this info.
Spoiler:
Image


That info above is fairly straight forward for people familiar with editing. If it's not familiar I'll let you know that we are dealing with line 782905, which is "I don't want to discourage you, but the Council doesn't move too quickly, especially when distracted by the war." The way to confirm this is to extract the tlk from the game and search for that string of 782905 to make sure you have the correct line you want to edit.

Side note: Unk3 seems to always go up in value one tick for all items on the Entries list. I have no idea if that's significant or not.

Let's move on to some stuff that matters a bit more. Starting with Unkown List 1.
You'll notice that the first column is in hex, which, as expected, matches the decimal values of names under the Header tab -> Names and is also conveniently listed here for us so we know what's what.
Spoiler:
Image


Skipping ahead to Unkown List 3, we see a bunch of hex. Convert those values to decimal and once again you'll see them match up fairly well with Header tab -> Names info. Sort of though. At this point, Unkown List 3 is largely that; unknown. For now we'll just skip it. I've done some odd things with it but I don't get it yet.
Spoiler:
Image


Finally, the meat of this tutorial, Unkown List 2. Here is what I really do know.

The decimal numbers in column 1, go up in value and then will start again and go up again. This happens the exact number of times there are objects in Unkown List 1 and Unkown List 3.

Side note: Unkown List 1 and Unkown List 3 always seem to have the same number of items.

It turns out, column 1 is full of times. The times are key frames. Notice how you only see column 1 with a time and then followed by zeroes never more than twice in a row? That's kind of how we know they're key frames.

Column 2 is positions of objects in time.

Columns 3 and 4 always seem to be paired up; either with zeroes or non-zeroes, but I have yet to see one with zeroes and one with any other value. At this time I do not know exactly what columns 3 and 4 represent. I suspect they are exaggerated positions of the current animation.

The list of names from Unkown List 1 are in the exact order that the times go up and reset in Unkown List 2. For example, in the screenshot below, lines 0 through 26 are for m_EE. From 27 to, well, the screen doesn't show it, but to line 53, is for m_Jaw+, and so on.

Spoiler:
Image


So how exactly do we use this information to do anything worthwhile!? I see two uses right now. The first is to shorten lines. The second is to cut lines and make animations happen earlier, or later.

In this example, I wanted to have Shepard just say, "I don't want to discourage you, but the Council doesn't move too quickly." But Broshep says the line too fast compared to Femshep. The only way to use that shortened line for both Sheps was to hold Broshep's mouth shut for a second longer so his lips wouldn't move to allow Femshep to have time to finish her line. This is why you see awkwardly long pauses with "your" Shepard at times because one of the actors outpaced the other and the game only has InterpLengths for both Sheps together.

By taking the original audio and, in my editor of choice, Audacity, I could see when Broshep took his brief pause between "quickly" and "especially." Once I knew the time, I then had to go through and for all mouth related animations which is items 0 through 14, I simply would put ; 0 ; 0 ; 0 after the later times for all 15 items. In other words, starting at 2.6 seconds I knew it would have to keep the animations from happening. So I took line 94 and made it 2.702585 ; 0 ; 0 ; 0. And I repeated this process for all animations after 2.6 seconds for Broshep.

Even Femshep needing a little tweaking to leave enough of a pause to make the camera cut timed like most others are in the game.

Now the second idea I have; make animations happen earlier. This still relates to shortening lines as well. Kolyat had a line which was, "I appreciate the offer, but Councilor Valern has recently orated about funding science during wartime. My odds are good." I wanted him to instead say, "I appreciate the offer, but Councilor Valern has recently orated about funding science. My odds are good."

Same process as before. Look for when the words "during wartime" are said. Then go through all his mouth related animations and make them <time/key frame> ; 0 ; 0; 0. And to make it not have too long of a pause, I then edited the sound to have "My odds are good" happen earlier. Luckily one second exactly worked well so it made taking those animations for that phrase a lot easier to do the maths.

Here are some final thoughts. I think the we can't change the number of items per animation. If the lines 0 through 26 are for m_EE, then we're stuck with those 27 lines being for m_EE. This is why taking animations from other parts of the game is so difficult. Even if it's from ME3 and we can match the Unkown List 1 and Unkown List 3 with a hex editor, we're still stuck with also needing to match the number of items or getting close in Unkown List 2. The other problem with lines from other files is I still don't understand Unkown List 3. I think I can even match that list in hex but when I tried to make Kasumi say, "Nobody's perfect" instead of "We get shore leave" it had hilarious results.

Another fun part of this is that even if the exact same line is spoken by the NPC, the timings are actually a little different for *Owner_M and *Owner_F lines. So in this case Kolyat's response had to be done twice which is just kind of annoying but it is what it is.

I certainly could be forgetting some stuff but I at least wanted to get some tutorial out there for people who want to tackle it. I may be missing stuff and there's still stuff to learn but hopefully this can get people started if they'd like to try.

Deager has been thanked by:
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby giftfish » 22 Jun 2016, 19:11

Finally got around to reading this Deager. Thanks again for all the work :)

One thing I wonder, is if the new Curve editor would be helpful for this.

Here's the best web resources I can find on UDK/BW and FaceFX (there's a couple YT tutorials, but I'm not including those):

http://social.bioware.com/wiki/datoolse ... php/FaceFX
https://udn.epicgames.com/Three/Introdu ... aceFX.html
https://udn.epicgames.com/Three/FaceFXP ... kflow.html
http://www.lagspike.com/facefx-tutorial/

One thing that's clear from these -- especially the DAO wiki page -- is once FaceFX autogenerates the lipsync and emotional expression, a curve editor is used to display the data and allow refinement. Each bone has it's own curve, and I'm pretty sure that's the we're looking at in the FaceFXAnimset editor.

My hunch is that Sir would just need to code up a translator like for the camera stuff.

@Sir, can you comment on that at all?

Also, I have some other info and questions on this, but I'm going to be posting it on the TM forum, Deager. It's more tangential, and more related to what, if any, possibilities we have to work on this stuff in UDK.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: FaceFXAnimSet

Postby SirCxyrtyx » 22 Jun 2016, 22:34

Curves huh? I'm pretty sure that the numbers in Column 3 and 4 are tangents then. I think that's the final piece of the puzzle. Thanks for your hard work figuring this out Deager, I think it should be possible to make a much easier to use FaceFXAnimsetEditor now. Can't say when that will happen though. Hopefully relatively soon.

SirCxyrtyx has been thanked by:
User avatar
SirCxyrtyx
Toolset Lead
 
Posts: 345
Joined: 16 Apr 2014, 00:20
Has thanked: 28 time
Have thanks: 288 time

Re: FaceFXAnimSet

Postby giftfish » 22 Jun 2016, 22:42

SirCxyrtyx wrote:Curves huh? I'm pretty sure that the numbers in Column 3 and 4 are tangents then. I think that's the final piece of the puzzle. Thanks for your hard work figuring this out Deager, I think it should be possible to make a much easier to use FaceFXAnimsetEditor now. Can't say when that will happen though. Hopefully relatively soon.

Wewwwwwwwwt!
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: FaceFXAnimSet

Postby Deager » 22 Jun 2016, 23:41

SirCxyrtyx wrote:Curves huh? I'm pretty sure that the numbers in Column 3 and 4 are tangents then. I think that's the final piece of the puzzle. Thanks for your hard work figuring this out Deager, I think it should be possible to make a much easier to use FaceFXAnimsetEditor now. Can't say when that will happen though. Hopefully relatively soon.


Nice. This video is already outdated but I'm sharing it anyway. At the very least so maybe I can learn how to pronounce your name.

User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby SirCxyrtyx » 25 Jun 2016, 18:20

I've figured out list 3. It's the number of keys for each animation. If you sum up all the values in list 3, it's always equal to the length of list 2.

Image
For example, in the above image, the first 0x45(69) keys make up m_EE, the next 0x4E(78) keys make up m_Jaw+, and so on. Animations with 0 keys, such as m_TH and m_Flap obviously don't play. This means that editing list 3 manually is a recipe for disaster, as any change will completely throw off all following animations.

SirCxyrtyx has been thanked by:
User avatar
SirCxyrtyx
Toolset Lead
 
Posts: 345
Joined: 16 Apr 2014, 00:20
Has thanked: 28 time
Have thanks: 288 time

Re: FaceFXAnimSet

Postby giftfish » 25 Jun 2016, 20:18

@Sir -- Ha!!

And my hypothesis about categories/links is rejected. Nice. I was right about the "0" at least :D

So, is list 3 basically autogenerated, then? Seems weird that it's there at all if it's just a keyframe count.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: FaceFXAnimSet

Postby SirCxyrtyx » 25 Jun 2016, 22:42

Well, the keyframe count needs to be stored somewhere. But it does seem an odd setup to store what is essentially an <int,array> Dictionary as an array of keys, an array of all array values, and then an array of array lengths.
User avatar
SirCxyrtyx
Toolset Lead
 
Posts: 345
Joined: 16 Apr 2014, 00:20
Has thanked: 28 time
Have thanks: 288 time

Re: FaceFXAnimSet

Postby Deager » 26 Jun 2016, 02:27

Nice! That finally makes sense since gift and I were still kind of scratching our heads on that one.

OK...oh my. Well, I'm not going to get into this but now I would know how to move the FaceFX object to the end of a file, point the header info and size changes, and then do whatever I want with keyframes. I'm not exactly saying I want to do that of course, just that it's possible.

But knowing you Sir, I have a feeling you're going to give us an editor which can do all that stuff.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time


Return to Modders' Tutorials

Who is online

Users browsing this forum: No registered users and 0 guests

suspicion-preferred