Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

FaceFXAnimSet

Semi-technical area to discuss content modding research and discoveries. Technical information necessary for coding tools should be posted in Technical Research on the Coders board.

FaceFXAnimSet

Postby Deager » 30 May 2016, 07:20

It's been bugging me for a while that the animations are there, in a lot of hex, and I figured there has to be some way to work with it for lip syncs. I don't know if it'll ever happen but here's what I've found, which is probably not new to the programmers out there. Also, I'm not sure if this has already been researched and I'm just missing the tutorial on it.

I'm specifically looking at the BioD_CitApt_Thane_LOC_INT.pcc file, in the FaceFXAnimSet Editor at object 1173, citapt_kolyat_m_D.FXA_citapt_kolyat_m_Player_M and under the Data tab, at number 21, ID(782905).

The unknown lists appear to be this.

Unknown 1 controls animations from lip movement and probably head movement.

Unknown 3 relates to Unknown 1 with similarly named things and also they always seem to have the same number of things. I can see that Unknown 3 is the hex for the information under the Header tab, Names, which are shown in decimal.

However, Unknown 1 seems to always have unique names whereas Unknown 3 will repeat things, like most conversations ending with m_EE or m_Jaw+

Unknown 2 is pretty crazy long and my hunch is, it's responsible for calling the timing of the animations.

Here are my tests so far.

If I set Unknown 1 to all being 04 or m_Jaw+, Shep opens his mouth and kind of holds it there. Animations for the head, face, keep happening and, the edges of the mouth move a little...trying to do the lip sync but with the center of the mouth being static.

If I set Unknown 3 to all being 03 or m_EE, then the mouth stays closed entirely. But, the head and other face animations still happen.

Not surprisingly if I change a few of the items in Unknown 3 by removing some lip looking stuff, the lips still move but are off.

At this point I know the game is calling for these 30 things via time and that probably relates to Unknown 2. Here's the really annoying bit in the testing.

If I zero everything out, the face and head do their thing like normal and no lip movement. Cool, until.

If I zero everything out from line 20 to 456, no lip movement but everything else seemed fine.

Zero everything out from 100 to 456 and the animation seems to start for one syllable. Then the mouth is stuck closed but the edges of the mouth are trying to do the lip animations from that state.

Zero out from 383 to 456 and everything seems to play as if nothing has changed from the original scene. However, that sort of makes sense because Broshep has his mouth closed for a while at the end of the scene since he says the line faster than Femshep. And so far tweaking these only seems to affect mouth movement.

Now, there may be no way to decipher what the pattern is here. I don't think it's news that the game is calling from Unknown 1 and Unknown 3. I don't know why they each can seem to shut down lip animation entirely or quite a bit...they have some sort of relationship I don't get. I'm pretty sure Unknown 2 is the time of things happening since it's a bunch of decimal numbers (floats in this case if I have my terms right) and they are all quite unique. You can practically search for a number and only find it once in the entire file. UPDATE: Not entirely true, but still, usually the numbers are quite unique /UPDATE

I can keep brute force testing this to see if I can find anything else out. It's difficult to know that it seems like it's close to being cracked and at the same time there doesn't seem to be a great pattern. A bunch of random looking numbers in Unknown 2 is really tough to figure out for me.

UPDATE: Well, I can at least tell this. It appears that all Unknown 2 things have a number in the first spot and then sometimes in the second, and if any more than that, it's always 4 spots, never just 3. I knew that already so I decided to replace all the first spots with zeros and leave the other numbers in place. Sure enough, no animation played for a much shorter line...it's possible that even the head/fact animations didn't fire either but it's 3:50AM and my testing could be getting sloppy.

Anyway, the first set of the numbers are what seem to do the main call to the animation. I'm guessing the other numbers fine tune it. I still would like to figure out how they're linked so it would be possible to edit, even if it's just in hex.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby giftfish » 30 May 2016, 13:47

Yikes. Thanks for the research, Deager.

I know there's a fxA research thread on the TM forums, but not too much in it if memory serves. What I do know is that FaceFX autogenerates the sync and facial expression via the text/audio and bone definitions. So, for a lot of lines, BW doesn't do anything but refine it just a bit and maybe add a small head or neck movement here and there. So, it kills me that it's so impossible for us to edit. Even if you can get the FXA exported, according to the description on FaceFX website, the plugins for Maya/etc don't allow generating of new FXA.

One thing I've been wondering about and that's that UDK includes FaceFX (http://udn.epicgames.com/Three/FaceFX.html). Therefore, if we could get the proper game data exported from the toolset, compatible with UDK, and then opened in UDK, we should be able to edit/autogenerate new FXA. I think. Then, of course, get it re-imported into the toolset. I'm not certain where/if a 3D modeling programs needs to fit in there, or if UDK can do what we'd need it to on its own. Only blender is free as far as modeling/animation programs go, and from what I know, there's no FXA plugin for it -- only for Maya and Max... and maybe one more. I forget.

@Sir -- have you looked into this at any point?

giftfish has been thanked by:
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: FaceFXAnimSet

Postby Deager » 30 May 2016, 15:05

If @Sir can get a handle on it that'd be very, very cool.

I should add that I also was looking for any type of pattern between unknown 2's total count and anything else. I didn't see it.

My gut tells me that it's some type of setup where the stuff in hex is not at all linear in time but instead, what are they, arrays? And the trick will be finding the pattern for how the game is calling for the various animations. All I want right now is to shorten a few lines for Shepard but in these cases one Shep is still jabbering on the first part of a line while the other Shep has moved onto the second part of the line so instead I had to use more dialogue to replace it and they're just not good replacements to me.

Plus, I was getting bored and I figured it was time to try to do something impossible to me.

UPDATE: I can also add that if I zero out the bytes in the 2nd, 3rd, and 4th spots but leave the first set there, that also completely breaks the animation. I think that means that any Unknown 2 thing is the full package to make that line work.

I've switched to working with line 19 since it's much shorter "He was one of us," and with 126 lines it's much more manageable to do testing; just enough syllables to see patterns but short enough to not go crazy testing. I'll keep looking into it until I read otherwise.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby sansuni » 31 May 2016, 07:55

Nice research..

I tried to do a similar thing for my mod in the past (with utter failure.) I used the method of hex cloning (I copied from a point where I believed Shepard's mouth moved to a place where I believed it didn't) but best I could menage looked like a talking horse effect. (where Shepard's mouth moved very randomly) In the end I messed it up so badly that animation didn't even play anymore. It would be nice to have a tool for this, but I imagine it would be a bit difficult.
sansuni
Modder
 
Posts: 157
Joined: 18 Jan 2014, 09:49
Has thanked: 12 time
Have thanks: 7 time

Re: FaceFXAnimSet

Postby Deager » 01 Jun 2016, 15:44

Well, given the lack of activity on this thread, and knowing that others have looked into this, my hopes are not high. The good news is, I don't care, and I will continue to try and find a pattern for that #19 line since 126 lines of Unknown 2 is doable enough that I can try a few patterns out and see if I can figure out how it works.

Whatever I test and fail I'll list here so if others try to pick it up later they can. Hopefully I'll luck out and figure out kind of how it works.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby CreeperLava » 01 Jun 2016, 16:29

Just wanted you to know that I'm watching the thread with interest, even if I don't participate (mostly because of lack of knowledge^^).
User avatar
CreeperLava
User
 
Posts: 844
Joined: 07 Feb 2015, 21:52
Has thanked: 119 time
Have thanks: 83 time

Re: FaceFXAnimSet

Postby FemShep » 01 Jun 2016, 21:39

Same here. Don't know anything but still interested.
Image
ME3Tweaks has modding guides, tools, forums for mods, a modding wiki, and ModMaker, an online mod creation tool.
ME3 Mod Manager, the civilized way of installing and managing ME3 mods.
ME3Tweaks Facebook Page
User avatar
FemShep
Modder
 
Posts: 1101
Joined: 18 Oct 2012, 20:48
Has thanked: 42 time
Have thanks: 76 time

Re: FaceFXAnimSet

Postby Deager » 01 Jun 2016, 22:36

Well, hopefully I'm not going all "Beautiful Mind" here, but I maybe saw a pattern.

It appears that the first set of bytes per line in Unknown 2 always go up in value, then reset to whatever number and go up again. The total amount of times this occurs is close to the number of total things listed in Unknown 1 and 3...and something just hit me as I typed. Unknown 3 is the one with repeating values....I'm going to see if the number of times Unknown 2 goes up matches the number of unique items in Unknown 3. At the very least, I finally see a pattern and I'll do a lot of testing to see if each "ramp up" is perhaps controlling a specific part of the animation. Stay tuned for more bizarre ideas and results.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby Deager » 02 Jun 2016, 00:27

OK, I think I know the pattern but I have to go through the videos. My hunch is the Unknown 1 stuff is what's getting used in Unknown 2. And I consistently see fewer ascending number patterns than total things in Unknown 1, but I suspect some are just extra animations which are not called and essentially garbage code.

I'll post results in a video soon but it appears the first thing to start "failing" is the lip animation slowly over each ascending number sequence I remove and sure enough, on Unknown 1, those are things like m_EE, m_Jaw+, m_EH, etc. And at the very end is when the head finally stopped moving, which is stuff like E_GESTURE_HeadRight, etc.

So, I think the floats or whatever the numbers are called are indeed time calls for the animations. It's referencing Unknown 1. I have no real clue what Unknown 3 is for exactly although it seems when one Shep finishes a line faster than another, the faster Shep has quite a few repeating values in Unknown 3 which seem to be keeping the mouth closed.

Stay tuned. We may be able to crack this a bit. Still...all I think that may be possible is just holding lip syncs up so I can truncate lines for both Sheps so that the faster Shep can stop animating. This will free me up massively. It's kind of a crap situation right now for ThaneMOD and basically all other mods. Technically it's probably possible to manually control the lip movements in hex by changing the timing using the Unknown 2 values but I would not envy anyone trying to do new lipsyncs with that.

However, maybe this will be enough information that instead of Unknown lists we can have more options with the tool...I know editors cannot be easy to make but if we can understand the relationships, maybe there's a chance.

UPDATE: Here's what I'm looking at now.
User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Re: FaceFXAnimSet

Postby Deager » 02 Jun 2016, 01:48

Testing is done for now. Zeroing out at the end of stuff is not a good idea. So, I tried to put a time waaaay in the future. Actually, this is what I was hoping would happen. What I'm trying to figure out now is how this all really works. I'm starting to think that the ascending sets of numbers are following things which must happen. Like, there's no way to skip them. However, I still haven't screwed around by throwing in negative numbers when I want to skip something and then moving the file one or two time slots to an early time and hoping they hold. That'll be my next test I think but I'm at the point I need to sleep on this and let my subconscious mull it over.

EDIT: I do think trying to figure out why some Unknown 2 entries have 1 number, 2, or 4 needs to be figured it out. And I still need to figure out if it's possible to skip animations or if all we can do is change the timing, which in the test below would indicate that's not really helpful. /EDIT

EDIT2: Seriously I need to stop. But something hit me. I do see the same number occasionally in Unknown 2. Well, when I could each of those as a separate whatever, I do get a matching number to the number of things in Unknown 1 and 3. So, 2.5, 3.5, 3.5, 3.5, 1.0 is ultimately 4 objects/things it appears. So, now I have to confirm that it's really pulling straight from Unknown 1 for the order and once I know that, then I have to see if I really can control the timing in any reasonable way. /EDIT2

User avatar
Deager
Modder
 
Posts: 805
Joined: 16 Feb 2013, 01:37
Has thanked: 284 time
Have thanks: 164 time

Next

Return to Modders' Research

Who is online

Users browsing this forum: No registered users and 0 guests

suspicion-preferred