Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

All those negative numbers in the TLK

Semi-technical area to discuss content modding research and discoveries. Technical information necessary for coding tools should be posted in Technical Research on the Coders board.

Re: TUTORIAL: Dialogue Editing

Postby giftfish » 09 Sep 2014, 18:28

TankMaster wrote:Well, good thing you mentioned that, I forgot I had to use a small workaround for the entries starting at around 134217729 (the last 4 in the main TLK), and use something like <String id="134217729" alwaysAdd="true">BIOGame</String>. I think a newer version of the TLK editor does that without needing the alwaysAdd attribute, just not in the version I uploaded.

Yeah, those are the same numbers used in all the TLKs, as you probably know.

Right now, ThaneMOD uses 140000000+ for all it's new TLK content, which is what Inquiring originally set up. Originally, she had the male TLK entry located above (rather than at the end) of all out new content to keep all the strings in order. But, when I transitioned to the new DLC TLK I ran into some problems with that. I think in part it was a result of my finding a couple other things that were wrong. Once I "fixed" them, then I was having a problem with "Male" appearing in one of the in-game emails, lol.

So, I changed the male/female strings to new values and that fixed it. My guess is that this could be an issue if you autonumber things, though. It would probably mean modders can't use anything over 134217729.

I'm hoping I can change everything to new values starting at 900K. I still need to finish checking the DLC TLKs to make sure the game stops numbering before that, though. If so, then I'll renumber everything and there won't be an issue with the male/female entries anymore.

TankMaster wrote:As for the rest, let me look at the code some. I just know I replaced a -214 once and it showed up in a subtitle that had no dialog, lol.

That would make sense if you replaced it with the "real" number. That's exactly what would happen if you put in a different string used by the game and text into its data field. The subtitle would appear in whichever dialogue referred to that string, but there won't be any audio/FaceFX, since there's none assigned to the line.
giftfish
 

Re: TUTORIAL: Dialogue Editing

Postby TankMaster » 09 Sep 2014, 18:55

Ok quick post, from what I found, the -214 numbers are the Id of what we'd expect:

Example's:

-2147148449 (#1)
335200 (#2)

#1 actually is -335199.
I came to this by doing the following: Integer's (Int32) MinValue is -2147483647

Int32.MinValue - #1 = -335199
(-2147483647 - -2147148449) = -335199

-335199 is the entry before 335200, just negative. That's why I thought they were decreasing (the negative values). It's possible they did this to be kind of a separator between real strings and placeholders for other things (my guess).

I'll try to explain better if needed. But none the less, the -214 are the right id, just negative.
TankMaster
User
 
Posts: 173
Joined: 02 Nov 2012, 01:19
Location: Indiana
Has thanked: 6 time
Have thanks: 64 time

Re: TUTORIAL: Dialogue Editing

Postby TankMaster » 09 Sep 2014, 19:01

Yeah, the 134217729 is actually a limitation of the tlk format itself. In the binary form, a TLK has 2 lists of strings, though, more often than not, all in the first list (male list). I'd have to look more into the code to see if I can figure out (or remember) why. Now the exact reason they did it this way, who knows. lol
TankMaster
User
 
Posts: 173
Joined: 02 Nov 2012, 01:19
Location: Indiana
Has thanked: 6 time
Have thanks: 64 time

Re: TUTORIAL: Dialogue Editing

Postby giftfish » 09 Sep 2014, 19:21

@TankMaster --

I think we're just thinking of this in different ways, maybe.

-2147148449 might be the "right" ID (as far as BW's placeholder rationale goes) but I doubt you could actually use it in game. To use it in game you would need to change it to what I call the "real" string, which would be 335199. That's the value that you see in the Dialogue Editor, and that a modder could potentially use to insert new dialogue. When the XML/TLK says "-2147148449" there's no way for the modder to know that it means "335199" without some deductive reasoning...or doing some math, apparently, lol. Which also means there is literally no way to know--by looking at the TLK-- what strings the exist in dialogue files and which don't. In short, which strings are used but are not present.

The reason why this sucks is because the TLK has all the game's strings in one place, rather than scattered about in different LOC_INT.pcc files. That makes it the ideal place to "hunt" to see if a string is used. But in the world of Ctrl+F, -2147148499 does not equal 335199, and that's the problem.

However, what you seem to be saying is that we can't change this. BW has set it up this way, and if we changed all entries that look like this:

<string>
<id>-2147148449</id>
<position>4815</position>
<data>-1</data>
</string>

to this (which is much more informative and still correct):
<string>
<id>335199</id>
<position>4815</position>
<data></data> (or <data>-1</data>)
</string>

That this TLK file would no longer work?

--------------------------------
Yeah, the 134217729 is actually a limitation of the tlk format itself. In the binary form, a TLK has 2 lists of strings, though, more often than not, all in the first list (male list). I'd have to look more into the code to see if I can figure out (or remember) why. Now the exact reason they did it this way, who knows. lol

Well, it isn't actually. As I just said, all of ThaneMOD's entries are above this. You just have to change the strings that correspond to male/female. Put in new ones that are higher than whatever new TLK strings you've added and it's fine. I can post a pastebin of our TLK if you want to see it.

It would be useful if the TLK Editor would automatically increase the strings of the male/female entries if the modder did want to use higher numbers than 134217729. Or, if the editor could prompt the user as to what strings should be used for the male/female entries. Either should work.

I've been wondering about the purpose of the male/female thing for ages, lol.
giftfish
 

Re: TUTORIAL: Dialogue Editing

Postby TankMaster » 09 Sep 2014, 19:43

Ah yeah, you're right. I guess I need to start sleeping more, lol. Anyways. I'd have to try it again, but I did once change -2146800909 to 682739 (682378 is "Reset Tali's Powers") and changed it from -1 to "Reset Javik's Powers" and that's what showed up in the subtitles. The place it showed up was in the discussion with Udina in his office after talking with Ashley for the 3rd time. But that was just one instance, I'd have to try others to check, but like you said, finding what maps where is a pain lol.

I have a feeling the male/female thing may have just been leftovers from an early version of the TLK format and they just didn't change it to keep compatibility.
TankMaster
User
 
Posts: 173
Joined: 02 Nov 2012, 01:19
Location: Indiana
Has thanked: 6 time
Have thanks: 64 time

Re: TUTORIAL: Dialogue Editing

Postby giftfish » 09 Sep 2014, 20:20

TankMaster wrote:Ah yeah, you're right. I guess I need to start sleeping more, lol. Anyways. I'd have to try it again, but I did once change -2146800909 to 682739 (682738 is "Reset Tali's Powers") and changed it from -1 to "Reset Javik's Powers" and that's what showed up in the subtitles. The place it showed up was in the discussion with Udina in his office after talking with Ashley for the 3rd time. But that was just one instance, I'd have to try others to check, but like you said, finding what maps where is a pain lol.

That's the exact situation I'm trying to avoid. The whole reason why you chose 682739 is because -2146800909 was in the "right spot" and looked like an unused string. You likely thought that the game didn't use 682739 (b/c you did a Ctrl+F and the XML editor told you it didn't exist!). But, it lies. The game does use it. It's just not obvious b/c the TLK doesn't have it listed. Instead, -2146800909 is in it's place. I had exact the same problem I ran into when working on Thane's dialogue. I was all like "Oh, this string isn't in the TLK I can use it, /la la happydance." And, then I found out that doesn't matter if the string technically isn't in the TLK. What matters is if any LOC_INT.pcc file refers to it. If so, then you can't use it for modding...but good luck with figuring out which (if any) files refer to it. Which is exactly why being able to properly search the TLK for a string is important.

By the looks of it, your problem was here:
Image

But since 682739 isn't anywhere in the TLK, you had no way of knowing it was used somewhere else in game. So, if all those -214 numbers could be replaced with their "real" strings, then other modders wouldn't make the same mistake. Instead, the strings listed in the TLK would appropriately reflect those used in the game files. Seems...logical, lol.

Keep in mind that I assumed this was an error with the tool. If you are saying it MUST be this way, b/c that's how BW formatted the TLK, then that's fine, now I know. But, I've been wondering about it for ages and was hoping it could be fixed.

Think about it (and how to handle those male/female numbers) a bit more and let me know. In the meantime I'm going to stick to the ME3Exp TLK Editor as I test your changes to the Natives Editor. I don't want to implement too many changes at once in case something stops working.

TankMaster wrote:I have a feeling the male/female thing may have just been leftovers from an early version of the TLK format and they just didn't change it to keep compatibility.

I was thinking the same thing. Like for FemShep and BroShep, presumably. I find myself wondering what the ME1 TLK looks like, now that I'm familiar with the format of the ME2/3 TLK, lol.
giftfish
 

Re: TUTORIAL: Dialogue Editing

Postby TankMaster » 09 Sep 2014, 21:20

Well it was partly that and partly I used it to see what would happen. Prior to that test, I was using a high number for Javik's power reset button. Another reason I did it was I knew that position mattered in the TLK from when I was changing how it saved & loaded, which was a long time ago, heh. At that time I did very little with dialog editing and mainly just mapping plot variables & conditions.

Anyways, I am having to look at the code more, been too long so I have forgotten things.

Another thing I noticed, I was wrong about my "alwaysAdd" property, seems I had it in code but it is not actually used. So my apologies on that. If you come into any issues let me know, you'll be more apt to run into those things than me since I have done very little with dialog.

Also, I think the only reason the position mattered at all was because of those few entries at the end, if it was sorted by id and value, Female would be added to the wrong list and likewise for Male, thus causing issues. Granted I am remembering right when I originally tried to completely get rid of position (and use a sorted list)
TankMaster
User
 
Posts: 173
Joined: 02 Nov 2012, 01:19
Location: Indiana
Has thanked: 6 time
Have thanks: 64 time

Re: TUTORIAL: Dialogue Editing

Postby giftfish » 09 Sep 2014, 23:01

TankMaster wrote:Well it was partly that and partly I used it to see what would happen. Prior to that test, I was using a high number for Javik's power reset button. Another reason I did it was I knew that position mattered in the TLK from when I was changing how it saved & loaded, which was a long time ago, heh. At that time I did very little with dialog editing and mainly just mapping plot variables & conditions.

The only rules I've found that matter are:

-String integers don't need to be 100% consecutive (they can "skip"), but they must increase with position. In other words, you can't put a larger number before a smaller number in the list.
-Position integers need to start at 0 and must be consecutive.
-Male and Female entries need to have the same string. They should have different positions.

FYI, this is how ThaneMOD's TLK was structured for v1.0; it replaced the BIOGame_INT.tlk:
Spoiler:
<?xml version="1.0" encoding="utf-8"?>
<tlkFile TLKToolVersion="1.0.0">
...(entries 0-104326)
<string>
<id>134217731</id>
<position>104327</position>
<data>Male</data>
</string>
<string>
<id>140000000</id>
<position>104328</position>
<data>TITLE - THANE HUERTA FRIEND EMAIL</data>
</string>
...(entries 104329-10407)
<string>
<id>140000080</id>
<position>104408</position>
<data>BODY - THANE KRIOS WAR ASSET DESCRIPTION</data>
</string>
<string>
<id>134217731</id>
<position>104409</position>
<data>Female</data>
</string>
</tlkFile>
Despite the "Male' entry coming before the newly added entries, and despite the "Female" entry string being lower than the entry above it, everything worked fine.


This is how it's structured for v2.0. In this case it's a DLC TLK that stands on it's own:
Spoiler:
<?xml version="1.0" encoding="utf-8"?>
<tlkFile TLKToolVersion="1.0.0">
... (entries 0-83)
<string>
<id>738369</id>
<position>84</position>
<data>ThaneMOD</data>
</string>
<string>
<id>140000000</id>
<position>85</position>
<data>*ENCRYPTED MESSAGE*</data>
</string>
... (entries 85-211)
<string>
<id>140000127</id>
<position>212</position>
<data>Male</data>
</string>
<string>
<id>140000127</id>
<position>213</position>
<data>Female</data>
</string>
</tlkFile>
This method also works fine. In this case, I've moved the "Male" entry to the end and assigned it a new string consecutive to the final ThaneMOD content string. The Female entry was assigned the same string, but a consecutive integer for the position.
giftfish
 

Re: TUTORIAL: Dialogue Editing

Postby TankMaster » 10 Sep 2014, 04:38

Yeah, I have always just put my entries before the entry with a value of BIOGame (in the case of the main tlk file) and it works. Also, you are correct about string id's and position's. I never ran into the male/female entry issue as I always put my new entries before id's 134217729. (Unless I misread what you said, lol).

As far as the position value's in the previous TLK editor, it was purely something the original maker of the tool used as a way to quickly compare 2 or more strings with the same id, but different position in the XML. Given what I found with the -214 entries today, I could probably eliminate that all together in the program, since the position is not actually stored in the binary format, just the id and value.

The other thing I wonder is, that among all the TLK files, the last 4 (in the case of the main tlk) are the same among the rest of the tlk's. I figure that those few ID's are actually not meant to change as the game may use them for telling 2 tlk files apart.

Example:
134217729 in BIOGame_INT.tlk is BIOGame, but in DLC_CON_APP01_INT.tlk it is DLC_CON_APP01.
134217730 in BIOGame_INT.tlk is en-US, but in BIOGame_DEU.tlk it is de-de.

Another thing I just noticed in the _DEU.tlk file, it does have entries in the Female list. Identical Id's and almost identical values, except the text has a slight change when it needs the female gender vs the male gender.

Example:
Id 331788, almost identical in the Male & Female lists, only at Line 1, column 773 the male entry uses der, but the female uses die.
"der" being the male form in German, "die" being female.

_INT.tlk is the only set of files that apparently has all entries in the first (male gender) list, since English is one language that doesn't have gender cases like French, German, Spanish, etc.

But as I said, the position value is purely something the original maker of the tool used and does not actually get saved to the tlk itself. Right now, if the male list does not have the id, it adds it there, else it will try to add it to the female list.

This mid section is kinda a quick observation I made while finding some the examples I used too.

But anyways, thanks for help clear some things up for me. I will finish converting the rest of the tlk files tonight, so you have all the locale's (or anyone else that needs them for that matter).
TankMaster
User
 
Posts: 173
Joined: 02 Nov 2012, 01:19
Location: Indiana
Has thanked: 6 time
Have thanks: 64 time

Re: TUTORIAL: Dialogue Editing

Postby giftfish » 10 Sep 2014, 14:53

@TankMaster --

Thanks for the additional info. Also, please don't make too much work for yourself -- I don't need all the localizations. ThaneMOD is only in English and I only use a LOC_INT.tlk. I just needed copies (or information) about the DLC that I don't own/have >.<


Btw, I was thinking about your strategy of dividing into multiple xmls overnight, and I see a problem with it when it comes to practical usage. This problem would be rectified if you would provide the option to keep the entire TLK in one xml. Here's why:
Spoiler:
I've spent a lot of time in the TLK (I'd guess more than most other modders at this point) and what I do more than anything else is Ctrl+F. I use it to locate specific text and strings. Scrolling through a half million lines is extremely inefficient, and using the "Go To" option doesn't help b/c it refers to the program's line label and not the document's string.

Sometimes I'm researching and trying to find all instances of a certain word. "Jacob" for example. Other times I'm trying to locate a line from the gibbed dump that I want to change. Using a small part of the line and Ctrl+F in the TLK is the quickest and most efficient way of finding it. Even if I already know the string from using the DE, I'm still going to Ctrl+F in the TLK just to navigate *to* the string.

These are easy things to do when the TLK is a single file -- they take seconds. But, the more you break it up--into 5, 10, 15 files, or whatever--it means that those steps have to be repeated that many times. It means if I'm looking for all instances of a word, I have to repeat the search in multiple files. Or, if I'm trying to locate a specific string, I need to first identify what "subfile" of the TLK contains that string. It also means if I'm trying to locate a phrase from the gibbed dump, then again, I'll likely need to do searches in multiple documents.

And, since I tend to work on a lot of dialogue, it means I'd always need all the TLK files open, anyway. Having one or two open wouldn't cut it. Not to mention that I tend to keep a dozen or so documents open in Notepad++ at one time. I need to for reference. But...now I'm going to have a 6-12 documents just for the TLK that used to be one document? That's cumbersome to manage in my world, lol.

The point I'm trying to make is that dividing the TLK into multiple files makes it inefficient to use. Dealing with a single file is much more efficient.
The biggest benefit I see in transitioning to your editor is the character fixes and positions. Especially for those with DLC mods, this is a huge help, b/c otherwise you have to number positions by hand. Mistakes can easily be made and your new editor will prevent them completely. But, I'd really appreciate you providing the option to keep everything in one xml.

Like I said previously, Notepad++ does hiccup a bit whenever I open the main game TLK, but after the initial 15s or so (and an attempt at scrolling), it's fine. I've never had it crash (even with a ridiculous amount of documents open) or corrupt any data while saving.

------------------------------------
The other thing I wonder is, that among all the TLK files, the last 4 (in the case of the main tlk) are the same among the rest of the tlk's. I figure that those few ID's are actually not meant to change as the game may use them for telling 2 tlk files apart.

Example:
134217729 in BIOGame_INT.tlk is BIOGame, but in DLC_CON_APP01_INT.tlk it is DLC_CON_APP01.
134217730 in BIOGame_INT.tlk is en-US, but in BIOGame_DEU.tlk it is de-de.
Yes, that's correct. They are the same in the main game and all DLC. Some DLC have ONLY these 4 entries and 1 other. The ones you mention here identify the folder the TLK is in and the language of it. There is also the the male/female, as you know, and there is a string that contains the "Pretty Name" of the DLC. For example, this is the simplest TLK file:

Spoiler:
<?xml version="1.0" encoding="utf-8"?>
<tlkFile TLKToolVersion="1.0.0">
<!--Male entries section begin (ends at position 3)-->
<string>
<id>724024</id>
<position>0</position>
<data>Collector's Edition Online Pass Entitlement</data> (DLC Pretty Name; Mount.dlc file refers to this string)
</string>
<string>
<id>134217729</id>
<position>1</position>
<data>DLC_OnlinePassHidCE</data> (Game folder)
</string>
<string>
<id>134217730</id>
<position>2</position>
<data>en-us</data> (language)
</string>
<string>
<id>134217731</id>
<position>3</position>
<data>Male</data> (male section)
</string>
<!--Male entries section end-->
<!--Female entries section begin (ends at position 4)-->
<string>
<id>134217731</id>
<position>4</position>
<data>Female</data> (female section)
</string>
<!--Female entries section end-->
</tlkFile>

In BW's TLKs, I would not recommend changing the strings for the DLC Name, Game folder, or language. But, I can tell you this:

1. You *can* change the strings for the male/female, but they need to be identical. Safest best is to assign them the highest string in the TLK.

2. The language entry isn't needed for new TLKs. JAM and ThaneMOD's DLC tlks don't have that entry at all (and they actually don't have the Game Folder entry, either). Instead, our new TLKs follow the same name convention as the other TLKs (which use the folder name and language suffix; DLC_CON_THANEMOD_INT.tlk), and they keep the string the Mount.dlc file refers to. These are the important aspects, and what JohnP outlined in his DLC mod tutorial.
giftfish
 

PreviousNext

Return to Modders' Research

Who is online

Users browsing this forum: No registered users and 1 guest

cron
suspicion-preferred