Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

Tutorial : Codex Editing

Made by modders for modders. Step-by-step instructions on how to achieve specific modding goals with ME3Explorer. We *strongly* encourage you to post new tutorials on the wiki.

Tutorial : Codex Editing

Postby giftfish » 10 Sep 2013, 20:20

TUTORIAL | ADDING A NEW CODEX ENTRY


NOTICE: December 9, 2014
The tutorial below highlights use of the ME3Explorer Codex Editor and has been amended to reflect new understanding of codex file structure. It's important to realize this editor cannot add new codices to the game, but can only replace codices (some ME3 data is actually for ME1 & ME2 codices and can safely be re-purposed). Below the original tutorial, I've also added some information on TankMaster's new Natives Editor which actually allows the addition of new codices, categories, etc.


How to Read the New Editor
There are 426 rows of data in the Codex Editor. Each row consists of a sequence of 11 variables separated by a space. Each variable is a different piece of data that goes into creating the entry. Rows 0-262 consist of section/category names and entries for ME1 and ME2. Rows 263-343 are ME3 primary codex entries. Rows 344-426 are ME3 secondary codex entries. When starting a new game, there are 18 primary entries and 34 secondary entries for a total of 52 entries in the codex.

I'm going to use rows 293 and 294 as an example to explain what's in the editor. These correspond to Anderson's codex entries.
Spoiler:
Image
position 1 -- 293 -- Row number (display organization; does not affect entry)
position 2 -- @0x000024B4 -- Hex Offset
position 3 -- : -- Separator (display organization; does not affect entry)
position 4 --20759 -- PlotStateID (plot bool)
position 5 -- 4 --Instance Version (4 = ME3)
position 6 -- 678430 -- Title String ID
position 7 -- 678429 -- Description String ID (no description = -1 or 0)
position 8 -- 3 -- Texture Index (nID; points to list in coalesced)
position 9 -- 2 -- Sort Priority (ranges 0-5; 3 is a "median" of sorts)
position 10 -- -15 -- Sound Property (all non-zero integers are negative)
position 11 -- 58251 -- Section/IsPrimary (this one is "Known Associates")

A couple comments on the Texture Index.
  • This field is an easy way to spot primary codex entries since they all have custom images
  • Values 1-89 correspond to primary codex entries
  • Values of -1 and 0 indicate no image
  • Most secondary codex entries = 0 since they don't have a custom image

Here's how the Sort Priority works when it comes to displaying entries:
  • values of 0 indicate codex section headers (Known Associates)
  • a value of 3 seems to be a median value; it's also the value most secondary codex entries use
  • values of 1 or 2 places the entry nearer the top
  • values of 4 or 5 places the entry nearer the bottom
  • entries assigned the same values will appear alphabetically.

For example, in the "Known Associates" section, here are how a selection of names appear (their respective numbers included):
  • Steven Hackett=1
  • David Anderson=2
  • Dr. Liara T'soni=3
  • EDI=3
  • Garrus Vakarian=3
  • Kaidan Alenko/Ashley Williams=3
  • Tali'Zorah=3
  • Dr. Mordin Solus=4
  • Jacob Taylor=4
  • Jeff "Joker" Moreau=4
  • Miranda Lawson=4
  • Dr. Karin Chakwas=5

Non-zero integers for Sound Property are all unique since every narration is different. Only a single entry for Anderson should ever exist in a PT (indicated by the identical nID), but due to plot variations from ME1/2 there's two versions of the entry (indicated by the different string IDs and Sound Property numbers). According to TankMaster, negative values means ME3 is pulling the sound file from the pcc's import table. Positive means it is pulling it from the pcc's export table. Entries with no sound (sections, categories, secondary entries) have "0" or "-1" in this location. Important note: putting a non-zero integer here when making a new entry that doesn't have a sound file will lead to the game crashes when viewing the new entry in game.

The SectionID/IsPrimary field contains numerical identifiers that create the codex layout you see in game. A value of "1" indicates a primary codex section header. For example, "Known Associates" or "Aliens: Council Races" when it appears in the primary codex section. A value of "0" indicates a secondary codex section header OR that the line of data is unused in ME3. Integers under 50K correspond to ME1 and ME2 codex entries (also called "pages"). Integers over 50K correspond to ME3 codex entries/pages.


Locating an Entry to Edit
1. Backup your entire BIOGame folder.

2. Open up the TLK Editor in ME3Explorer. You'll need to cross reference the entries in the Codex Editor against the contents of the BIOGame_INT.tlk file. For help with the TLK file see this post in the Dialogue Editing Tutorial.

3. Now open the Codex Editor and expand the codex map.

4. In theory, any entry not used in ME3 can be commandeered. However, it helps to look for a few things.

  • Focus on entries with string IDs under 500K, which corresponds to ME1 and ME2 TLK material
  • Having 2 string ID's present means that its an entry and not a section/category header, so focus on rows 60-262
Spoiler:
Image
We're going to try looking up the entry contained in row #96, as it meets all our above criteria. To do so, locate the number in either position 6 or 7. These are the string ID's contained in the TLK file. Let's use position 6, which is 340863.

5. In Notepad++ do a Ctrl+F, put in your string ID number and click Find Next. It may take a little while, since the TLK is a half million lines, ha.
Spoiler:
Image
What you see here is a drell codex entry. Since the string is under 500K, that means this is probably the entry from ME2, but to be sure that it isn't the one used in ME3, we should do one more check. Let's make sure there's another drell codex entry present that IS used in ME3, since we know one exists.

6. Open up your game (or the Mass Effect wiki) and find the drell codex entry. Write down/copy the first few words of the entry. Don't assume that the entry is identical from ME1 to ME2 to ME3. It might not be, so be sure to look up the exact wording of the ME3 entry.

7. Go back into your XML and do another Ctrl+F. We want to use the phrase: "Two centuries ago, the hanar." When we do this, here's what we find:
Spoiler:
Image
These string IDs are different, so this definitely looks like a duplicate entry. Let's go back and see if we can find them in the Codex Editor to make sure this is the entry that is used in ME3.

8. This is the most tedious part of the process because you can't do a Ctrl+F in the Codex Editor (yet). You need to scan down the entries by hand. Luckily, most of these entries tend to be in the same place. And...here's what we have:
Spoiler:
Image
Looking at this entry, we see a few things: we have Title and Description strings, a nID, a Sort Priority, and a Sound Property. This is the ME3 drell codex entry. That means we can commandeer the unused ME2 entry to create our Thane Krios codex entry.


Editing the Entry
Before we can actually edit the entry, we have to figure out which new variables to substitute for the old ones. Positions 1, 2, 3, 6, and 7 should stay the same. You'll need to edit positions 4, 8, 9, 10, and 11.

9. Position 4, PlotStateID=18819 -- Acquisition of codex entries in ME3 are triggered by State Events setting plot bools. While some entries do exist be default at the start of a new game, the mechanics of adding to this list are uncertain due to differences in how the game handles imports versus new playthroughs (see this post for more information).

For the purposes of this tutorial, we want ours to trigger after speaking to Thane in Huerta, so we're using the plot bool for this conversion, which is 18819. Some important points:
  • When linking a new entry to an existing plot bool, you may only link one entry. Linking more than one will result in only one triggering.
  • If you want to add a new entry that will only appear at the beginning of the game for non-imports you must also edit a list in the Coalesced.bin at bioui.ini > sfxgame > biosfhandler > newcharacter > newgamestartingcodexentries
Spoiler:
Image
Important note: The entry won't pop in a save loaded after the fact. Therefore, if you want it to pop after the completion of a mission, you'll need to find the correct plot bool and then test it in game by completing that mission. If you want the entry to be there at the start of a new game, you'll have to track down the correct bool by finding a comparable entry and seeing what value it has in this position.

10. Position 8, nID=90 -- If creating a primary entry/page, use an integer here greater than 89. This number will match one used in Coalesced.bin later. Creators of new secondary codex entries should place "0" in this location.

11. Position 9, Sorting Priority = 4 -- Any primary entry/page can be numbered from 1-5 depending on where you want it to appear in relation to the other entries in that section. For Thane, we know we want him to appear in "Known Associates" like his ME2 colleagues, and we also want his entry to follow any sorting conventions the game uses. For example, we found out earlier that Hackett and Anderson are at the top of the section, followed by the ME3 squad, followed by ME2 squad/major crew, followed by minor crew (Chakwas/Michele). Other than that, they follow alphabetical order. This means Thane should be set to rank "4" to come alphabetically after Joker in this example:
Spoiler:
Image
For your own entry, you'll need to locate a small assortment of existing entries in the correct location by doing strategic word searches in the .tlk to find their string IDs, and then cross-referencing those in the Codex Editor. Then you'll be able to identify the existing sorting mechanism for where you intend on placing your entry. Remember, most secondary entries have "3" here, but always double check to be certain.

12. Position 10, Sound Property = 0 -- New entries added by modders should always place "0" in this position, since we can't yet add new audio to the game. Failing to do this will cause game crashes/freezes when trying to view your new entry.

13. Position 11, Section = 58251 -- Use this parameter to define what section the new entry/page should appear in. The only section I've identified so far is this one for "Known Associates". I did so by following the same procedure already discussed. Use strategic search terms in the TLK to find a specific entry, then cross-reference the string IDs in the Codex Editor and see what they have for this value.

14. Now that you know your values, double click on the row you want to edit. Fill in the new values in the box and hit "Ok." Then you'll have edited yours to look something like this:
Spoiler:
Image
Note: Disregard the incorrect value for Sound Property. This screenshot was taken before we understood what the property meant; it should be set to 0 and not -92.

15. Save changes and close the Codex Editor. Be sure to keep all your numbers handy, since you aren't done yet!

16. Go back to the TLK file. Ctrl+F again and locate the original string ID's that correspond to the old entry you are replacing. We want to replace this text with the text for our new entry. You can either type it in or copy and paste. Notice that the text always comes before the header. After you finish, it should look something like this:
Spoiler:
Image
17. Save and close the XML when you're done.

18. Go back to ME3Explorer and reopen the TLK Editor. This time you want to "Create" the new TLK. Follow the instructions and consult the post linked earlier if you run into trouble. Don't forget to place the edited TLK in your Main CookedPCC or into your DLC mod (whichever you're working on).

Almost there! Adding an Image
The final thing you need to do for a primary entry is set an image. I don't know a lot about textures, so I'll cover one method of replacement. There are other methods. You may of course use any image you wish. Screenshot, 3D render, fan art, whatever. You can even just refer to an image already present in the game files. Be aware if you are releasing the new entry as part of a mod and you use a piece of fan art, you should ask permission from the artist beforehand. If you are trying to keep things immersive, try to stay "in the flavor" of the original Codex images. The Mass Effect Wiki is a good reference to help with that.

19. Codex images are like GAW asset images. They are DDS textures 512x256 in size, that lack mipmaps and use DXT1 compression. You must properly format your image in this manner using an editor like Paint.NET, Photoshop, GIMP, etc. I'm not going to explain this process; there are other tutorials around this site and other sites, for that :]

Important Note: These images tend not to scale well on HD monitors. BW's do for the most part, but replacement images don't for some (strange) reason. I'd encourage you to use images of twice the resolution (1024x512) to keep them from appearing too fuzzy in game, as for many people they will be scaled significantly over 512x256 in size. With the advent of higher resolutions like 2560x1440 and 4K, you may even want to go higher.

20. For the most part, you can't actually ADD new textures to the game files; you have to select a current one to replace. The trick is finding something the game doesn't use and is of the appropriate size. Start by opening Texplorer.

21. Navigate to GUI_GameManual in the texture tree. This folder contains several textures for the Kinect and Xbox. These will not be used on the PC, so they are good candidates for replacement. ThaneMOD replaces all four textures in SFXImages_GameManual_10.pcc, therefore if you want to retain compatibility with our mod, don't use these.

Important Note: GUI images like these are saved to the pcc file itself. They are not saved in the Textures.tfc. Texplorer doesn't understand this, though, so it may generate a CustTextures0.tfc upon replacement and tell you that's the new image is located there. Or, it may not create a CustTextures0.tfc, will replace the image and say the replacement is written to the Textures.tfc. Either way...it lies! The texture is saved to the .pcc file. For a list of all PCC-stored textures consult the Coalesced.bin: bioengine.ini > sfxgame > sfxengine > dynamicloadmapping.
Spoiler:
Image

22. Choose the image you want to replace by right-clicking on the thumbnail and selecting Advanced > Replace. Navigate to the location on your HDD where the image is stored and click "Open". Follow any additional instructions in the dialogue. Once you have the new image added, right-click on the thumbnail again and select Preview. You should see your new image. Save the changes.
Spoiler:
Image
Image
23. The final step is to set the image path using the nID in the Coalesced.bin. I use Wenchy, because I like the interface. Open up your editor of choice and load this file.

24. Navigate to bioui.ini > sfxgame > sfxgui > journalcodex > lstimages. Scroll to the bottom.
Spoiler:
Image
25. You can see that the list ends at 89, which is the final vanilla codex image entry. This is what we expected from what we saw in the Codex Editor. All we need to do is add our image path to the bottom, following the same convention that the other entries use (Folder.File name). In our case that is:
Code: Select all
(nID=90, sName="GUI_GameManual.Kinect_SquadOrders_512x256")
Spoiler:
Image
Save and then quit.

26. Do aquick TOC.bin update and then log into the game and check your work. If you've done everything correctly, your entry should pop after the specified plot event has been completed with the new image and text you've added.
Spoiler:
Image
Image
That's it! Enjoy your new codex entry :)

-------------------------------------------------------------------------

Using TankMaster's new Natives Editor
TankMaster's Native's Editor (download here) gathers all the same codex information as the ME3Exp Codex Editor, but bundles it with a couple other editors and an extremely user-friendly GUI.

Upon opening the editor the Codex Map tab should already be selected, as seen here:
Spoiler:
Image
The biggest difference between the two editors (besides the GUI) is 1-separation of sections from pages, and 2-the ability to ADD and REMOVE pages/sections.

ME3Exp has a single list of codex data, whereas the Natives Editor actually divides this data up into "section" and "page" categories accessed by a dropdown menu on the upper left. This division is actually very convenient when it comes to browsing and editing codices. Obviously having the fields labeled is helpful, too. Below, you can clearly see a section that does not occur in the primary codex:
Spoiler:
Image
A quick TLK search for the string shows this defines the section for "Weapons, Armor, and Equipment" (the one above is "Known Associates"):
Code: Select all
     <string>
        <id>712583</id>
        <position>95774</position>
        <data>Weapons, Armor, and Equipment</data>
    </string>

Codex Pages (entries) are viewed by changing the dropdown, as seen here:
Spoiler:
Image
Editing a page or section is as simple as adjusting the values and saving the file. With this tool, however, it's also possible to add and remove pages and sections. The modder doesn't have to work off of a current entry like with the ME3Exp tool. Instead, it's possible to use the ADD feature to set up an entirely new entry. FYI, I'm unclear as to why there are two "ID" fields listed, but the values are always identical, regardless of section or page. So, stick to that if you use the editor.

Happy codex editing :)
Last edited by Anonymous on 10 Mar 2014, 16:15, edited 4 times in total.

User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: Tutorial : Codex Editing

Postby Renmiri » 12 Sep 2013, 13:57

Awesome tutorial, ty ty ty !
Renmiri
Emeritus
 
Posts: 207
Joined: 31 Aug 2012, 20:42
Has thanked: 141 time
Have thanks: 37 time

Re: Tutorial : Codex Editing

Postby giftfish » 24 Feb 2014, 19:19

Updated some information about the image replacement (GUI images are saved directly to the pcc file).
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: Tutorial : Codex Editing

Postby giftfish » 10 Mar 2014, 16:16

Updated the tutorial with a new codex setting I found in the coalesced that pertains to new characters in ME3.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: Tutorial : Codex Editing

Postby giftfish » 09 Dec 2014, 22:52

FYI -- Tutorial updated with latest information. Now also includes a small section on the codex editor in @TankMaster's Native's Editor :)
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time


Return to Modders' Tutorials

Who is online

Users browsing this forum: No registered users and 0 guests

suspicion-preferred