Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

Ben | DirectX work on Meshplorer

Development board for ME3Explorer tools. Please try to keep discussions for each tool inside its own thread.

Re: Ben | DirectX work on Meshplorer

Postby Ottemis » 05 Dec 2016, 20:56

I'm going to take it for a spin now =) I'll try and break it too, keep you posted!
User avatar
Ottemis
Harbinger
 
Posts: 824
Joined: 11 Mar 2013, 12:14
Has thanked: 225 time
Have thanks: 247 time

Re: Ben | DirectX work on Meshplorer

Postby benji » 13 Dec 2016, 08:28

It's been two weeks since I released the build. Can I get some feedback, @Ottemis and others?
benji
User
 
Posts: 18
Joined: 07 Nov 2016, 05:24
Has thanked: 1 time
Have thanks: 6 time

Re: Ben | DirectX work on Meshplorer

Postby Kinkojiro » 13 Dec 2016, 09:20

It seems to work very well. The rendering is good, fast, easy to move things around and look at them. It seemed stable, I had no crashes etc.

The detail of the middle panel showing the underlying materials is useful, especially where there are multiple materials and textures (e.g. faces).

The importing of new meshes from UDK/PSK works well, and seems to fix the bug with it imports going on forever until you click a different mesh that is in 3.0.1.

Obviously your rendering system doesn't show custom or DLC textures, but that is ok. If somebody has upgraded the texture to a HR one, with the same name, it seems to default to the vanilla game version, even though the tfc reference is different.

How often should the scanning for imports happen? It occurred a few times for me. Doesn't it save the results for future reference?

I had one issue with a few models of ships
- when rotating around it would be half in and half out of the wireframe.
- It didn't render the textures.
- Try looking at BioD_Omg001_Space.pcc I wonder whether it is to do with the scaling of the model. In unreal units these ship models are huge.

Overall this is a big improvement. Thanks for your work, and I hope it gets incorporated in the full version shortly.
User avatar
Kinkojiro
Modder
 
Posts: 578
Joined: 02 Dec 2013, 04:14
Has thanked: 233 time
Have thanks: 249 time

Re: Ben | DirectX work on Meshplorer

Postby giftfish » 13 Dec 2016, 13:47

Kinkojiro wrote:Obviously your rendering system doesn't show custom or DLC textures, but that is ok. If somebody has upgraded the texture to a HR one, with the same name, it seems to default to the vanilla game version, even though the tfc reference is different.

Really? This seems strange (and problematic) if I'm understanding you correctly.

Does this mean that:

1. No meshes from BioWare DLC display the correct texture, as referenced inside that DLC?
2. No meshes from DLC mods display the correct texture, as referenced inside those DLC mods?

We need DLC mod support for tools (and obviously BW DLC support). Especially for a tool like this where the entire point of this improvement is to show textures on the mesh. Mesh modders are taking advantage of the DLC mod system to implement their mods; they will undoubtedly have custom textures, which should be viewable through Meshplorer. They are the prime users of this tool. Display of incorrect textures will lead to confused modders and users.

Almost every tool in the toolset predates DLC mods and DLC unpacking, which is why WV created them to do things like "only scan on base game files". There's several tools in the toolset that use this mechanic, but it's not a feature we want to carry forward.

-----

I'm glad to hear that on the whole everything looks good :) It looks like 3 things need to be addressed:

1. DLC texture support
2. Model ship support
3. Scanning
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: Ben | DirectX work on Meshplorer

Postby benji » 13 Dec 2016, 17:48

Kinkojiro wrote:The importing of new meshes from UDK/PSK works well, and seems to fix the bug with it imports going on forever until you click a different mesh that is in 3.0.1.


I don't recall touching the UDK/PSK import/export code at all, but I'm glad things are working! :D

Kinkojiro wrote:Obviously your rendering system doesn't show custom or DLC textures, but that is ok. If somebody has upgraded the texture to a HR one, with the same name, it seems to default to the vanilla game version, even though the tfc reference is different.

I will start looking for a fix right away. For the custom textures, I use the Unreal.Classes.Texture2D class and then call its generatePreviewTexture method, which calls its extractRawData method, which appears to take tfcs into account. (https://github.com/epicabsol/ME3Explore ... 2D.cs#L213)
I think the issue is with using the first export that matches the texture fullpath. This would be an issue if the tfc reference is only changed in one export, and not all of the identical exports across pccs. There are often multiple exports with the same name and texture data (when vanilla) throughout the pccs, and I'm not sure how the game chooses the one to use, so I just load the first one the scan finds and ignore the rest. I'm hoping that's where the issue is, but I'm not sure of the correct way to pick which export to use when multiple exports match the name in the import.
Or maybe that's not where the issue is? I'll do some debugging after I finally get around to learning how to mod textures. :D

Kinkojiro wrote:How often should the scanning for imports happen? It occurred a few times for me. Doesn't it save the results for future reference?

No, actually, it doesn't save the results for future reference. Meshplorer scans each time it is opened. I assumed that because trying to detect texture changes automatically would take just as long as a scan, users would be responsible for knowing to manually start the rescan after every texture addition / removal, otherwise they could be confused as to why their texture doesn't show up. I can change this to auto load and save if that's what the community wants.

Kinkojiro wrote:I had one issue with a few models of ships
- when rotating around it would be half in and half out of the wireframe.
- It didn't render the textures.
- Try looking at BioD_Omg001_Space.pcc I wonder whether it is to do with the scaling of the model. In unreal units these ship models are huge.

This sounds like an issue with the viewport camera's maximum depth. The more I increase it, the less precise that depth for near objects is. It is normally a fixed number hardcoded in, and increasing it will be super simple, but that will just put off the problem until another larger model comes around. It would be cool to instead scale the preview model down, depending on what size it is...

Kinkojiro wrote:Overall this is a big improvement. Thanks for your work, and I hope it gets incorporated in the full version shortly.

My pleasure. :D Looks like I still have a bit more work to do though!

giftfish wrote:We need DLC mod support for tools (and obviously BW DLC support). Especially for a tool like this where the entire point of this improvement is to show textures on the mesh. Mesh modders are taking advantage of the DLC mod system to implement their mods; they will undoubtedly have custom textures, which should be viewable through Meshplorer. They are the prime users of this tool. Display of incorrect textures will lead to confused modders and users.

Absolutely. I will look into this.

Thanks for the feedback!
benji
User
 
Posts: 18
Joined: 07 Nov 2016, 05:24
Has thanked: 1 time
Have thanks: 6 time

Re: Ben | DirectX work on Meshplorer

Postby giftfish » 13 Dec 2016, 18:37

benji wrote:I think the issue is with using the first export that matches the texture fullpath. This would be an issue if the tfc reference is only changed in one export, and not all of the identical exports across pccs.

Texture replacements are per PCC, not across PCCs. Players and modders have control over which PCCs new textures are installed into. In other words, the texture for HMF_CTHb_Nav_Diff (I might be making that name up) can be different in two PCCs. The ability to control this is partly how DLC modders create new textures.

benji wrote:There are often multiple exports with the same name and texture data (when vanilla) throughout the pccs, and I'm not sure how the game chooses the one to use, so I just load the first one the scan finds and ignore the rest. I'm hoping that's where the issue is, but I'm not sure of the correct way to pick which export to use when multiple exports match the name in the import.

Is this comment on in relation to the scanning and imports? If not, then we need to discuss a few things about how ME3 loads it's assets. I don't want to go into that discussion if it isn't necessary, so let me know.

Kinkojiro wrote:How often should the scanning for imports happen? It occurred a few times for me. Doesn't it save the results for future reference?
benji wrote:No, actually, it doesn't save the results for future reference. Meshplorer scans each time it is opened. I assumed that because trying to detect texture changes automatically would take just as long as a scan, users would be responsible for knowing to manually start the rescan after every texture addition / removal, otherwise they could be confused as to why their texture doesn't show up. I can change this to auto load and save if that's what the community wants.

Saving the results seems logical to avoid unnecessary scans. However, at minimum, wouldn't the scan need to take place every time a different PCC is loaded, if it hasn't been scanned before? Since every PCC has different texture imports and those imports are the purpose of the scan?

benji wrote:It would be cool to instead scale the preview model down, depending on what size it is...

This definitely sounds like a more foolproof solution. Either that or insert a new GUI element that allows the user to control the scale themselves. That way, they can scale it down (or up) manually, if/when necessary.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: Ben | DirectX work on Meshplorer

Postby Kinkojiro » 13 Dec 2016, 23:56

Benji, happy to help. I think it is important to lay out how texture loading works (sry gift if I am just jumping in):

(a) Exports

Texture2D - name reference and GUID for tfc:
=> ME3/Unreal -> looks for tfc in the default Cooked folder.
If not found:
=> Look in DLC cooked folders. Note it will only find the tfc if it ends in the folder name - i.e. Textures_DLC_MOD_EGM.tfc is in DLC_MOD_EGM\Cooked

So you maybe able to find the tfc by taking the name reference and parsing the DLC folder.

(b) Imported textures. Imported textures that actually get rendered in game are relatively few. Mainly VFX. What happens is the master materials using default standard, black/white/norm textures that are in the SFXGame.pcc. The vast majority of times these are not actually used but instances are created that replace the defaults with the actual textures.

Importing textures is very unstable and inefficient in unreal so especially in ME3 it is rarely used. The load order becomes vitally important. Take 3 files loaded in this order: A, B, C.

B or C can import from A
B can import from A but not C
A cannot import from B or C.

Given the game has limited control over the load order, especially when loading from a save, bioware have a few rules:

(1) All master materials and engine used textures are in SFXGame.pcc, Engine.pcc, Startup.pcc, BIO_COMMON.pcc/BIO_MP_COMMON.pcc. Note MP DLC also have their own startup files. These are primarily loaded during the boot screen.

(2) When loading a level BioP is loaded first and handles the rest of loading. Sometimes these do contain assets that are imported by files from across a level.

(3) LOC_INT files load before their main file. These are used for localized conversations and therefore not really relevant to this convo.

BioD to BioD or BioA to BioA files never AFIK have imports between each other. It is just too unstable.

So the bottom line with imports is you can capture 99% of texture imports by scanning the main game files and BioPs.
User avatar
Kinkojiro
Modder
 
Posts: 578
Joined: 02 Dec 2013, 04:14
Has thanked: 233 time
Have thanks: 249 time

Re: Ben | DirectX work on Meshplorer

Postby giftfish » 14 Dec 2016, 00:13

@Kinko -- Np. You explained it much better than I could have. Working on Bailey's overhaul, I've only now just really started to learn about the relationship between some of the global normals and VFX textures. The information you laid out above was still a bit muddled in my head; I don't think I could have explained it logically yet.

I *was* going to emphasize the per-PCC replacement of textures and that it's never a matter of the game loading "one" when it comes to exports. If 7 exports are loaded and there are 7 copies of EYE_diff in those 7 PCCs, then the game loads all copies, since each is part of a different level and referenced by a different mesh (even if the mesh may have the same name). I think we learned when I was working on robodog/varren that things can get a little wonky sometimes, but this is the general rule. The mechanic itself is illogical to folks who are used to working with games that contain a single central repository: one mesh in the entire game for "table A", one texture in the entire game for "table A", etc. I just wanted to make sure Ben wasn't confused about how the game worked with exports.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: Ben | DirectX work on Meshplorer

Postby benji » 14 Dec 2016, 09:38

Thanks, Kinkojiro and giftfish.

giftfish wrote:If 7 exports are loaded and there are 7 copies of EYE_diff in those 7 PCCs, then the game loads all copies

This is where I was misunderstanding, along with a misunderstanding of how imports are resolved. When a texture is imported (and this seems to happen a bit from what I can tell), right now this is all of the data on the import:
Image
That's not a lot to go on. I had hoped that the variable "PackageFile" would be the location of the texture export, but no such luck (unless I am using Package Editor, which also says the Wall02_Diff should be in Engine.pcc, horribly wrong). I'm not sure what the "PackageFile" variable means for imports, but it's not where the connected export is stored, unless I'm looking in the wrong Engine.pcc. A brute force scan by my scanning code can find the matching exports, but it doesn't know which one is the correct one to load. How should this work? Help?

(I'm using the BioT_Normandy_Int.Wall02_Diff import in BioA_Nor_210CIC.pcc as a test case)
benji
User
 
Posts: 18
Joined: 07 Nov 2016, 05:24
Has thanked: 1 time
Have thanks: 6 time

Re: Ben | DirectX work on Meshplorer

Postby Ottemis » 14 Dec 2016, 13:27

I have nothing new to add in terms of issues, though Meshplorer scanning every time I open it is less than optimal but I can 'work through it' so it's not an issue per say. The features you added are definitely spiffy.
Being able to see the texture associated with the mesh is handy, I doubt I'll use it often but It's actually saved me some effort the past week for meshes I was less familiar with.

If I run into any bugs not yet reported, I'll shout.
User avatar
Ottemis
Harbinger
 
Posts: 824
Joined: 11 Mar 2013, 12:14
Has thanked: 225 time
Have thanks: 247 time

PreviousNext

Return to ME3Explorer Toolset Development

Who is online

Users browsing this forum: No registered users and 0 guests

suspicion-preferred