Ben | DirectX work on Meshplorer

Development board for ME3Explorer tools. Please try to keep discussions for each tool inside its own thread.
giftfish
Toolset Developer
Joined: 08 Jan 2016, 02:35

16 Nov 2016, 17:43 #1

Ben is doing some DirectX work with Meshplorer, a bit of which can be seen in this issue on GitHub.

Ben, please feel free to post your progress, any questions, etc.
Reply

benji
User
Joined: 07 Nov 2016, 05:24

17 Nov 2016, 08:57 #2

Summary:
I'm working on bringing the toolkit's legacy DirectX code up to date by completely reimplementing how 3d previews work as a reusable control. This includes hopefully eventually removing the Managed DirectX libraries, which were abandoned in 2005, and replacing them with SharpDX, as well as upgrading from fixed-function Direct3D9 to Direct3D11.
This alone doesn't mean much for the average user - but wait, there's more! I'm also going to see if I can add some new features to the viewport, such as rotating & panning & zooming & optional wireframe & optional grid & optional view axes, as well as maybe accurately supporting more material types (there is a *lot* of variation between materials and not much consistency). Then perhaps material editing and reassigning, probably OBJ export, and whatever else I have the time for.
EDIT November 18 2016: Change of plans: keeping a super low profile with this project. Don't add new features. Trying not to change any UI.

At the moment, Meshplorer is the only tool I'm working on, but once this new viewport control is satisfactory, I plan to replace all the old viewports.
I should probably release testing builds. I really feel like I don't want to release half-finished code, even if it's stable, but on the other hand, I don't actually use the modding tools for modding, so I'm sure to break a few things unintentionally and need people to test my code out.

There's a ton of work to be done and, let's be realistic, it might not even get finished. I will be leaving for college this summer, so I'll try to get in as much work before then as possible. With this in mind, I have a TODO list (in maybe priority order).

TODO List:
Viewport Rewrite:
[X] Fix material switching
[X] Persist viewport prefs in registry via Properties.Settings
[X] Remove Alt key requirement for camera movement
[X] Viewport first person support (for level editor consistency)
[X] Use new viewport in Meshplorer2
[X] Fix depth buffer issues
[ ] Implement correct import load precedence
[ ] Cache texture scan results
[ ] Use new viewport in Level Editor
[ ] Remove Managed DirectX from ME3Explorer project (but not from ME3Creator)

Beyond:
[ ] Skeletalmesh material list add / remove entries
[ ] Investigate FBX export for external use of rigged skeletal meshes?
[ ] optionally render origin / coordinate axes?
[ ] optionally render grid?
[ ] bring back those AABB things from the old viewport (?)
[ ] Investigate shadercache.


Feel free to offer suggestions! Or give it a test by compiling from my GitHub fork, branch New3DView https://github.com/epicabsol/ME3Explorer/tree/New3DView :D

Screenshots:
[+] spoiler
November 16th 2016:
Image
Image
Image
Image
November 17th 2016:
Image
November 19th 2016:
Image
November 26 2016:
Image
Last edited by benji on 15 Dec 2016, 09:12, edited 11 times in total.
Reply

benji
User
Joined: 07 Nov 2016, 05:24

18 Nov 2016, 08:45 #3

I've doodled up a UI sketch for a new "Referenced Textures" tab in the center pane of Meshplorer. It would give you detailed information about the state of the textures used by the selected model.

Image
The checkered textures you see would be applied in the viewport as well.

Can I get some feedback?
Reply

giftfish
Toolset Developer
Joined: 08 Jan 2016, 02:35

18 Nov 2016, 14:57 #4

@Ben -- In relation to your GUI question:

I'm assuming you're still doing all this in Winforms, correct?

Unless UI changes are absolutely necessary for the new features you are making to the tool, then making them is, imo, a waste of time. As I mentioned via PM, the entire tool needs to be re-written in WPF. That includes all the work you are doing now. When that happens, the new GUI will be completely redesigned to adhere to the new GUI template that K, Sir, and I have already created (see the mock up I linked you via PM to the new WIP Audio Editor for an example).

Since it looks like you do need a way to reference textures, I'd just say to add a new node to the current tree-structure in Meshplorer, "Textures", and put a list of the exports/imports numbers and their names. In other words, what the Materials node looks like in the current stable, just with the actual names, since that is useful information. Format/compression isn't needed, since that's available in Texplorer/PackEd, and changing the texture itself isn't something that should be done in Meshplorer. File information isn't needed since all texture export/imports are located in the file that's loaded.

----

Some things I'm not clear about:

1. What are your goals in displaying the textures? They look pretty, but Meshplorer has always been about meshes and materials, not textures. In this case, the wire frame was functional, but certainly not pretty. If folks wanted to see the texture wrapped on the model, they needed to use 3D modeling software, like Max. Is this about having the ability to assign different materials and see how they will interact with the texture? Or to simply display what the skinned model looks like in game so folks can extract/import, locate the correct textures for a specific model, etc?

2. I'm not clear why the texture scan is needed. Meshplorer is made to handle specific files. You open a file and it displays the meshes inside. Those meshes all refer to texture exports or imports in that same file. What's the scan for? To detect if a texture has been changed? If that's the case, why not just scan a file in the background whenever it's loaded?

3. What new editing features, if any, are you thinking about adding? This is something that we'd need to consider in the context of workflow in other tools.

----

A few comments about your to do list and some of your other questions:

1. Ultimately, Meshplorer should provide a way to visualize the links between texture/mesh/material in a file. A way for the user to load up a file and see the various static or skeletal models in it -- fully skinned with textures. This way, texture modders can easily the texture's appearance and name, and head into Texplorer to view, extract, and replace them. Mesh modders can export/import meshes and reassign materials. These are (or should be) the core functionalities of Meshplorer. Viewing different texture maps, processing effects (bloom), shaders, testing different textures on the model (I'm assuming this is what the "override" feature is), are all things that can be done/observed in other programs. They should come last in priority.

2. All materials, pretty much, have "parents". It's common to have a material > USER > MASTER relationship. Things get very complicated, very quickly. Kinkojiro has done the most work with materials around here. Sir and I probably come next. Ottemis is a texture/mesh modder and has done a significant amount of work in Meshplorer, but has more limited knowledge of the back end of things with the flow of materials and properties as can be seen in PackEd.

3. Keep in mind that unlike most games, in Mass Effect, the same mesh can be found in many, many files. Since Meshplorer loads one file at a time, any GUI elements related to file name of a mesh/texture are unnecessary, since all texture import/exports references must be in that same file.

4. Displaying texture format/compression is unnecessary in Meshplorer. If that information ends up important to a user, then they will see it when they extract the texture using Texplorer. It's not important in the context of viewing the skinned mesh.

----

@Sir -- Looking forward to your feedback :)
Reply

CreeperLava
User
Joined: 07 Feb 2015, 21:52

18 Nov 2016, 16:22 #5

giftfish wrote:1. What are your goals in displaying the textures? They look pretty, but Meshplorer has always been about meshes and materials, not textures. In this case, the wire frame was functional, but certainly not pretty. If folks wanted to see the texture wrapped on the model, they needed to use 3D modeling software, like Max. Is this about having the ability to assign different materials and see how they will interact with the texture? Or to simply display what the skinned model looks like in game so folks can extract/import, locate the correct textures for a specific model, etc?
Just chiming in on this : I personally love the idea of being able to see how the textures will render on the meshes.
Reply

giftfish
Toolset Developer
Joined: 08 Jan 2016, 02:35

18 Nov 2016, 17:00 #6

CreeperLava wrote:
giftfish wrote:1. What are your goals in displaying the textures? They look pretty, but Meshplorer has always been about meshes and materials, not textures. In this case, the wire frame was functional, but certainly not pretty. If folks wanted to see the texture wrapped on the model, they needed to use 3D modeling software, like Max. Is this about having the ability to assign different materials and see how they will interact with the texture? Or to simply display what the skinned model looks like in game so folks can extract/import, locate the correct textures for a specific model, etc?
Just chiming in on this : I personally love the idea of being able to see how the textures will render on the meshes.
I agree, if only for texture identification purposes (see my edit, which I made while you were posting). I'm just trying to clarify goals and intent. Both with display and editing.

Basically, it looks like Ben is trying to make Meshplorer more like Nifskope. This is great, but ME3 is a strange animal, game-wise, and isn't structured or modded like many other games. Those differences need to translate into the tool.
Reply

Kinkojiro
Modder
Joined: 02 Dec 2013, 04:14

18 Nov 2016, 23:38 #7

Ben - this looks really great. Meshplorer has always been one of those things that got left behind as the rest of the toolset got updated.

From a practical (users) standpoint - seeing the textures in the meshplorer is good, but it will never be that close to how it renders in game (spec/emissive / tint etc).

Each material is coded in the binary and the shadercache. You cannot add new materials to the game because they don't have the shadercache. The material expressions etc are generally much more simple versions (versus ME1/2). I am guessing when they cook the material the cooker removes the most common settings etc and puts them in the cache.

Each material may have material instances - these replace individual textures/settings in the main material. This is the parent/child relationship you spoke of. An instance may also have other instances (i.e. a chain).

Bioware uses its spec maps to carry a lot of information - each color channel generally operates differently - i.e. it can be red = spec, green = spec power, blue = reflectivity, alpha = emissive. Beware though these can and do change with every material. This info is not in the material AFIK and is why I think it will be impossible to accurately view specs in the viewer.

For you to think about: as a heavy user this is what I really want from Meshplorer:

Skeletal Meshes
View model
View material on the model
Set the materials (including adding extra materials or importing the number of materials from UDK)
Import from UDK (currently we can only import and overwrite a model with exactly the same number of bones)
Write import bone names to name table

Static Meshes
View model
Set the materials
Set the RBBodySetup
Import from UDK.

Hope it helps.
Reply

giftfish
Toolset Developer
Joined: 08 Jan 2016, 02:35

18 Nov 2016, 23:58 #8

Hugely helpful, Kinko. Thanks very much for the explanation :)
Reply

benji
User
Joined: 07 Nov 2016, 05:24

19 Nov 2016, 01:15 #9

Whoops, I kind of forgot about the impending WPF rewrite. (not really, but I figure some improvement in the meantime couldn't hurt, right??)

It seems like I was way overdesigning this to be super user friendly.

Point taken that I was trying to put way too much information per texture. The idea was to make diagnosing model texture issues easy, but I guess that's not needed. Thanks for the input. The tree already has texture entries, I added them earlier. They don't look very nice though, but, like I said, I forgot about the impending rewrite.

Also, for the texture override feature, *I'm not talking about changing the texture*. I want to clear this up [edit - you seem to understand now]. Here's the use case: you are editing a face texture to remove seams. You save the texture as "texture.png". You tell Meshplorer to override that texture on the model *preview*, and whenever you make a change and save (export) the texture to the png, Meshplorer updates the preview in the viewport. The proposed texture preview does *not* alter any files, it is a temporary way to preview your texture edits in real-time on the model, as opposed to applying the change and the opening mass effect, repeat. Don't worry, I understand actually changing textures is texplorer.

The goals with previewing textures: What's the point of a preview if it isn't accurate? Has anyone ever said, "I wish this model viewer only showed a blue checkerboard instead of what the model looks like"? I saw how only a few models would show their textures and assumed that was a bug to improve. Modders should be able to preview their textures on models without actually having to open the game, right? :?

Also, at the moment, I'm not clear on how to use the toolkit to modify what texture a material references. I don't mean change the texture pixels, I mean edit the material diffuse property to point to a different in-game texture. I was hoping to add this functionality, but I really don't think that one massive treeview is the best way to do it (?)

Yes imports / exports are located in the same file as the mesh, but imports don't have *any* texture information (according to the code we use - it used to just give up if the texture was an import) (unless I'm seriously missing something). In order to actually view the texture, get its size, etc, Meshplorer needs to know what export in which pcc the import is pointing to. That's what the texture pcc filename is about. But wait - the import doesn't specify where the actual texture export is, so my scanner has to search the pccs for it. That's what the scan is for. But, if you don't care about what the texture looks like, you don't need to wait for the scan to complete. It runs in the background, and textures appear on the model as the locations that the imports reference to are found.


I'm still very new to how the actual Mass Effect asset storage works. My hope was to display the loaded mesh / texture information better and make the viewport more user friendly, not to add extra modding features - I'm just trying to pick up the modding system as I go along. I may have become over-ambitious :D . Anyway, that's why I'm working on 'last priority' things - because my relevant knowledge here is mainly around DirectX (although I've never done GPU skinning before) and not at all in modding Mass Effect games.

@Kinkojiro Is there a way in code to load the parent / un-instanced MaterialInstanceConstant for a material? Also, I want to at least try to get materials looking *more* accurate, if not just for the challenge of doing it. Unless you guys don't want that? On the shadercache: is there some documentation on its file structure (because vaguely saying something is impossible just makes me want to try, you know)? As for spec handling, again, sounds like a healthy challenge, thanks for the heads up.

Sorry I'm not of more use for things that aren't coming in last priority. :?

Ben
Reply

giftfish
Toolset Developer
Joined: 08 Jan 2016, 02:35

19 Nov 2016, 03:08 #10

benji wrote:Also, for the texture override feature, *I'm not talking about changing the texture*. I want to clear this up [edit - you seem to understand now]. Here's the use case: you are editing a face texture to remove seams. You save the texture as "texture.png". You tell Meshplorer to override that texture on the model *preview*, and whenever you make a change and save (export) the texture to the png, Meshplorer updates the preview in the viewport. The proposed texture preview does *not* alter any files, it is a temporary way to preview your texture edits in real-time on the model, as opposed to applying the change and the opening mass effect, repeat. Don't worry, I understand actually changing textures is texplorer.

The goals with previewing textures: What's the point of a preview if it isn't accurate? Has anyone ever said, "I wish this model viewer only showed a blue checkerboard instead of what the model looks like"? I saw how only a few models would show their textures and assumed that was a bug to improve. Modders should be able to preview their textures on models without actually having to open the game, right? :?
First, PNGs cannot be used for any game in the trilogy; only DDS files can be used. This means only textures in DDS format should be allowed to be loaded, despite it being a test. Otherwise, novices are going to get confused.

There's nothing wrong with adding a skinning feature, however, realize that any texture modder worth their salt will be using 3DSMax as they create and/or edit their textures. They do this exactly to monitor things like seams. Meshplorer would be a poor way to do this, and it should not be treated as a substitute for a legitimate 3D modeling program. I almost brought this up earlier, but decided to let you respond to my questions first. The real value in you displaying the textures is that, A-they are certainly nice to see, B-I'm pretty sure they are needed to visualize materials properly, and C- a novice texture modder will now be able to easily locate which textures a mesh uses (both visually and via the new fields you've added).
benji wrote:Also, at the moment, I'm not clear on how to use the toolkit to modify what texture a material references. I don't mean change the texture pixels, I mean edit the material diffuse property to point to a different in-game texture. I was hoping to add this functionality, but I really don't think that one massive treeview is the best way to do it (?)
This is done in PackEd, as they are defined via Unreal properties. This probably isn't a feature that should be duplicated in Meshplorer, and gets into some of the previous conversations we've had about modding workflow, which tools do what, avoiding duplication of functionality, etc.
benji wrote:Yes imports / exports are located in the same file as the mesh, but imports don't have *any* texture information (according to the code we use - it used to just give up if the texture was an import) (unless I'm seriously missing something). In order to actually view the texture, get its size, etc, Meshplorer needs to know what export in which pcc the import is pointing to. That's what the texture pcc filename is about. But wait - the import doesn't specify where the actual texture export is, so my scanner has to search the pccs for it. That's what the scan is for. But, if you don't care about what the texture looks like, you don't need to wait for the scan to complete. It runs in the background, and textures appear on the model as the locations that the imports reference to are found.
Okay, so the scan is for imported textures, only. That makes sense, but I don't think there needs to be a button for the feature, nor does it need to list the source file. Just have it scan in the background when loading the file and load the texture when ready. As far as the location of the imported texture, that information is already in PackEd. Displaying it in Meshplorer isn't necessary, as 99% of the time, it will make no difference to the user. If it does become important, then the user is getting into some more advanced modding and will realize at that point to look in PackEd.
benji wrote:I'm still very new to how the actual Mass Effect asset storage works. My hope was to display the loaded mesh / texture information better and make the viewport more user friendly, not to add extra modding features - I'm just trying to pick up the modding system as I go along. I may have become over-ambitious :D . Anyway, that's why I'm working on 'last priority' things - because my relevant knowledge here is mainly around DirectX (although I've never done GPU skinning before) and not at all in modding Mass Effect games.
This creates something of a challenge because new additions to the GUI -- even just in a viewing capacity -- should reflect how the modding community needs to use the tool. Kinkojiro has given you some very good advice. He's the most experienced user of Meshplorer we have, and he's basically given you a roadmap of what we need due to how Meshplorer is actually used. I strongly encourage you to take his advice.
benji wrote:Also, I want to at least try to get materials looking *more* accurate, if not just for the challenge of doing it. Unless you guys don't want that? On the shadercache: is there some documentation on its file structure (because vaguely saying something is impossible just makes me want to try, you know)? ... As for spec handling, again, sounds like a healthy challenge, thanks for the heads up.
Shadercache is an enormous challenge, and I don't believe there's any Unreal documentation. Sir and Kinko will have more info on that for you.

As far as getting the materials to "look more accurate", the thing to realize is that ME3 uses a very strong color palette, a variety of post processing effects, and most importantly, has huge variations in lighting across and within scenes. This has an enormous effect on how color, reflections, specularity, etc, are handled in game. It's possible for each line in a single conversation to use a different light rig. This means there is no way for you to replicate or mimic exactly how the material will look in game (which I'm guessing is what Kinko meant by his statement). This is why I said these things should be last priority. Because they cannot be done with any real level of accuracy, and if that's the case, then the feature likely shouldn't be implemented. This is different situation than, say, Skyrim, which basically doesn't use lighting changes for conversations and scenes at all. It's all ambient/environmental lighting.

Spec handling will be very difficult, which is something I learned more recently. There is no hard and fast rule. It will change on a per-texture basis, so I have no idea how you'd code for that. BW also encodes color information in their specs, which is not normal. They like to use the alpha channel in certain DXT5 textures in place of building in a separate spec texture, which means it's not holding transparency data in the typical sense. Again, this changes, per texture.

In short, BioWare likes to do things their own way. Shocker.
Reply