Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

Alternate texture replacement tool - MassEffectModder

Non-toolset utilities for the ME Trilogy developed by community coders. Note: Restricted to coder- or modder-oriented tools. User-oriented tools belong where they can be adequately supported (e.g., Nexusmods).

Re: Alternate texture replacement tool - MassEffectModder

Postby aquadran » 23 Oct 2016, 11:54

I did speed benchamark (first time) :)
Computer: Windows 10, i5 3,4Ghz 2 cores+HT, 8GB RAM, SSD drive.
ME3 has 17 original downloable DLCs in DLC directory in my case.

Unpack DLCs: 4:38
Scan game data: 4:55
Remove null pointers: 8:27
Replace 2472 textures: 23:02

I think replacing testures (saving PCC) could be done a bit faster, but I drop more speed improvements as this speed is enough for me.
aquadran
Emeritus
 
Posts: 78
Joined: 23 Mar 2013, 05:07
Has thanked: 11 time
Have thanks: 23 time

Re: Alternate texture replacement tool - MassEffectModder

Postby CreeperLava » 23 Oct 2016, 15:01

There isn't any password with aquarium TPF, and I'm not sure it's even possible to password protect TPFs ?

Very impressive benchmark, you did a superb job ! I'm eagerly awaiting K's or Sir's opinion on including your improvements to ME3Explorer. (TPF Tools 4.0 ? :))
User avatar
CreeperLava
User
 
Posts: 844
Joined: 07 Feb 2015, 21:52
Has thanked: 119 time
Have thanks: 83 time

Re: Alternate texture replacement tool - MassEffectModder

Postby giftfish » 23 Oct 2016, 18:21

@aquadran -- Appreciate you letting us know about your new tool :]

[quote="aquadran"]Problem looks in mods, something is done different way comparable to vanilia game data. When I tried include ME3 DLC mods for texture replacements game crashed while using that parts. Issue may came from different way of replacement of textures or differences in saved PCC data in DLC mods.

When was the last time you looked at toolset save methods, aquadran? There have been lots of changes over the past year, and over the past few months, especially. My guess is it doesn't have anything to do with DLC mods; it's the save method of the PCCs itself. Maybe whether it has new imports, exports, etc. Either that, or it could be a shadercache thing.

[quote="aquadran"]Regarding compatibility:
- Can load ME3Explorer textures mods as legacy, it match texture also.
You can save as MassEffectModder compatible mod which is compressed.
You can extarct to indiviual files as matched textures.
- Can load TPF files with limited matched textures process.
You can also save and extract textures.
- Can apply non textures mods
Can load ME3Explorer mods related Export Data as replacement

When you say "mods" are you talking about files with the .mod extension?

----

Just a few reminders for everyone in this thread:

1. Detailed discussions and questions from modders and users about aquadran's new tool should be handled on his Github page. Basic questions here are no problem; this just shouldn't turn into a support thread.

@aquadran, now that you've announced your tool, please consider including a more complete explanation on your GitHub page. That will likely head off a lot of questions. Since all content mods are released as DLC these days and many texture users also use DLC mods, information regarding lack of compatibility with your tool is quite vital.

2. Please take any discussions as to how this may impact development of K's tools into his thread in the toolset development area.

3. K's re-write is separate from the toolset and has no projected date of completion (he's working at his own pace). Once complete, the re-write will need to undergo significant testing and then _may or may not_ be rolled into the toolset. If and when it is, toolset staff will examine the features and code of the re-write in the context of the existing toolset and current modding practices. There's a very good chance it will need to be adjusted for compatibility.

Shiny, new tools are exciting, and I'm not saying there's not stuff here worth considering (there is). I'm saying that at this point the capabilities of the toolset go far beyond texture editing and .mod files. Any changes to those functionalities need to be evaluated in the context of other features and goals. When a tool is written outside of the toolset things are much simpler. Once inside of the toolset, the new tool now must work under a different set of goals and constraints: file structure/saving, shader cache adjustments, compression, and many more. What's possible outside the toolset may or may not be possible/desirable inside the toolset.

We also just this April implemented a streamlined, robust texture replacement system that was two years in the making. Something would have to be profoundly better to prompt an overhaul, considering what a giant pain in the *** it is. I'm not trying to be a wet blanket, I'm just ensuring folks have realistic expectations.
User avatar
giftfish
Toolset Developer
 
Posts: 1247
Joined: 08 Jan 2016, 02:35
Has thanked: 129 time
Have thanks: 75 time

Re: Alternate texture replacement tool - MassEffectModder

Postby aquadran » 23 Oct 2016, 20:09

giftfish wrote:When was the last time you looked at toolset save methods, aquadran? There have been lots of changes over the past year, and over the past few months, especially. My guess is it doesn't have anything to do with DLC mods; it's the save method of the PCCs itself. Maybe whether it has new imports, exports, etc. Either that, or it could be a shadercache thing.

I don't believe it's saving method. I think it's just how mipmaps 4x4 and less are handled. ME3Explorer altering them, but works for engine somehow.
I don't understand yet what exactly expectations are for engine. I tried different ways, for now I could not find a rule. I checked ME3explorer code, but still not clear why it fit for engine.
At this point in my tool I simply not altering that mipmaps, there are in same number and resolutions as they are in original, with only content replaced.
But that part I can improve when I find a rule and make DLC mods be compatible.
There is also another thing related moved tables to the and of PCC file (if such cases exists in ME3 DLC mods). I should check that too.

Regarding texture groups and null pointers I think this should be considered on ME3exploer side.

giftfish wrote:When you say "mods" are you talking about files with the .mod extension?

yes, tool is able load them and I mean texture mods.

giftfish wrote:1. Detailed discussions and questions from modders and users about aquadran's new tool should be handled on his Github page. Basic questions here are no problem; this just shouldn't turn into a support thread.

@aquadran, now that you've announced your tool, please consider including a more complete explanation on your GitHub page. That will likely head off a lot of questions. Since all content mods are released as DLC these days and many texture users also use DLC mods, information regarding lack of compatibility with your tool is quite vital.
2. Please take any discussions as to how this may impact development of K's tools into his thread in the toolset development area.


I agree.

Few more words:
This was always as personal project but I wanted to share at some point.
So I don't consider using my tool for mass use, just pointing some alternate method for users and devs.
I prefer things go over ME3explorer as it's common tool for users. I hope some ideas from tool will be considered into ME3explorer.
aquadran
Emeritus
 
Posts: 78
Joined: 23 Mar 2013, 05:07
Has thanked: 11 time
Have thanks: 23 time

Re: Alternate texture replacement tool - MassEffectModder

Postby aquadran » 23 Oct 2016, 20:30

CreeperLava wrote:There isn't any password with aquarium TPF, and I'm not sure it's even possible to password protect TPFs ?

Very impressive benchmark, you did a superb job ! I'm eagerly awaiting K's or Sir's opinion on including your improvements to ME3Explorer. (TPF Tools 4.0 ? :))


ok... how long it takes to replace ALOT bundle in me3explorer?

Yes, there is a password bundled with TPF files by Texmod, but for MEIUTM TPF file does not. I'm not much familiar by Texmod, maybe there was different version of Texmod used. Anyway I know why it doesn't work for me.
aquadran
Emeritus
 
Posts: 78
Joined: 23 Mar 2013, 05:07
Has thanked: 11 time
Have thanks: 23 time

Re: Alternate texture replacement tool - MassEffectModder

Postby CreeperLava » 23 Oct 2016, 20:48

aquadran wrote:ok... how long it takes to replace ALOT bundle in me3explorer?


Depends on the CPU, SSD or HDD, and the version of ME3Explorer.
Before 3.0, it took a few hours on SSD with a relatively good CPU (10-12 hours for everything, including DLC extraction, tree building and texture installation for all DLCs).
After 3.0, texture installation time was cut down to under 1h30-2h (everything included). I believe tree building and DLC extraction also got a bit faster. Those never were the bottleneck anyways, the textures were. There is apparently an issue where AMD CPUs don't benefit from this improvement, but I'm not 100% sure that is the case yet.
I didn't think it would be possible to go faster than this, I was already very happy with 3.0. Your tool blows 3.0 out of the water :) (Sorry Sir, K^^).
User avatar
CreeperLava
User
 
Posts: 844
Joined: 07 Feb 2015, 21:52
Has thanked: 119 time
Have thanks: 83 time

Re: Alternate texture replacement tool - MassEffectModder

Postby aquadran » 23 Oct 2016, 22:32

I had to make speed up for developement purposes, when you need find out which texture create a problem from 2000 textures speed does matter a lot.
Tool opening and saving PCC file only once for whole textures bundle process.
CRC calculation and Zlib/LZX compression/decompression is done on multi cores.
aquadran
Emeritus
 
Posts: 78
Joined: 23 Mar 2013, 05:07
Has thanked: 11 time
Have thanks: 23 time

Re: Alternate texture replacement tool - MassEffectModder

Postby KFreon » 24 Oct 2016, 22:54

Wow nice benchmarking. Definitely have to look into this.
I do want to note though especially with the scan, I've got the basegame down near there too, but the DLC's are slower for some reason. Can't remember why.
Not trying to take away your victory.

TPFs are generally passworded too, with the same password.

The null pointers and texturegroups are very interesting especially if it fixes the aquarium etc issues. That's been very annoying for me. Just couldn't get my head into working through the file structure etc since it was already done.
User avatar
KFreon
Toolset Developer
 
Posts: 1665
Joined: 16 Apr 2013, 00:57
Has thanked: 83 time
Have thanks: 520 time

Re: Alternate texture replacement tool - MassEffectModder

Postby Alvaro » 28 Oct 2016, 21:23

Hi:

That´s very interesting. The more the merrier!

@aquadran: I´m for one that would love to give a try to your tool but I´m afraid some assitance would be required here. PM sent!

Thanks!
Alvaro
User avatar
Alvaro
User
 
Posts: 83
Joined: 15 Mar 2015, 13:06
Has thanked: 8 time
Have thanks: 12 time

Re: Alternate texture replacement tool - MassEffectModder

Postby FemShep » 31 Oct 2016, 15:36

If you have any info/algorithms for decompressing LZX I'd love to see it. I am trying to crack open the Xbox SFAR files but I can find no actual implementation of LZX compression/decompression beyond quickbms.
Image
ME3Tweaks has modding guides, tools, forums for mods, a modding wiki, and ModMaker, an online mod creation tool.
ME3 Mod Manager, the civilized way of installing and managing ME3 mods.
ME3Tweaks Facebook Page
User avatar
FemShep
Modder
 
Posts: 1101
Joined: 18 Oct 2012, 20:48
Has thanked: 42 time
Have thanks: 76 time

PreviousNext

Return to Third-Party Utilities

Who is online

Users browsing this forum: No registered users and 0 guests

suspicion-preferred