Welcome
Ladies and Gents:

These forums are now closed and registration disabled.

Please join us at our new forum on Proboards. Our hope is that these new forums are more stable, provide more and better features, and allow continuation of the project forums in a safer, more secure, long term environment.

me3explorer.proboards.com

--The ME3Explorer Team

Done research: Properties

Technical research related to the structure of Mass Effect game files.

Done research: Properties

Postby WarrantyVoider » 13 Oct 2013, 18:03

Proper Property Explanation


What are properties?
Properties are static values, saved into pcc objects. They define, well, "properties", of an object. An object here always means an export entry, because only these contain data, that you can see in hexpreview. Alot objects consist only of properties! Here an example object

Image

Is that everything inside objects?
Well obviously you cant (well can, but wouldnt) define an entire game from properties, you have "binary" data, to save storage space. This includes textures, meshes, sounds, etc... In this example above you have f.e. a Texture2D object. This is a "class" that has not only properties but binary data. The order is always:
1. Index/Stack(more on this later)
2. Properties
3. Binaries
Now even if the object has no properties at all, then the last (and with no other, only) property is a "None" property. After that instantly follows the binary data, if there is some.

Now where does it all come from?
Image

from the objects data! the stuff you see in the hex. Also notice the Flags, it does not contain a "HasStack" flag, so the data should start with an index, 4 byte. Im not completely sure what it is used for, but usually you just have to keep it. To make more sense of the data press the "Interpret" button and then "Start Scan".

Image

dont be overwhelmed by the markings :D first off, if you click on such a line like "TextureFileCacheName", you get instantly send to its starting position in hex view. As you can see the first property starts at 0004, that means the first 4 bytes are the "index" I talked about above. Now lets take them apart. Every property starts like this:

Name : 64bit, 8byte long name table entry
Type : 64bit, 8byte long name table entry
Size : 32bit, 4byte int that defines that size of data and 4 bytes 0, they are called "indexes" but afaik never used
Data : well, data, as long as defined in Size (with exceptions)


So lets try this out:
first name is in hex:
22 00 00 00  00 00 00 00

there are never more than 2^32 names in name table, so we can ignore the last 4 zeros. then this is little endian, so to convert to normal hex writing:
0x12345678 becomes 0x78563412

or in this case
0x00000022 = 0x22 = (use calc.exe^^) 34

ok so we know that the name is the entry 34 in names list, lets look it up...
Image

ok found it! the same goes for type! now as I said there are exceptions to "size". Usually you can trust the value to skip any unknown properties, but in this case its a "StructProperty", which has additional 8 bytes for its name ("Guid") and then as stated 0x10 = 16 bytes data. And exactly after that comes the next property. Well fine so far.

What types are there?
None
This is an exception in size, as its only the 8 bytes for its name, no type/size definition
StructProperty
This is an exception in size, it has additionally 8 bytes for its subtype name, so add data + 8 to skip it, data can be complex and nested (another list of properties or just a list of values)
IntProperty
Very basic property. 4 bytes data contain value
FloatProperty
see IntProperty
ObjectProperty
see IntProperty
NameProperty
see IntProperty, data 8 bytes long
BoolProperty
This is an exception in size, it says 0 but has actually one single byte data, which is either 0x00 or 0x01
ByteProperty
This is an exception in size, has additional 8 bytes for subtype name
StrProperty
data starts with 4 byte int, which contains length of string, multiply with -1 and read so many widechars afterwards (read char, drop zero, read char...)
StringRefProperty
see IntProperty, is a reference to tlk file
DelegateProperty
should have correct size, could maybe(?) contain complex data

ArrayProperty
ah, and the most hated one, the array property. this gets an extra section. because it can contain another list of properties, a "list of lists" (complex properties), more subarrays or simply a list of raw values. in any case, the size is correct and the data starts with 4 byte int, that holds the count. but you cant always use (size - 4) / count, to get the element length, because complex elements could vary in size! So yeah, nested properties is not fun, (especially recreating them, see dialog editor :P) and unless you have access to unrealscript (which we havent) you cant say what "type" of elements you will get (has to do with class inheritance). So you have to test it^^ Btw, the interpreter simply tries to parse it as list of lists of properties, after the name-type-size-pattern and if no result will read them 4byte wise

and last but no least, the start of the binary: here the marked none property and the start of the binary
Image

So what was that again with the HasStackFlag
ah yeah, almost forgot. If the object has this flag, the start of the Properties is not after the first 4 bytes, but instead 0x1E = 30 bytes!

Image

I hope this clears up all future questions about properties!

greetz WV
always backup your files!
mess with the best or die like the rest!
"I tried everything!" - "mkay, please list that..." ; please dont pm me for help, we have a help section

WarrantyVoider has been thanked by:
User avatar
WarrantyVoider
Emeritus
 
Posts: 2270
Joined: 22 Aug 2012, 11:33
Has thanked: 480 time
Have thanks: 626 time

Re: Done research: Properties

Postby AmaroK86 » 13 Oct 2013, 21:03

Wow, you find another connection! GREAT! Do you have any idea of what those 30 bytes are for? And have you checked that the size is always 30 bytes length with that flag?
My multiplayer ID: AmaroK86_64
User avatar
AmaroK86
Emeritus
 
Posts: 66
Joined: 22 Aug 2012, 12:51
Has thanked: 25 time
Have thanks: 36 time

Re: Done research: Properties

Postby WarrantyVoider » 13 Oct 2013, 22:53

well I just explained my code here, tell if it doesnt work for something^^ well the stack is used for unreal script, is called "execution stack"

http://udn.epicgames.com/Three/UnrealScriptStates.html#State%20Stacking
http://udn.epicgames.com/Two/UnrealScriptDebugger.html#Call%20Stack
https://code.google.com/p/unreal-debugger/wiki/CallstackPanel

from what I can see (not sure if this is correct):

Ar << StateFrame->Node;		//  4 bytes
Ar << StateFrame->StateNode; // 4 bytes
Ar << StateFrame->ProbeMask; // 4 bytes
Ar << LatentAction; // 4 bytes
Ar << StateFrame->StateStack; // 6 bytes?
Ar << Offset; // 4 bytes
SerializeNetIndex(Ar); // 4 bytes, the usual index (called netindex)
//=30 bytes


/** index into Outermost's NetObjects array, used for replicating references to this object
* INDEX_None means references to this object cannot be replicated
*/
INT NetIndex;



/**
* Contains information about script execution at the main stack
* level. This part of an object's script state is saveable at
* any time.
*/
struct FStateFrame : public FFrame



/** Current state node */
UState* StateNode;



/** Mask for all functions the current state is probing */
DWORD ProbeMask;



/** Current latent action, 0 for no active latent */
WORD LatentAction;



/** List of the currently pushed states */
TArray<FPushedState> StateStack;
struct FPushedState
{
public:
UState* State;
UStruct* Node;
BYTE* Code;

FPushedState()
: State(NULL)
, Node(NULL)
, Code(NULL)
{
}
};



INT Offset = StateFrame->Code ? StateFrame->Code - &StateFrame->Node->Script(0) : INDEX_NONE;
Ar << Offset;



INDEX_NONE = -1


a bit messy reply, but I think you get what I mean :D

UnObj.h / line 1671

greetz WV
always backup your files!
mess with the best or die like the rest!
"I tried everything!" - "mkay, please list that..." ; please dont pm me for help, we have a help section

WarrantyVoider has been thanked by:
User avatar
WarrantyVoider
Emeritus
 
Posts: 2270
Joined: 22 Aug 2012, 11:33
Has thanked: 480 time
Have thanks: 626 time

Re: Done research: Properties

Postby AmaroK86 » 16 Oct 2013, 10:16

Helpful as always man!!! I'm gonna do a "brute force check" on all .pcc to see if it's always 30 bytes size.
I believe you, I'm just trying to make sure there aren't any other flags that could interfere with the properties search

AK86
My multiplayer ID: AmaroK86_64
User avatar
AmaroK86
Emeritus
 
Posts: 66
Joined: 22 Aug 2012, 12:51
Has thanked: 25 time
Have thanks: 36 time

Re: Done research: Properties

Postby The Fob » 16 Oct 2013, 11:08

Really nice summery on properties. Easy to follow and with lot's of pictures. Great work WV. I'll link this in the sequence modding tutorial so that people get a better idea of what they are looking at when they dive into HEX editing.
User avatar
The Fob
Modder
 
Posts: 702
Joined: 08 Oct 2012, 04:37
Has thanked: 242 time
Have thanks: 212 time


Return to Technical Research

Who is online

Users browsing this forum: No registered users and 0 guests

suspicion-preferred