Cutting Hair File Size

[Ambular]

Maxwell's interesting thread on reducing the size of the game installation got me thinking it might be worth posting what I've learned about hair files (which tend to be horrible bloated things full of unecessary duplicate files) and how their size can be dramatically reduced.  Beyond just saving hard drive space, you can cut down the number of textures the game has to load into RAM, which should improve performance as well.

This is not a tutorial.  It assumes either basic familiarity and competence with SimPE and .package files, or the ability to poke around and figure things out yourself without much hand-holding.  If you need them, you can find tutorials and articles that explain the terminology used here over at MTS2.  This also assumes that you have the spare time and patience to sit and putz around with these things for a while.  None of this is all that difficult, but it can be tedious.

Regardless of your level of experience, as always, back up everything before doing anything!

1.  Color bin the hair using Theo's color binning tool in SimPE.  (This usually applies even if the recolors are already color binned by the creator, unless they packaged all their colors in one file instead of separately.)

Load each file in its appropriate color tab and uncheck the gray textures in all but the black recolor (ignore the gray tab, unless there's actually a separate gray package and the other recolors do not include gray textures.)  Uncheck any YA textures as well, then select the Adult texture, go to the Properties/Ages tab, and check both Adult and YA.  In the Package/Options/Output menu, make sure Remove Unchecked Recolors and Compress Textures are checked, and Generate Single Package if that's your preference, and save the hair.

If you have a hair style that comes in a lot of colors, and you want all of them binned as the same family, you can bin them in groups of four (or less) but have all of them use the same gray textures.  Just follow the above directions for the first four you do, leaving the gray textures in the black file, BUT before you save the file(s), go to Package/Options and copy the GUID to a temporary safe place.  Then bin the rest of the recolors using the same process, but uncheck the gray textures for all the files and copy the original GUID to the appropriate field in Package/Options before you save.

This will remove several redundant textures and leave a single Elder recolor in CAS.  (If the hair you're working with is an age or gender conversion of a Maxis hair, see #3.)

2.  This one is more time-consuming, but in some cases it can pare a file down enormously.  Open the hair in SimPE, find any exact duplicate textures (look carefully to be sure they're really identical!) and jot down the filenames of any that are.  Then go look through the Material Definitions and change the corresponding stdMatBaseTextureName references so that they all point to a single texture (just pick one of the names you wrote down, and anywhere you see the filename of one of the duplicates on your list, replace it with that one.  Copy/paste ftw.)  When you are sure no Material Definition is referring to a particular texture anymore, go and delete it.  Continue doing this until no duplicate textures remain.

3.  If you have hair files that are just conversions of Maxis hairs to different ages or genders, you may be able to reduce them even further by changing the Material Definition references to point to the original Maxis textures (filename will be found in the 'File List' tab of the Material Definition) and then removing all the textures in the package.

Note that Theo's Color Binning Tool also has an option to do this, though it's a bit iffy since it's difficult to be sure which references you have and haven't changed (and you also have to be careful in the cases of some later hairs that use, for example, the hairband texture from one color on the hair texture of another): select the texture in the main window, go to the Materials tab, right-click each line and choose Use base texture.

4.  Move the original hair files temporarily to a safe place while you load the game, and check thoroughly in Bodyshop to be sure that your edited version works correctly.  (I say Bodyshop and not CAS, because you can easily check out the YA textures there.)  If it does, remove the originals and any .bak files SimPE has left behind.  Then Compressorize everything.

If you take the time to do all this, you can reduce the sizes of some files by a factor of ten or more.  If you have more than a handful of custom hairs, I recommend going to the folder where you keep hair recolors and sorting them by file size, then going through and paring down the largest ones first.

[Jelenedra]

Is this how you make a hair file meet Pescado's Seal of Approval? Or is there a way to make hair files even smaller?

[Ambular]

Pes probably cuts his files down with a plasma torch.  :p  I have no idea, this is just the stuff I do myself.

[Jelenedra]

Is there a way to do this all via SimPE? I stopped using Theo's tool when Wardrobe Wrangler came out.

Only with the WW, you can remove property sets, but not the textures, so you have to go into SimPE and manually remove extra textures.

[Ambular]

Umm, yes, I believe there is, but I'm not certain I know all of the necessary steps involved to really do it correctly.  Theo's binning tool is much easier.

I'll take a stab at it, though.   Let's see.  If you're going to be mucking around manually anyway, you might just as well start by going through the steps in item #2.  Otherwise you're messing with a bunch of Property Set values for less of a return on file size.  In addition, you would find the Material Definitions for the gray textures in all but one recolor (I'd use black) and point all the stdMatBaseTextureName references toward the gray recolors in that file.  Then delete all the gray textures in the other files, and double-check to make sure that the Family string in the Property Sets and Hairtone XML's of all the recolors is the same (change it if necessary.)  I *think* that should do it.

Also note that you can treat some textures for things that are identical from one hair color to another, like hats and veils, exactly the same way as gray recolors: get rid of them in all but one package and just plug in the texture references in the others.  (The potential downside to this, of course, is that if you make them all up as separate packages and later forget and delete the one with those textures, you bork all the rest.  This is one of the reasons I prefer to consolidate mine in single packages.  Which can, btw, also be done manually in Windows Explorer: open the folder where your recolors are stored, start SimPE, choose File/New, and then select all the recolors you want in that package in the Explorer window and drag them into SimPE's Resource List window.  All the resources from those packages will be added to the new file, which you can then save with the name of your choice.)

[Jelenedra]

Well, I guess it doesn't really matter which binning tool you use, since most of the paring down takes place in SimPE. I'm gonna have to fiddle around with the material definitions, that's the one place I've never mucked with. Property sets are easy once everything is all family'd.

[Ambular]

Well, I guess it doesn't really matter which binning tool you use, since most of the paring down takes place in SimPE. I'm gonna have to fiddle around with the material definitions, that's the one place I've never mucked with. Property sets are easy once everything is all family'd.

Material Definitions work much like Property Sets as far as editing goes.  All you really have to find in there for the purposes of this discussion is the stdMatBaseTextureName field.  The most annoying thing is that in custom hairs, all the texture references are random alphanumeric strings, which makes it a pain to remember which one you're supposed to delete unless you write it down.

[Jelenedra]

So, if you're combining YA and Adult by changing the age to "48" (via SimPE) You can then delete the material definition that used to point to the YA textures?

It doesn't cause borkeness to point Material Definitions for a child mesh to an adult texture? As long as said texture is the same?

[Theo]

So, if you're combining YA and Adult by changing the age to "48" (via SimPE) You can then delete the material definition that used to point to the YA textures?

Indeed you can. The same goes for those makeup and eyebrow packages that insist on having separate property sets, when the the texture is always the same; you can combine the age field (and possibly, gender field) and use just one property set.

It doesn't cause borkeness to point Material Definitions for a child mesh to an adult texture? As long as said texture is the same?

No, it's perfectly safe. It will result in the texture being scaled down to fit another mesh.

[Jelenedra]

Ah, yes, one of these days I need to organize/bin my eyebrow files.

[J. M. Pescado]

No, it's perfectly safe. It will result in the texture being scaled down to fit another mesh.

The texture isn't even scaled down, as the textures have the same resolution and everything, so it is literally, bit-for-bit, an identical texture.

[Theo]

No, it's perfectly safe. It will result in the texture being scaled down to fit another mesh.

The texture isn't even scaled down, as the textures have the same resolution and everything, so it is literally, bit-for-bit, an identical texture.

What I meant was that since those meshes have different surface areas, then the texture being wrapped around will be scaled in runtime, so that it fits nicely and doesn't leave any "excess texture".

[Solowren]

Question: In the first step, when we're unchecking YA and making the adult texture work for it, if we notice that teen uses the same texture as well, can we do the same thing for teen? Like, in the adult texture in the Properties-Ages tab, have adult, YA, and teen checked?

[Ambular]

Question: In the first step, when we're unchecking YA and making the adult texture work for it, if we notice that teen uses the same texture as well, can we do the same thing for teen? Like, in the adult texture in the Properties-Ages tab, have adult, YA, and teen checked?

Unfortunately no.  Adult, young adult and elder hair and clothes are all more or less interchangeable because those three ages are the same size and height.  Teen, child and toddler hair and clothing all use different meshes at different heights, and can't be switched around without editing the meshes themselves. 

[Jelenedra]

Yes, I tried that once, and I ended up with a thumbnail that shows the hair floating over the sim's forehead. I really should go back and fix it one of these days...

[Solowren]

Unfortunately no.  Adult, young adult and elder hair and clothes are all more or less interchangeable because those three ages are the same size and height.  Teen, child and toddler hair and clothing all use different meshes at different heights, and can't be switched around without editing the meshes themselves. 

Ahh, I see. Good thing I didn't try it, then.

[Angie]

Hi Amber!  Tonight I started using your advice to shrink some of my hair files-- I figure if they're small, then I can still afford to keep all 8 kazillion of them.   

So this one group of Peggy hairs turned out to be especially huge, and when I opened up the first of the files in SimPE, I saw that Peggy included not one, not two, but TEN copies of the same texture image in this one file.  They are all the exact same image, but they have different sizes, as you can see in the screenshot below.  So, my question is, how many of these images do I really need to keep?  Would one suffice?  And which one should I choose?  (Okay, that was 3 questions.)

[J. M. Pescado]

That part is normal, it is the mipmapping thing, which creates a series of reduced images like that. It's not your main space loss there.

[Ambular]

Hi Amber!  Tonight I started using your advice to shrink some of my hair files-- I figure if they're small, then I can still afford to keep all 8 kazillion of them.   

So this one group of Peggy hairs turned out to be especially huge, and when I opened up the first of the files in SimPE, I saw that Peggy included not one, not two, but TEN copies of the same texture image in this one file.  They are all the exact same image, but they have different sizes, as you can see in the screenshot below.  So, my question is, how many of these images do I really need to keep?  Would one suffice?  And which one should I choose?  (Okay, that was 3 questions.)

Okay, that's something different.  Those are the mipmap levels for a single image, and they're necessary for the texture to scale properly when you zoom in and out in the game.

Over in the Resource Tree at the left where you see Texture Images (X), the X indicates how many Texture Images you're actually looking at.  It appears there's just the one there, so you can't recover any more space by that method.  The size of the file may be related to the size of the main image Peggy used rather than the number of images.

HOWEVER, if you want to go really in-depth to trim it a bit further (and are willing to risk sacrificing some texture quality--you definitely want to try this with a backup copy,) there's a couple of other steps you can take.  Also, the Nvidia DDS utilities and a good image editing program like Photoshop or Gimp that can handle layers are pretty much necessary for this stuff, so if you don't already have them installed I would recommend not bothering.  If you do, though:

1.  Click Export and save the image (the biggest one) to a folder of your choosing.
2.  Open the image in Photoshop (or equivalent) and resize it to the next level down.  The image proportions must remain the same and it must be sized in increments of 64 pixels (64, 128, 256, 1024, etc.)  Resave the image in the new size.
3.  Right-click the image in SimPE and choose Build DXT.  Choose the file you just resized, choose DXT 5 in the drop down menu, set Sharpen to None, and change Levels to 9.  Then click Build, and when the image has been imported, push Commit.  This will eliminate some file size, but it may also make for fuzzier textures in game.

Another more complicated trick is as follows, if you want to try it.  It will help most if you have a texture image that contains a lot of areas that are not actually used on the mesh (which often happens if the creator re-uses textures originally made for a different hairstyle.)  You can do this and the last one at the same time if you want, just follow these additional steps before reimporting the file:

1.  Right-click the image in SimPE and choose Export Alpha Channel.  Save it to the same place as the image itself and open it in your graphics program.

2.  Copy the alpha image and paste it as a separate layer over the main image.  Select the black area in the alpha (if you can, adjust the sensitivity of the selector so it leaves a wee bit of space around the white parts unselected) then delete the alpha layer, leaving that space selected on the main image.

3.  Fill in the selected area in pure white or black and save the image, then reimport as per the instructions above.

The reason this reduces the file size is because only the parts of the image that are white in the alpha will be displayed in the game, and because large patches of a single color require less space than complex patterns of many colors.  So you may as well blank out the areas that are not used anyway and reclaim a few bytes in the process.

[Angie]

Wow, that is waaaay too advanced for my skill level.  But thanks for the instructions anyway.   

*backs away slowly before she breaks anything

It's a shame, really, because those hair files are over 5 MB each.  Oh well.

[spambi]

Amber --

Assuming you keep the original file name for your hair files, would you be willing to share the files you've fixed? I think a lot of people would love to have these.

I was going to take a stab at fixing some hair files this weekend. I wouldn't mind sharing mine.

[Ambular]

Amber --

Assuming you keep the original file name for your hair files, would you be willing to share the files you've fixed? I think a lot of people would love to have these.

I was going to take a stab at fixing some hair files this weekend. I wouldn't mind sharing mine.

Oi.  I'll have to take a look through mine and see what I've got that might be suitable for sharing.  I tend to also bundle hairstyles up in single packages of all four colors under a similar but different name, so I'm not sure how suitable they would be, but we'll see.

[Jelenedra]

Well, for those that keep their hair packages in a very clear filing system, that would be no problem. Erase the originals, place your file in the folder with the pretty picture.

[CatOfEvilGenius]

Theo's color binning tool is quite nifty, but I believe it doesn't have an *automatic* duplicate TXTR removal feature, right?  If that's correct, then you may be interested in my duplicate TXTR remover project at MTS2.  Coding and preliminary testing should be done in a week or two, so I would love to have some beta testers then, if any of you are willing.

[Ambular]

Theo's color binning tool is quite nifty, but I believe it doesn't have an *automatic* duplicate TXTR removal feature, right?  If that's correct, then you may be interested in my duplicate TXTR remover project at MTS2.  Coding and preliminary testing should be done in a week or two, so I would love to have some beta testers then, if any of you are willing.

Cool beans, thanks for the tip!