More Awesome Than You!

Maxwell's interesting thread on reducing the size of the game installation got me thinking it might be worth posting what I've learned about hair files (which tend to be horrible bloated things full of unecessary duplicate files) and how their size can be dramatically reduced.  Beyond just saving hard drive space, you can cut down the number of textures the game has to load into RAM, which should improve performance as well.

This is not a tutorial.  It assumes either basic familiarity and competence with SimPE and .package files, or the ability to poke around and figure things out yourself without much hand-holding.  If you need them, you can find tutorials and articles that explain the terminology used here over at MTS2.  This also assumes that you have the spare time and patience to sit and putz around with these things for a while.  None of this is all that difficult, but it can be tedious.

Regardless of your level of experience, as always, back up everything before doing anything!

1.  Color bin the hair using Theo's color binning tool in SimPE.  (This usually applies even if the recolors are already color binned by the creator, unless they packaged all their colors in one file instead of separately.)

Load each file in its appropriate color tab and uncheck the gray textures in all but the black recolor (ignore the gray tab, unless there's actually a separate gray package and the other recolors do not include gray textures.)  Uncheck any YA textures as well, then select the Adult texture, go to the Properties/Ages tab, and check both Adult and YA.  In the Package/Options/Output menu, make sure Remove Unchecked Recolors and Compress Textures are checked, and Generate Single Package if that's your preference, and save the hair.

If you have a hair style that comes in a lot of colors, and you want all of them binned as the same family, you can bin them in groups of four (or less) but have all of them use the same gray textures.  Just follow the above directions for the first four you do, leaving the gray textures in the black file, BUT before you save the file(s), go to Package/Options and copy the GUID to a temporary safe place.  Then bin the rest of the recolors using the same process, but uncheck the gray textures for all the files and copy the original GUID to the appropriate field in Package/Options before you save.

This will remove several redundant textures and leave a single Elder recolor in CAS.  (If the hair you're working with is an age or gender conversion of a Maxis hair, see #3.)

2.  This one is more time-consuming, but in some cases it can pare a file down enormously.  Open the hair in SimPE, find any exact duplicate textures (look carefully to be sure they're really identical!) and jot down the filenames of any that are.  Then go look through the Material Definitions and change the corresponding stdMatBaseTextureName references so that they all point to a single texture (just pick one of the names you wrote down, and anywhere you see the filename of one of the duplicates on your list, replace it with that one.  Copy/paste ftw.)  When you are sure no Material Definition is referring to a particular texture anymore, go and delete it.  Continue doing this until no duplicate textures remain.

3.  If you have hair files that are just conversions of Maxis hairs to different ages or genders, you may be able to reduce them even further by changing the Material Definition references to point to the original Maxis textures (filename will be found in the 'File List' tab of the Material Definition) and then removing all the textures in the package.

Note that Theo's Color Binning Tool also has an option to do this, though it's a bit iffy since it's difficult to be sure which references you have and haven't changed (and you also have to be careful in the cases of some later hairs that use, for example, the hairband texture from one color on the hair texture of another): select the texture in the main window, go to the Materials tab, right-click each line and choose Use base texture.

4.  Move the original hair files temporarily to a safe place while you load the game, and check thoroughly in Bodyshop to be sure that your edited version works correctly.  (I say Bodyshop and not CAS, because you can easily check out the YA textures there.)  If it does, remove the originals and any .bak files SimPE has left behind.  Then Compressorize everything.

If you take the time to do all this, you can reduce the sizes of some files by a factor of ten or more.  If you have more than a handful of custom hairs, I recommend going to the folder where you keep hair recolors and sorting them by file size, then going through and paring down the largest ones first.

Is this how you make a hair file meet Pescado's Seal of Approval? Or is there a way to make hair files even smaller?

Pes probably cuts his files down with a plasma torch.  :p  I have no idea, this is just the stuff I do myself.

Is there a way to do this all via SimPE? I stopped using Theo's tool when Wardrobe Wrangler came out.

Only with the WW, you can remove property sets, but not the textures, so you have to go into SimPE and manually remove extra textures.

Umm, yes, I believe there is, but I'm not certain I know all of the necessary steps involved to really do it correctly.  Theo's binning tool is much easier.

I'll take a stab at it, though.   Let's see.  If you're going to be mucking around manually anyway, you might just as well start by going through the steps in item #2.  Otherwise you're messing with a bunch of Property Set values for less of a return on file size.  In addition, you would find the Material Definitions for the gray textures in all but one recolor (I'd use black) and point all the stdMatBaseTextureName references toward the gray recolors in that file.  Then delete all the gray textures in the other files, and double-check to make sure that the Family string in the Property Sets and Hairtone XML's of all the recolors is the same (change it if necessary.)  I *think* that should do it.

Also note that you can treat some textures for things that are identical from one hair color to another, like hats and veils, exactly the same way as gray recolors: get rid of them in all but one package and just plug in the texture references in the others.  (The potential downside to this, of course, is that if you make them all up as separate packages and later forget and delete the one with those textures, you bork all the rest.  This is one of the reasons I prefer to consolidate mine in single packages.  Which can, btw, also be done manually in Windows Explorer: open the folder where your recolors are stored, start SimPE, choose File/New, and then select all the recolors you want in that package in the Explorer window and drag them into SimPE's Resource List window.  All the resources from those packages will be added to the new file, which you can then save with the name of your choice.)

Well, I guess it doesn't really matter which binning tool you use, since most of the paring down takes place in SimPE. I'm gonna have to fiddle around with the material definitions, that's the one place I've never mucked with. Property sets are easy once everything is all family'd.

Material Definitions work much like Property Sets as far as editing goes.  All you really have to find in there for the purposes of this discussion is the stdMatBaseTextureName field.  The most annoying thing is that in custom hairs, all the texture references are random alphanumeric strings, which makes it a pain to remember which one you're supposed to delete unless you write it down.

So, if you're combining YA and Adult by changing the age to "48" (via SimPE) You can then delete the material definition that used to point to the YA textures?

It doesn't cause borkeness to point Material Definitions for a child mesh to an adult texture? As long as said texture is the same?

Indeed you can. The same goes for those makeup and eyebrow packages that insist on having separate property sets, when the the texture is always the same; you can combine the age field (and possibly, gender field) and use just one property set.

No, it's perfectly safe. It will result in the texture being scaled down to fit another mesh.

Ah, yes, one of these days I need to organize/bin my eyebrow files.

The texture isn't even scaled down, as the textures have the same resolution and everything, so it is literally, bit-for-bit, an identical texture.

What I meant was that since those meshes have different surface areas, then the texture being wrapped around will be scaled in runtime, so that it fits nicely and doesn't leave any "excess texture".

Question: In the first step, when we're unchecking YA and making the adult texture work for it, if we notice that teen uses the same texture as well, can we do the same thing for teen? Like, in the adult texture in the Properties-Ages tab, have adult, YA, and teen checked?

Unfortunately no.  Adult, young adult and elder hair and clothes are all more or less interchangeable because those three ages are the same size and height.  Teen, child and toddler hair and clothing all use different meshes at different heights, and can't be switched around without editing the meshes themselves. 

Yes, I tried that once, and I ended up with a thumbnail that shows the hair floating over the sim's forehead. I really should go back and fix it one of these days...

Ahh, I see. Good thing I didn't try it, then.

Hi Amber!  Tonight I started using your advice to shrink some of my hair files-- I figure if they're small, then I can still afford to keep all 8 kazillion of them.   ;)

So this one group of Peggy hairs turned out to be especially huge, and when I opened up the first of the files in SimPE, I saw that Peggy included not one, not two, but TEN copies of the same texture image in this one file.  They are all the exact same image, but they have different sizes, as you can see in the screenshot below.  So, my question is, how many of these images do I really need to keep?  Would one suffice?  And which one should I choose?  (Okay, that was 3 questions.)

(http://i333.photobucket.com/albums/m395/annabelle222/Sims%20pics/PeggyHairSimPE.jpg)

That part is normal, it is the mipmapping thing, which creates a series of reduced images like that. It's not your main space loss there.

Okay, that's something different.  Those are the mipmap levels for a single image, and they're necessary for the texture to scale properly when you zoom in and out in the game.

Over in the Resource Tree at the left where you see Texture Images (X), the X indicates how many Texture Images you're actually looking at.  It appears there's just the one there, so you can't recover any more space by that method.  The size of the file may be related to the size of the main image Peggy used rather than the number of images.

HOWEVER, if you want to go really in-depth to trim it a bit further (and are willing to risk sacrificing some texture quality--you definitely want to try this with a backup copy,) there's a couple of other steps you can take.  Also, the Nvidia DDS utilities and a good image editing program like Photoshop or Gimp that can handle layers are pretty much necessary for this stuff, so if you don't already have them installed I would recommend not bothering.  If you do, though:

1.  Click Export and save the image (the biggest one) to a folder of your choosing.
2.  Open the image in Photoshop (or equivalent) and resize it to the next level down.  The image proportions must remain the same and it must be sized in increments of 64 pixels (64, 128, 256, 1024, etc.)  Resave the image in the new size.
3.  Right-click the image in SimPE and choose Build DXT.  Choose the file you just resized, choose DXT 5 in the drop down menu, set Sharpen to None, and change Levels to 9.  Then click Build, and when the image has been imported, push Commit.  This will eliminate some file size, but it may also make for fuzzier textures in game.

Another more complicated trick is as follows, if you want to try it.  It will help most if you have a texture image that contains a lot of areas that are not actually used on the mesh (which often happens if the creator re-uses textures originally made for a different hairstyle.)  You can do this and the last one at the same time if you want, just follow these additional steps before reimporting the file:

1.  Right-click the image in SimPE and choose Export Alpha Channel.  Save it to the same place as the image itself and open it in your graphics program.

2.  Copy the alpha image and paste it as a separate layer over the main image.  Select the black area in the alpha (if you can, adjust the sensitivity of the selector so it leaves a wee bit of space around the white parts unselected) then delete the alpha layer, leaving that space selected on the main image.

3.  Fill in the selected area in pure white or black and save the image, then reimport as per the instructions above.

The reason this reduces the file size is because only the parts of the image that are white in the alpha will be displayed in the game, and because large patches of a single color require less space than complex patterns of many colors.  So you may as well blank out the areas that are not used anyway and reclaim a few bytes in the process.

Wow, that is waaaay too advanced for my skill level.  But thanks for the instructions anyway.   :)

*backs away slowly before she breaks anything

It's a shame, really, because those hair files are over 5 MB each.  Oh well.

Amber --

Assuming you keep the original file name for your hair files, would you be willing to share the files you've fixed? I think a lot of people would love to have these.

I was going to take a stab at fixing some hair files this weekend. I wouldn't mind sharing mine.

Oi.  I'll have to take a look through mine and see what I've got that might be suitable for sharing.  I tend to also bundle hairstyles up in single packages of all four colors under a similar but different name, so I'm not sure how suitable they would be, but we'll see.

Well, for those that keep their hair packages in a very clear filing system, that would be no problem. Erase the originals, place your file in the folder with the pretty picture.

Theo's color binning tool is quite nifty, but I believe it doesn't have an *automatic* duplicate TXTR removal feature, right?  If that's correct, then you may be interested in my duplicate TXTR remover project (http://www.modthesims2.com/showthread.php?t=318324) at MTS2.  Coding and preliminary testing should be done in a week or two, so I would love to have some beta testers then, if any of you are willing.

Cool beans, thanks for the tip!

I need to know which of these cases is more common:

CASE A: hair recolor with duplicate textures that REALLY ARE duplicates, byte for byte

CASE B: hair recolor with several textures that look like duplicates at first glance, but upon extremely close inspection, they're jittered by one or two pixels so an automated comparison considers them different, even though for all intents and purposes, they're the same damn texture and one wonders why the hell the recolor creator made them this way - edit: AmberDiceless points out there may occasionaly be legitimate reasons for this

I see Case A frequently in clothing and can handle it fine.  I've seen Case B in a few hair recolors.  Case B would require a UI that shows the textures and lets the user select which ones are duplicates.  This would be a pain in the butt to write so I need to know how often Case B occurs.  I mostly do clothing, not hair, so I don't know.  If Case B is infrequent, we can just keep fixing it manually in SimPE like we've been doing.  If its common, I'll write the GUI.

This is the "trivial" case, so its commonality is not really the issue.

I don't BELIEVE it is that common, but one potential cause is that the images are saved in different compression formats, resulting in compressual artifacts that are different. Unfortunately, there is no way a COMPUTER can identify this with any degree of accuracy, so you're probably stuck doing the UI for the nontrivial cases. Also, you were added to Bowels, in case you didn't notice.

Yes, A is the trivial case.  B I have seen in almost every hair package made by a certain creator, so perhaps its just that one person?  Been looking at more hair packages from different creators and different sites.  "Unfortunately", almost all the hair I have in Downloads is by creators who took the time to clean up their packages.  Anyone know a site with lots of hair packages in need of cleanup and dupe removal?

Or maybe someone can suggest some Maxis hair, that when recolored with BodyShop, gives you a recol package with duplicate TXTRs?  The "short cute" hair I tried recoloring with BodyShop had dupe images in Projects, but the final recol package in SavedSims did NOT have duplicate TXTRs.  When BodyShop recolors clothing with a single TXTR used by multiple TXMTs, the resulting recol has dupe TXTRs, but it doesn't make this mistake with hair, at least not for me with the hair I tried.

I did notice I got added to Bowels, thank you.

I'm afraid I can't say with any certainty which case is more common, but in the case of B, just for clarification, there may sometimes be a legitimate reason why the creator did it that way.

When more than one hair mesh group references the same texture, BodyShop will clone multiples of that texture.  If the mesh happens to have, say, five parts, you wind up with umpteen duplicates: one for each group in each age category.  Now, in the case of a simple recolor, the extras may be unneeded (thus the stuff we've been discussing in this thread.)  However, in the case of one hairstyle I modified myself, I needed some of those extra textures in order to apply slightly different alpha effects and details on different parts of the mesh.

Also, with certain hairstyles (but not all of them), it seems as though some mesh groups pick up their display properties from the Material Definition (TXMT) of the Sim's skin, while others will use the TXMT of the hair itself.  When that happens, it may be necessary to color one texture a bit differently so it blends with those using a different TXMT.  Otherwise you'll get an unsightly two-tone effect.

These odd cases are the reason I said to take a close look at all the textures and make certain they really are identical before you delete them.  A visual way to confirm that would be highly advisable.

An alpha version of the extraTXTRremove program is now available for testing.
It will automatically remove extra TXTRs in hair (and clothing) recolors.
If you want to test it and have any comments or bug reports, please put them in the MTS2 thread below.
I don't want to clutter this thread. ;)

extraTXTRremove alpha version available for testing here (http://www.modthesims2.com/showthread.php?t=318324)
.

The other useful form of reduction is to repack the Adult/YA propertysets together into a single propertyset and eliminate it from the BINX/3IDR/MATDs entirely, saving some more space that way. I also do this. I am ruthless like that.

Combining Property Sets and TXMT's is easy, but it's a bitch trying to figure out which 3DIR goes with what.

You may have noticed that hairs in some stuff packs postedlinked in MATY were packaged using the 3 keys method. This makes 3IDR/BINX identification a lot easier, as well as making the XHTN obsolete. Old custom hairs may also be repackaged using this method.

Actually, I hadn't, and I confess I'm not entirely sure what you just said.  XD  Can you link me to any discussions or explanations on the topic?

For example, take a look at any of the packages in "Fixed TS2 Store Hairs.zip", linked here (http://www.moreawesomethanyou.com/smf/index.php/topic,13036.0.html). You'll find a number of differences:

• The XHTN resource has been replaced by a clothing-type COLL.
• The property sets no longer share their 3IDR's with the BINX. In fact, there are now twice as many 3IDR's (plus one for the COLL). But this is not a problem, since they're small in size.
• The property set references are no longer offset by 1. In the old packaging, PSET #1 was referenced by 3IDR #2, and so forth. (*)

(*) This explanation may have left you even more bamboozled, but in a nutshell, there are three types of resources involved, and each one has its own 3IDR resource (associated by Group and Instance).
It runs more or less like this:
— Begin with a BINX; find a 3IDR with the same id.

— On the BINX, get the 'binidx' value; this is the index of the line within the 3IDR that points to the COLL.
— The COLL (plus its own 3IDR) is used for cataloging purposes. You don't want to mess with, or delete this resource; leave it alone.

— On the BINX, get the 'objectidx' value; this is the index of the line within the 3IDR that points to the property set (GZPS).
— Get the GZPS; find a 3IDR with the same id. This is the object in hand.

Having understood that, we can now start with a property set we wish to delete, and work our way back to the resources that are referencing it, and delete them too. Easy, huh?

The old packaging (alas, the most common), combines a non-straightforward association of objects, with the requirement of 3IDR editing - in the event you'd wish to delete a resource.

Combining adult / YA GZPSz and hunting references in 3IDRs sounds like something it would be useful to automate.  While I'm not yet very familiar with hair, I have chased instance / group references in clothing and makeup.  If you all would like me to take a crack at automating the processes Theo and Pescado discussed, I would be glad to.  Theo, if you already have plans to incorporate this into your color binner, let me know, so I don't duplicate your efforts.  Either way's fine with me.

My idea for now would consist of a converter to this packaging method; it would accept a normal [hair|clothing|makeup] package, and then split and rearrange the 3 object types.

But I think you should finish the TXTR cleaner as a standalone tool first; It's a lot of work as it is now, and the more you add features, the more it becomes easy to miss the point, very much like what happened with the - now obsolete - binning tool. Also, because I need the TXTR cleaner NAO! ;D

I agree with you that adding the 3 object repackaging feature should wait until the texture cleaning is all done and thoroughly tested.  I've put it on my list'o'stuff'todo.  The repackager will probably end up being a separate program.

I also need it now, that's why I wrote it. :D  If you can live with no GUI, then the command line, single package version is available now:
http://www.modthesims2.com/showthread.php?t=318324

I will add a GUI and the ability to look for duplicates across multiple files, check back in a week or so.  I'll also add the optional ability to tell the program to remove user selected textures that are "almost identical", since this sometimes occurs in hair files.

I don't consider the binning tool obsolete.  I still use it all the time.  Granted, jfade's Wardrobe Wrangler can also be used to bin hair, but the binning tool is frankly quicker and has a more straightforward interface that I much prefer (sorry jfade!)  Those extra features occasionally prove very useful.

(I'm aware of the issue with families and hair placed in the Custom bin, but since I seldom mess with putting anything in that bin anyway, it's not that big a deal to me.)

Can the technique described in part 2 of the original post be applied to skins as well? (Summary: Find duplicate textures and note their filenames. Go to Material Definitions and change the corresponding stdMatBaseTextureName references so they all point to a single texture. Delete the now unused duplicate textures.)

I have several skin packages where, for example, faces and scalps are shared across all genders and age groups. Could I delete all but one of the duplicate textures, and then point the material definitions to the remaining texture? Or would that just bork the package completely?

Yes, you can (and should) do this with skins.  I have some CC skins created this way (not mine, I'm not that artistic).  If you want, try my automatic duplicate texture remover (http://www.modthesims2.com/showthread.php?t=318324).  Back up your files first.  I think it works fine, but it's still in testing.

Hope I'm not being too off topicy discussing skins here, but the question seemed related.

Yep, as Cat says, this is totally a good idea.  Just be very careful to verify that the files you think are duplicates, really are duplicates; sometimes they look very very similar but have subtle differences in shading that can leave seams on your Sim.  Cat's duplicate texture remover is probably the way to go, but make sure you test the cleaned up skin before you permanently delete the original.

Thank you, Cat and Amber, for your advice.

I am eagerly anticipating the duplicate texture remover, but I'm holding out for the GUI. Command lines make me a little nervous. :)

The extra TXTR remover now has a GUI.
You can download it from MTS2, link below, it's in the FIRST post.

extra TXTR remover program (http://www.modthesims2.com/showthread.php?t=318324)

Hair File Cleaning, and One Gray, Not Four... on a Mac

Mac users who make custom hair can bin with Wardrobe Wrangler, but WW does not remove extra grays.  Theo's SimPE plugin can't be used on a Mac.  So I plan to write a Mac app for cleaning hair files.  It'll do what Theo's plugin does.  Theo, or anyone else, if you plan to do this already, just let me know.  I like plans where I do no work. ;)

Someone (Theo?) please tell me if the procedure for changing 4 grays to 1 gray is correct.  I did everything below manually with SimPE (not Theo's binner) on some hair with extra useless grays.  The updated hair worked fine in game, but I want to know if I missed anything, or did something that might break stuff without me noticing.

preliminary stuff...
1) Remove duplicate textures.
2) Combine A / YA
3) Make sure all 4 hairs are properly binned.
4) Make sure all 4 hairs are in the same family.
5) Give all hair correct genetics.  What are the correct genetic values for black, brown, red, blond?  I don't mean hairtone, I know about 1,2,3,4 and elder 5.  I mean, in XHTN, what does genetic=0 mean?  Should it be 0 for all four or not???

remove extra grays (unless you really, really want different shades of grey for different hair tones)
6) Nuke duplicate grey textures in the black hair and update TXMTs accordingly.
7) In the blond, red, and brown hairs, do the following
  a) nuke the GZPS for gray hair
  b) nuke all grey hair TXTRs
  c) find all TXMTs for gray hair, they have "emhair" or "efhair" in their name, note their Instance numbers
  d) find all 3IDRs that reference the Instance numbers of gray TXMTs from c), note the 3IDR Instance numbers
  e) nuke all BINXs with the same Instance numbers as the 3IDRs from d)
      nuke all 3IDRs from d)
      nuke all TXMTs from c)

Optionally, the user may want to townify the hair.  I assume this can be done?  Will changing the creator number to zeros do it?
Might make a separate "townifier" app that townifies hair / eyes / clothing, rather than bundling that function into the hair cleaner.

The user may also want to put the hairfile name in the tooltip, but that could also be its own separate app.
Might make that app if jfade doesn't include that function in the next version of Wardrobe Wrangler.

Is there anything else a hair cleaner should do?

The genetic binding is done by setting the hairtone guid in the property set to the value of the family guid in the XHTN.

The default XHTN's - which can be found in \TSData\Res\Catalog\Skins\Skins.package - have these 'genetic' values:
Black = 1
Brown = 1
Blond = 2
Red = 2
Grey = 0
Other = 0

Since hair binning binds your property sets to the default XHTN's, the custom XHTN seldom has any value of its own, except when custom hairtone guids are involved.

Yes, that's more or less how my tool worked, but since then I realized that the TXMT's were addressed according to this procedure:
7)
  a) Open GZPS; Fetch the values of all the 'overrideNresourcekeyidx';
  c) Open 3IDR with same instance as GZPS; Use the previously fetched indices against the resource list, to find the affected TXMT's.
Pseudocode:


The problem is that some hairs don't work like that.
For example, look at this hair package (http://modthesims2.com/download.php?t=95533). The elder/grey GZPS has instance=5; so you open 3IDR #5, to find this:
The addressing scheme here is indices 00-04 used by GZPS #5, and 05-08 by BINX #5 that links to GZPS #4(!)

If you delete 3IDR #5, not only will you break the link to GZPS #4, but also this hair will stop working in game (or at least it did when I tried).

So my advice is to tread carefully around these resources. Besides that, the quick and dirty way to remove the extra grey hairs is to set the 'hide' flag on the property sets. :D

Theo - Thanks for that.  You're right, can't match up GZPS, BINX, and 3IDR based just on the Instance # of the GZPS.  Have to double check reference numbers in the 3IDR.  Last night, I thought I was removing BINX and 3IDR for YA, and that broke my elder hair.  Good thing I back up. ;)

BINX: These are for catalog entries in CAS, right?  Are they also used when sims change appearance in a mirror?

3IDR: These are a collection of references to other resources, used for grouping them together for each age, right?

From what I've read, the concensus seems to be that the above two resource types don't have to be removed for unused ages.  Ideally, I would like to nuke all unused stuff, but if there isn't a 100% accurate way to determine which BINX and 3IDR go with which age, I think I should leave them so I don't break stuff. 

edit: 3IDRs have references you can chase, but from your example, it looks like I have to check almost every single line in a 3IDR to figure out what all it goes with.  I could write code to do it, maybe write code to make new 3IDRs that only reference one GZPS.  Maybe.  Not sure it's worth the trouble.

If you do that, and also change the Material Definition references and remove redundant textures, doesn't that clean up the vast majority of the resource-hogging and catalog-cluttering junk?  Everything else should be just 'invisible' data.  Seems like you'd have to have a very large number of custom hairstyles before it would become a serious problem.

That's correct, for example if you look at the interim packages that are created by BodyShop (the ones in the \Projects folder ), you'll find that these are the same as the final packages, except that they lack the BINX resources, and thus cannot be displayed anywhere but the Create Content section of BodyShop.

Period.

What really matters here are the main resources (BINX, GZPS, XTOL, etc...), that sometimes need to reference other resources.
They may do it directly, as seen in this example taken from BINX group 0x7F5AC619, instance 0x0CDF5EF4 (in \The Sims 2\TSData\Res\Catalog\Bins\globalcatbin.bundle.package):

Or, they can store those references on another resource (3IDR), and just use the list index:

How can you find a 3IDR that belongs to a given resource? Just match its group and instance numbers.



If total extirpation is not possible, then that may be considered a reasonable stump, yes.  ;D

When total extirpation is possible, you mentioned it's better to make the new file in the new format, with COLL and no XHTN.  Would such a file still be base game compatible or not?  If COLL is a collection resource, I'm guessing not BS compatible?

Ah yes, I forgot my previous statement:
is not entirely correct, since it turns out that these two are unrelated to each other. The XHTN may still be needed, if you're using custom hairtone guids, while at the same time, you can use a custom COLL resource instead of the 'magic COLLs' (http://modthesims2.com/showthread.php?t=268853).

I made a small diagram of the resource linking here: http://theos.chewbakkas.net/files/hair-problem.svg
See also if you agree with my proposal: http://theos.chewbakkas.net/files/hair-solution.svg

Regarding base game compatibility, I believe this referencing scheme is compatible, but it's just a matter of testing it with the base game starter (http://www.modthesims2.com/download.php?t=250555), if you want to be sure. ;)

Yup, I have Numenor's AnyGame and base game programs.  I'll test, just thought I would ask first, in case the answer was "it can safely be assumed to be compatible".  Probably better to be paranoid, assume nothing, and test.

I'm totally in favor of automatically splitting resources so that all resources associated with a certain age can be nuked without fear of breaking other ages.  That diagram is quite handy.  Right now, I clean hair packages by hand and with your binning tool.  Once I'm totally comfortable doing that, I'll write an automatic cleaner for Windoze and Mac.  After the duplicate TXTR remover is all finished, that is.  There's still some textures it doesn't recognize as identical, could be because of different compression schemes, as Pescado suggested.