More Awesome Than You!

The Whaterizor?
The Compressorizer! (It eats your sims for breakfast!) It's a little utility I made that shells out to the nifty little dbpf-recompress program that benrg made and uploaded to Bowels of Trogdor here (http://www.moreawesomethanyou.com/smf/index.php/topic,8279.0.html). Full credit to him for the awesome compression utility, I just made a pretty friendly, but ugly, interface to connect with it and make it easier for you to compress all your package files to a much smaller size!

But... Why?
There's no reason to not compress your files. EA/Maxis does and SimPE can (sometimes) but there was no easy way to do it, until now that is. With my tests, I compressed 73 MB of files to 37 MB. That's pretty much 50% smaller right there. Those of you with massive downloads folders will probably see even more benefit as this will help you make room for even more downloads! :P

OK, I'm Interested Now. Where do I start?
You'll need Microsoft's .NET Framework, version 1.1. This was written in C#.NET, and the source code will be released under GPL once I'm satisfied that it works properly. Since this is a debug version, it has all kinds of extra debug files with it, so the final version will be even smaller as it won't have those files along for the ride. Just extract the files to a folder somewhere, and start TheCompressorizer.exe. From there, it should be pretty self explanatory.

Important Stuff
As always, since this program's in beta, you'll want to BACK UP YOUR FILES before using it, especially since this program will change just about everything internally in a file that you point it at. Let me know about any problems by posting here, but I don't expect there to be any. It seems to work really well. :)

As it is now is pretty much the final product, though I will consider any suggestions about changes to the UI.

Enjoy! :)

Download: http://www.djssims.com/index.php?category=21
Screenshot: http://temp.djssims.com/beta/compressorizerscreen.jpg

Edit: Forgot the file links, heh.
Edit 2: The program has been officially released, get that version instead. :)

Does compressing actually benefit you and reduce your loading times?  I thought the game still has to take time to uncompress files again in memory, no?

Oh... and thanks (to both you and benrg), this is great! :D

Ste

Not actually sure if it has an impact on loading times, but if all the EA files are compressed, I don't see why we shouldn't have our downloads compressed. *Shrug*

I'd be interested to see the results though if someone did a benchmark type test on things and found out what impact, if any, compressed files have versus normal files.

Works in Linux (Mono v1.2.3) though it doesn't look that great and of course the process fails (it doesn't try to open the exe in Wine after all).

(http://img107.imageshack.us/img107/5018/thecompressorizerqi4.th.png) (http://img107.imageshack.us/my.php?image=thecompressorizerqi4.png)

Is this safe for textures? Many textures, if compressed in SimPE, result in corrupt packages.

Seems safe for textures!

Before: 1036MB
After: 489MB
------------------ -
Delta: 547MB

Very impressive.


A possible side-effect... my game loads significantly faster. I haven't made any configuration changes other than using this compresser, and it saved at least 2 minutes off the startup time.


----

If you're still taking feature requests: clear list would be nice (remove all files from list).
And an auto-focus, that if selecting a file, its before and after are highlighted to see the compression of any specific file.

----

Also, while "recursive" is the correct term, labelling it "scan subdirectories" may be more clear for all users.

Eh, the pherverstool and SimPE both use recursive.  I'd imagine most people using any sort of tool are used to that terminology at this point.

Anyway, if one were really attempting to accomodate the less-knowledgeable, one would have to use "subfolders" -- because your typical Windows user is not going to think in terms of directories.

How would this work on other package files, such as those in the lot catalog or neighborhoods? Or is that borrowing trouble?

Seems to compress them without issues. I wouldn't run it on the install folders though.

The fellow who first posted the library jfade used for this said that it knocked 100 MB off his base-game installation and everything seemed to work fine.  That said, if you want to experiment with the base files, back everything up first!

jfade, it screwed something of mine up, but then I tried to do over 2 gigs at once, and also I know there were a couple of files I had that didn't like being compressed in SimPE and made things crash.  I'll try it again tonight, incrementally, and see if it works better that way.

Compressing base game install folders seems to work without problems, although I won't be surprised if I can't install the future Seasons patch 2 now. Shaved off about 75MB from a TS2+everything install.

Seems to work fine for me. Only tested it with my objects, yet. I noticed one file was compressed from 5000 kb to about 500. I only wish I knew which file. Can't you make that files list a little bigger so that I can see the filenames? Now I will go and test my loading time  :)

I second that request--a resizeable window for the filenames would be awesome!  (Better yet, one that lists only the filename with the folder path displayed separately at the top, rather than the entire path of every file.)  Also, a "Stop compressing" button would not go amiss.

It seems to be working fine though.  Got most of my files compressed now, one folder at a time, and the game runs without a hitch; I'll finish up the rest as soon as I can and post a final report.

This is VERY shiny!

So, if a babygeek such as m'self wished to try this on, say, her Sims2 DL folder, the best way to do it would be to

1. backup DL folder (to external drive, or, at the bare minimum, b/up the folder(s) I am about to compress)
2. run Magic Ensquishifier tool
3. attempt to load Sims 2
4. lather, rinse, repeat until all folders in DL folder are de-embiggened?

Yes, no, "don't touch it, it is too shiny for the likes of you"?

That's about the size of it.

ETA: Probably a good idea to delete the groups.cache file from your EA Games/The Sims 2 folder before you run the game for the first time, too.

Oh, jfade, one other small thing that would be really nice is if the directory listing in the Add Files to List dialogue would remember the last folder you ensquishified...

I just compressed the DL folder itself.  All the contents were squishinated.

Well, that seems to work well for some people, not so much for others.  I just recommend folder by folder because that's what worked for me.  :p

And I for mine.  :-P

Working on all those features now. :) Thanks for the feedback so far! :)

Mine's at 13G, so I think I'd be courting trouble.

This won't cause other tools to fail, such as SimCat / S2PCleanInstall / S2HDetect, right? I can't see how it would, but if the ensquishinator is altering files....     

One of my files went from 2775 to 45. I fear for this file....

I'm now going to see if compressing a huge Downloads folder (19,570 files, 6.13 GB) will do anything to loading times.

SaraMK - look closer. The numbers do not scroll right, the left column is one more down than the right (or was it the other way? Anyway they're not in sync).

Load times definitely are affected by this tool. I went from ~6 minutes to less than 2.

That was the only file I had in a subfolder, so it definitely went down to 45 as long as the program is reporting correctly.

Huh, weird. But possible... I had one file that went down from over 2000 to less than 100 as well. It was riddled with text based resources and nothing compressed (text compresses really well). The file definitely works still.
It is a property set overrider I have for my own use, which I never compressed because it is too large to do so in SimPE (took more than 20 minutes before it completely hung, so I left it be). The compressorizer took far less time (about 90s or so).

I'm very frightened of this, but I'm going to give it a try.

Gwill, don't be. I reduced my downloads folder size by 60%, reduced load time by 3 minutes. I haven't tried it on my lot catalog yet, butI I'm hopeful there too.

I love debloatetized files.

Ohhhh...

This means that I can reduce my dowload folder and fit in more stuff!  Thank you!

So, if we was making new items could we compress them before offering them to the unsuppecting public?  Or would it be better to leave the files alone, and point folks to the tool?

I compress my hacks, since it improves loading speed (as proven by this tool!) and means less file size.

Looks like I shaved over a Gig off my downloads.  From 2,5 to 1,4 (those 2,5G includes pictures of all downloads).  Haven't tested to see if it's blown up my game yet, but I'm optimistic.

Edit: Remind me to never openly express optimism.  ::)   Game crashed at lot loading.  Will do some selective compressing in stead, to start with.

Ooh, you know, I've loaded the game, a neighborhood and CAS, but I haven't tried a lot yet./me goes to try it.

13GB?  Ye gods, woman, how many hours of load time is that?  O_o

A quick check shows me that Clean Installer still reads the compressed files just fine, so I imagine the other tools would as well...

My dl folder went from 1.34 GB to 799 MB. I did it one sub-folder at a time. There was one file that went from 2kb to 0kb that I was a little worried about.

The first time I ran my game it actually seemed to take longer to load but after that things went more smoothly. Probably I should have deleted the groups cache first but never even thought of it. I did load and even play a house for awhile so things seem to have gone really well.

It would be helpful to be able to see the file name but I see you are working on that already so that's cool.

 :-X  ;)

People who are experiencing crashes: if you have the time, please try to narrow down the problem. If you can find a single file which causes a crash when it's recompressed, and post it here, I can probably fix the bug.

The command-line tool verifies that all of the recompressed files match the originals, and backs out the changes if they don't, so it should be safe, but I may have overlooked something.

Very cool, thanks benrg, glad to see you still around and thanks again for making this new library, it's quite awesome! :D

Used the tool today and cut my Downloads from 2.68 gigs to 1.33 gigs. I compressed all but my hacks and face templates. Have entered and played three different lots and have had no issues whatsoever. Game load time is down slightly, from approximately 6-7 minutes to about 5.

Well, I haven't managed to narrow it down to one file yet, benrg, but by process of elimination it's starting to look as though the problem lies in my Object Recolors folder.  I'll see if I can sort out which file(s) it is...

My problem was probably related to the fact that I did the wole 2,5GB in one go.
If you don't mind, I'll delay giving it go folder by folder (I have a lot of subfolders) until the UI is a little more refined.

25 minutes. :)

On a laptop.

Running other programs at the same time.

Booyah!

What happens if the compressorizer is run on packages that have already been run through the compressorizer the first time?

My downloads go through a routine. When downloaded, they are put into the NEW folder. After proving them in game, they are moved to the appropriate folder such as objects, clothing, etc.

So say I ran the compressorizer on everything I have today that is not in the NEW folder. Later, I continue to download items, adding them to the NEW folder. My question is should I run the compressorizer on new items after they have been tested, but before moving them to their final destination, or can I keep doing what I have been doing and then run the compressorizer periodically on final destination folders that would have a mixture of both compressed and non-compressed items in them?


Is it doing the pictures? I am currently running the compressor on my Hair folder. Each style has its own subfolder which includes mesh, textures, and a pic. A small handful also have readme.txt where I wrote a note to myself such as "blonde color contains the mesh". A few hairstyles such as Maxis styles that were made available for other ages do not contain a picture.

Anyway, the Hair folder properties show 662 subfolders with 3907 files. However, the Compressor is showing 3275 files being processed.  3907-3275=632. So I ask myself, where are these 632 files? Since there are 662 subfolders and most contain a pic, I assumed that it was the pics left out. (If each subfolder contained one pic, then there are only 30 files left to account for. I'm assuming those files would be the styles that were missing pics).

So have I deducted correctly or are my assumptions wrong? The window that shows which files are being processed only shows Mydocs/owner/Sims2/ then the rest of the file is unreadable as the window cuts it off. I can't find a way to see the entire file name to actually know why there are a different number of files being processed than the folder is reporting having.

I dunno if I should be concerned, but during compression, I had some files get BIGGER after the compression...

Were you just looking at the UI, or did you actually check the files? I noticed this in the UI while it was running also, but when I actually checked the files afterward, none were bigger after compression.

@ magicmoon: If the file is already compressed it won't hurt it if you run the program again and it will only process .package files so any other files in those folders are ignored.

Mine did that too (stating the files got bigger). One went from 24 to 1700 kb. I chalk it up to the program getting confused, because the file definitely shrank afterwards.  As for re-compressing, it's fine. I did that by accident on a couple folders and it just ran through them again and didn't make a change to them.

Agreed. Apparently there's some sort of magic that happens that won't let a package get more squishified than it's suppsoed to (yeah, there is actually some computer process that does that, but it's Greek to me, so ...'magic".)

Also, I have successfully run this on my lot catalog and neighborhood files...not as much compression, but a noticeable improvement in speed. My game now loads in under 6 minutes (used to be 10), and maybe it's just me, but it seems that lots load a tad faster from the neighborhood screen.

Of course, it could be the drugs.

I got my 1.20 GB folder down to 644 MB. On the downside, my game takes a little longer to load. It used to load in 3 minutes, now it's just under 5. I'm not sure if game performance is any better, but I think I noticed lots loading a little faster, too.

Beta 2 is now available! Grab from link in first post or here:

http://temp.djssims.com/beta/CompressorizerBeta2.zip

Some of you mentioned files that got compressed to 0, but actually that just means the file was compressed to be less than 1 KB. It rounds down and not up so anything less than 1 becomes 0.

Hope this new UI helps, let me know if there's any other issues. :)

Not that it's a big deal, but now I get:

Ah, heh, I bet I know what that's caused by. I'm going to fix another bug Amber mentioned (again, minor) then upload it again with the same filename.

EDIT: Fixed Amber's problem for sure, I think I've fixed Dizzy's... Get the updated one from the same link.

Have you deleted your cache? It has been noticed that it can lengthen load time unless this is done.

I was a little worried about using this on some of the painting recolors (many of them saved in 32 bit color). Thought it might effect how they look in game. It didn't. They look the same as before.  Presquoosh, with close to 6 gigs of much needed crap my game loaded in roughly 15 minutes.  Post squoosh with 2.73 GB I can't say I've noticed a big change in load time either way. But I'm starting to suspect a highly bloated walls file (which didn't compress all that dramatically) as being the big culprit. 

Yeah, that was the first thing I did. It's not a big deal, just an extra minute. Um...I'm in shock at how long some of you have to wait for the game to load! You can fix a meal in that time, probably eat it, too.

Before compressing it, I had enough time to go downstairs, boil water, make a coffee, and take the cup back upstairs with me before the game began the loading music.
Now I just have time to yell "coffee!" down the stairwell.

Well, actually, the magic is more basic: It just blindly unpacks and restuffs everything. If it was already packed before, you'll just get the same output back out. Nothing interesting will happen.

There's reason to believe it may load faster simply because most of those files are already compressed, albeit poorly, so by compressing them better, less crap needs to be snorted into memory, resulting in less time spent reading the hard drive.

At this point it's just a few cosmetic problems. Otherwise it works fine with Mono.

(http://img366.imageshack.us/img366/4809/dbpf2bvm5.th.png) (http://img366.imageshack.us/my.php?image=dbpf2bvm5.png)

Interesting... I'll see if I can make a few tweaks to fix those things and then hopefully it'll be all spiffy looking. :)

How did you get it to actually run the dbpf-recompress.exe file? Wine?

I've squished some files with this, worked great, but as of yet haven't had a chance to play/check them out.

Questions: Am I reading this thread right: we can run this on the program file/EAgame blah blah folders and it will compress those? And then: what is in a lot file that needs to be compressed?

I was thinking this program basically compressed the texture and GMDC parts in a package--but I should be thinking it compresses ALL of a file(s)...similair to how Rar or Zip does?

On my old laptop, load time for the game was at least a half hour. I had no custom content whatsoever. It wasn't until I got this computer that I learned that those pretty cinematics shown in the credits actually existed in-game, and that mirrors could show reflections.

Cool toy ^^

I don't have the most extensive download folder these days (3279 files) but I ran this thing on my desktop and it went from 1159mb to 795mb.

I don't know if this has anything to do with anything, but I noticed that the files it took most out of were mainly in my Peggy folder. The bulk of the files are down to between 1/2 and 1/4, while a few are even down to 1/10-1/20!  :o

Can anyone say bloated?

Anyhoo, the game boots fine etc. Not sure if it's faster as I never thought it was slow before, but it didn't get slower either. Haven't got time to play the game right now but I'll edit this message if I run into anything worth mentioning later.

I'm sooooo pleased with this!  I cut my d/l folder from 2.08 GB to 1.04 GB!  Wheeee!  And loaded the game about 3 minutes faster (off a 10 min. load - I have a crappy comp).  Played lots, commercial and residential, with plenty of custom stuff, and no problems at all.

One thing I did notice with the Squishinator is that when you click "Clear file list" the progress bar doesn't reset till you begin again.  At first I thought it was a problem, but after the first time I realized it was just cosmetic.  Still, thought you'd like to know. 

Okay, here we go.  I cut my Downloads folder from about 2.24 GB to 1.38 GB and compressed my EA Games/The Sims 2 folder as well (didn't stop to check how much additional space I saved there.)

A couple of curiosities:

1. The game repeatedly choked after my large Object Recolors folder was compressed, until I split it into three parts and compressed them separately; then it worked fine.  Recombining the three folders afterward had no apparent ill effects.

2.  The game appears to load more slowly with the compressed Downloads folder in there, but that could possibly be my mind playing tricks on me, because it now appears to hang for some time at the opening copyright message and then runs through all the silly UI text really quickly right at the end.  Unfortunately, I didn't think to time my pre- and post-compression startups. :/

3.  When I ran the game with the compressed The Sims 2 folder before putting my Downloads back in, it started up like lightning!

I'm now weighing the possible risks versus benefits of compressing my base game files.  If I decide to go through with it I'll let you know how it works out, but in the meantime, thanks, jfade and benrg!  It's a great program, and something that was badly needed.  :D

Wine has been properly integrated with Mono in Ubuntu since Feisty beta, so that wasn't the issue. When I really took a look at getting that executable to work, I quickly discovered that it was failing because of a missing dll (msvcr71 - which you can easily just download).

Ah, ok. I'm just now running the upgrade to Feisty (Literally, right now it's upgrading) so I wasn't aware of that. Before I'd always just installed Mono via the package off the Mono site. Should I use synaptic to get it from the official Ubuntu sources instead?

Repo is the way to go although it's still just the v1.2.3, and that also goes for Wine (I just learned that the latest versions of Wine don't handle Mono integration properly).

It's not your mind playing tricks on you, I'm having the same hang up at the opening.

Hmm, good to know it's not just me.  I'm just not sure whether the entire startup sequence is *actually* taking longer, or if it only *feels* like it's taking longer because I'm staring at an unchanging screen for so much of it...

I add the files to the list and hit Go button but I get a pop-up that says MSVCR71.dll is not found, the application couldn't start etc.  ??? I couldn't close the pop-up no matter how much I click ok or close buttons, I had to restart the comp  :-\ Do you know what causes this?

spikesminx: Just a guess, but I think it's caused by a missing msvcr71.dll file.

Make sure you have .net 1.1 installed. Instructions are not optional!

If you are sure you already have the framework installed, just copy Msvcr71.dll from another directory on your system to the directory where the compressorizer is installed.

I have .net 1.1 installed. It's still the same.

Which windows version?
Does copying the DLL help?
Do other .net applications work?

I have Windows XP. I didn't try any other .net versions  :-\ And where should I copy the .dll file? *feels stupid*

Search for Msvcr71.dll on your system. This DLL file comes with most .net applications. If you have SimPE installed, it could be there.
Once found, copy it to wherever you unpacked the compressorizer archive.

The current version is just a beta, so I guess Jfade hasn't included all the dependancies (required helper files) in it. Most people have Msvcr71.dll somewhere in their path anyway.

It works perfectly now  :D Thanks!

Hrmmm, weird considering it works just fine without that file copied in the same directory. It should work normally anyhow....

When I go to release, I'll make sure to include the right file. Speaking of release, other than UI things, is there anything else I need to change about it? I think it's fairly straight forward and complete myself, but what do you all think? :)

I tried this programme today (beta 2 version), and it compressed everything quite nicely, but didn't seem to acknowledge subfolders, meaning that I had to separately go into each folder, as well as every subfolder, in order to compress all the package files in my Downloads directory. Also, I had to close and re-run the thing after doing each individual folder. Browsing the file-menu each and every time got to be quite tedious. Is this a feature or a bug? Is there any way to streamline the process a bit?

Wow, this is a great little tool. I downloaded a ton of cars from Sims2Carsource and ran them through this. Files went from from being huge (7, 8, 9, or even 10 MB) down to around 2 or 3 MB. Teh awesomez!!!1  ;D

Check the "Recursive" box and re-load the folder and it will recognize subfolders.

Well, that was quite a large 'duh' on my part. Is there a way to have the Recursive box ticked by default? That would certainly save a step or two (or 200 ;)).

Yup. And I'll fix the having to quit bug too while I'm at it. :P

Would it be possible to have the ability to add more than one directory to the list... I was going out for a while the other night and decided to do a few dir's at once but couldn't add another separate dir to the list after i added the first dir...

Ta for the shiney!!   ;D

I found something that people would probably want to know.

When I go on my little download binges, I usually create folders for everything, then collection files using the Collection Creator. Since I've been using this tool, I started compressing them first. Well, I couldn't figure out why the Collection Creator wasn't working, but then it dawned on me...try it uncompressed. Needless to say, it worked when the files were uncompressed. So, if you are planning on making collections, do it before you use this tool.

Are you sure you have the latest version of the collection creator? Version 2 (Electric Boogaloo!) fixes this problem. :)

I hadn't noticed that it had been updated so "baaaa" for pointing that out.  ;D

After running this on my folder for downloads, my startup time seemed to increase a bit, but after running it on both of my EA Games folder, the game seems to be loading much faster. Admittedly, it did take a good chunk of time, but it was definitely worth it.

I just want to report back.  I compressed my downloads in (large) chunks, loading the game imbetween to check that everything was working.
Everything is now compressed, and the game is running fine, all lots are loading and nothing is missing or borked.

Glad to hear it's all working. I've done some final tweaks, and will upload them in a bit. Then, it's off to release once you all approve this last set of tweaks. :)

Changes made. Get the new version here or in the first post:

http://temp.djssims.com/beta/CompressorizerBeta3.zip

you broke it!

EDITED all the crap out since it fixed itself.

Seems to be working with me, and the file that's "missing" is in the zip... Try re-extracting it all into the same folder? :)

well I don't know what happened....but when I clicked on it this morning (just to check), it worked fine.

User error. (shut up, Pescado.)

It works great, nothing to report. I'm glad that it clears everything now, too.

So far, nothing seems to be borked as a result of using this programme. It didn't speed my loading time any, but gaining more space on my HDD makes using it worthwhile (and this experience has prompted me to back up my files, which I don't do as often as I should). Thanks, jfade, for all your hard work.

Worked fine for me (I did subfolders at a time), shaved half a gig off the downloads.  Since I had less than 2G to begin with, that's a decent proportion.

I have noticed a bit of a slowdown in the game loading.  It generally would take between 5-6 minutes to load the game, and I'd say it's around 10 minutes.  I know this for sure because I boot up the DS and play a bit while the game loads, and I've gotten more DS time since I compressed.  It's not THAT much longer, though.

Did you clear the groups.cache file? This has been shown to help with load times after compression if you do it. :)

I'm noticing a trend. People who had relatively small downloads folders to begin with are loading slower and those with larger folders are loading faster. Interesting. My problem is the opening. It just sits there for almost 3 minutes sometimes 4 before it actually starts doing anything.

That's something that's been happening to me for a long time, and I haven't compressed anything yet.
I think you have a point in the "smaller downloads folders take more time, and larger downloads folders take less time" though.
But I can't say anything with certainty, as I am not willing to test it as a "smaller downloads folder" person. (My downloads folder is somewhere around 2GB.)

I have great expectations from this tool. For now, I'm just trying it out on my Hair folder.
And if it works: tomorrow, the clothing folder! Raaagh!
*marches away with an army on horseback*
*and by army, I mean something like three guys and a horse*

My downloads folder was about 550mb - does that qualify as small? It's down to 300-ish, and loading times have gone up a little (10-30 seconds - load time is now about 3mins).

Is there more than one groups.cache that I should be clearing? I deleted the one in my documents and the speed didn't improve. Still, it definitely did what it said on the box, my downloads have been almost halved :)

I've not compressed all my game files yet though. Will those increase the speed? Again, I'm not too bothered, what bothers me is the space this game takes up. After clearing my downloads folder down from over a gig to 550mb, it made loading so much faster that any increase from this tool really doesn't matter in comparison.

This reduced my downloads folder from 6493MB to 3877MB a saving of 2626MB! This took a little under two and half hours to compress.

Load time before compression 8mins from desktop to active Sim and 5mins after compression (groups.cache deleted first).

I'm downloading a mess o' lots to replace those that mysteriously vanished. Once I clean install 'em, where should I look to ensquishify them with this tool...or should I avoid messing about with lot files? I assume "lot catalogue" in the relevant hood folder is the right place. If not, or there are others to look for, LMK?

Also: is it safe to compress occupied houses? I don't plan to court trouble by doing so unless someone has done so, as I have enough mystery lot issues ATM.

Yes, 'lot catalogue' is the place where your in-game empty lots are stored.  I compressed mine, and haven't had any problems placing new lots.

great tool :D can I use it to compress the package files in program files? the ones under TSData? I'm sorta short on space ???

Yes, you can. It won't save you too much space, but it will speed up loading times. It really only saves about 25 MB per expansion compressed, and something like 40 MB for the original game.

ok thanks :D I'll run it through my program files then :)

Hmmmm, my Downloads folder must be terribly subpar: it went from 280 Mb to 261. Not a big change. I've been trying this program out on the My Documents/The Sims 2 folders one by one, but thus far have only seen an 8mb difference. Am I doing something wrong?  ???

Also, when I try to add the Music folder, no files show up in the list (recursive was selected at the time).

ETA: Oh, I see. Now that I've selected the My Documents/EA Games/The Sims 2 folder, everything is getting selected. I may notice a bigger difference now.

Well, I'd say it works. I started with a 17.1 GB download folder (yeah, yeah, I know, it's a sickness - I need Downloader's Anonymous and if you're wondering, load time was about 45 minutes) and after compression... 7.45 GB. Holy crap. My jubilation may be short lived, I still need to actually load things up to make sure the game doesn't puke and it may not make a lick of difference, but damn, I like that new number. I'm hoping this makes a difference because I was pretty much at the point with my game where I had to delete things if I wanted to add anything new (okay, maybe the game is trying to tell me something) to prevent crashing.

One positive, however, I had a HUGE skin file that I loved but couldn't justify keeping it in the game at 49 MB in size. I'd searched high and low for a replacement but no luck. I ran it through this utility and lo and behold, it came out at just under 3 MB in size. I can happily put it back in game now, so even if the rest of the downloads folder ends up being a bust (cross your fingers that it's not) I got something out of it. Any thoughts what had it so horrifically bloated to begin with?

One other note though, I had the same problem with the dll (I forget the exact name, it's listed earlier in the thread) file needing to be copied in the same file as the Compressorizor program. No big deal, just figured I'd let you know. No other problems (though I may be back tomorrow once I've had a chance to load my game up). THANKS for the great program!

Is it a standard skin file or a default replacement?  Sounds to me like someone edited the thing in SimPE for one reason or another, rather than just running it through a standard BodyShop edit, then imported the textures at the highest resolution and neglected to compress them afterward.  Though 49 MB seems excessively large even taking that into account.  Is it a really detailed photo-quality type skin?

how often does it borke things nowadays? Should I wait until after my weekly monthly "my system needs an overhaul" system backup before running it, just in case? Or is it safe enough to use pre-backup so I can save a disk or two when I do back up the system?

I have had nothing bork with this utility.

I'm too scared to try it on my Program Files Sims 2 folders. Are you sure it won't mess up ? Has it been tested? Thanks for the program

MOD EDIT: The only tag worse than [ move] is [blink]. In fact, [ move] is worse.

Hmm, just possibly, people have been talking about testing it all through this thread. Don't download it now though, they've re-hacked it and uploaded a new file which will blow up your game.

..Buh?

Downloaded yesterday or the day before and now I'm.. well.. a bit worried.. by this statement. lol

I already downloaded that one yesterday used it, ran my game, and it played fine. Maybe you are just doing something wrong did you delete groups.cache?

I did Sims 2 folders in the My Documents and My Programs files, squishinating everything down by over 10 GB (most of that was from my scary downloads folder, of course) and no borkedness noted. It shaved about 7 minutes off my load time, too. I'm a satisfied customer. As for that one 49 MB skin file I noted earlier, nothing special about it - no photo realness, not even anatomic correctness or "artwork" on it - it's just a darker skintone than S4, so no idea what caused its bloatedness. All better now, so whatever.

By the way, am I the only one who read Witch's post as extreme sarcasm?

KLGFCG, some people don't have their sarcasm filters turned off. (Which is odd...because isn't EVERYTHING Witch says laden with extreme sarcasm? (Hey..new Olympic sport!)

I have found that I don't gain MUCH space from running it on my lot catalog or neighborhood packages, but it doesn't hurt.

Sometimes the forum default titles are extremely accurate.

No.  ;D  Sometimes I am helpful and nice. See? Oops, you blinked, must have missed it.

It worked fine for me on recursive. I finished running it last night. The first time I ran it several days ago, it crashed, but that was my fault for moving things while it was running. So last night before bed I ran it on the entire downloads folder with recursive checked. When I woke, it was done.

Before: 4.10 GB
After: <1 GB

It would have had to seen and processed the subfolders because I don't have any files whatsoever in the main Downloads folder. All items are subfoldered.

ETA: Now my post just sounds stupid because the post I replied to has been deleted.

sorry ;) for some reason I got the recursive to work now, although I didn't do anything differently.

It seems to compress objects to less than half of the original size, hair and clothes not that much. I can't wait to test my loading times.

This would undoubtedly be because BodyShop automatically compresses textures (using, apparently, an inferior compression scheme?), while SimPE doesn't unless you do it manually.

Thank you for this-I'd just removed about 1GB a few hours before finding this as it was about 3.5GB! So its gone from 2.20GB to 1.39Gb, which is fantastic. Also means i don't have to keep placing or removing my themed downloads when am or not playing those neighbourhoods!  ;D

I guess I am just unlucky. I downloaded Beta3, spent several hours compressing stuff, folder by folder. Then started up my game. But, it wouldn't start, it wouldn't get past the blue screen. I ended up having to replace My documents folder with the backup copy :/ And yes, I did delete groups.cache before I tried to start the game.

I'll give it another try later in the week.

i'm amazed at how much compression some of my downloads will take, for instance a 9712 kb furniture object (a bed from avalon sims) compressed down to 519 kb.  about 50% of my files are compressing 50% or more. my 4esf folder compressed from 237 megs to 79 megs, whoa.  so after compressing everything in my downloads folder it went from 4.59 GB to 2.18 GB, and when i loaded my game it was just over 2 minutes faster than my usual 8 min load time.

i think i've found a conflict though, between it and Delphy's Download Organizer.  after compressing and running my game with no probs, i deleted a bunch of clothing files, and then when i exited my game i ran Delphy's program to locate the clothing meshes so i could delete them.  while it was scanning my downloads folder i got one of 2 different error messages (http://katgyrl.com/sims2/possiblecompressorconflict.gif) on nearly every single file.  so that sucks, since Delphy's program is essential to me, i always end up with so many orphaned meshes and they're a drag on the game.

Removed a bunch of crap before squishin', from 11G to 9.5G. zCompressed to 5.5. (W00t!)

Haven't had time to test the game load time (or see if it will load at all) yet.

This is surprisingly good at compressing big skin textures. It does every bit as good a job as pkzip. I really got to hand it to Ben. He really knows his onions when it comes to stuff like this.

Amberdiceless mentioned in Delphy's Download Organizer thread that some have reported that this compression program is causing errors with his program.

http://www.modthesims2.com/showthread.php?t=227925

Anyway, I compressed my hair folder before running his program and a handful of files (10) error-ed that had not error-ed before compressing. All the files that threw an error were in the hair folder (3898 files). I have now finished compressing my entire downloads folder and a great many new errors are now occuring with his program, files that scanned fine before compressing. Once a file throws an error with his program, that same file will always throw the error.

My next experiment will be to take all the files that are erroring, replace them with my backups, and then compress them again to see if the same files will always error when compressed.

ETA: By the way, my game is running fine, even the files that are showing errors with his program. Two of the hairstyles that have started throwing errors in his program are in use in the game and are working fine.

I might be having a problem with this program and the clean installer. I packaged a lot that was furnished, then opened the file to remove the phone line controller. When I tried to save it, the package got corrupted. This happened with several lots. I may be doing something wrong, though this is a normal routine for me, but could someone else try it and see if they get the same end result?

It affects only SHPE chunks that are in files compressed with the compressoriser, but as for why, I don't know.  I'm currently trying to track down the issue, so expect a DDO update in the next few days. :)

Edit: Okay I figured it out.  On *normal* Maxis/SimPE compressed SHPE chunks, the DDO works fine.  It is only on the SHPE chunks that are run through the Compressoriser that it borks.  The reason is that there is an extra byte inserted after the length of the sgResource string but before the actual string.

For example:

Normal compressed SHPE ref, in Hex view in SimPE: 66 42 72 61 6e 64 69... etc.  This is 1 byte for the string length (102 bytes) and then immediately followed by the actual string (Brandi...).
For a compressorised SHPE ref, in Hex view in SimPE: 95 01 62 72 61 6e 64 69... etc.  This is 1 byte for the string length (149 bytes) and then a 01 and then followed by the actual string (brandi...)

A string in the SHPE ref chunk *should* be encoded with 1 byte for the length and then the actual string.  I'm guessing Simpe either ignores this extra byte, or does something else with it - I can't find anything in the SimPE code specifically mentioning it. 

I can put a catch in the DBPFClass code to check if the first byte of compressed string is 01 and ignore it, but I'd like to know why it's there in the first place.  Perhaps my understanding of the actual string storage is not complete, but the code for the DDO works with basically every other file you can throw at it *except* the compressorised ones. :)

I started working on my Dls with Delphy's tool. My dls folder waqs at 5.5 gigs and 12000+ files. Delphy's tool IDed 1100 or so as being orphans. That wasn't exactly correct, some files were fine while others were orphans. I got tired of going thru 1100+ files and just deleted them :)

Then I started in with the Squisher. I squished each folder separately. It Size went to 2.8 roughly. Then tried to stat my game. Only, I had forgotten to delete the groupcache files. Soooo I quit (hard shutdown) and went back to see what it had done to the files, curious if it had decompressed them. And yes, it had! But only some -probably because I shut down in the middle of the load.

So I resquished but I got lazy and did all files at once. Squished to 1.8 this time! Deleted thecache files, added 1 or 2 minutes to load time :( but other than that all is well in my game. ANd the files stayed squished this time.

It's not choking on every compressed file. I'm throwing approx 50 errors now that the Downloads folder has been compressed, but there are 19,353 files in there, most of them package files, which the program is handling just fine (or so I assume since they are categorized correctly when the program finishes scanning).

Has anyone else noticed that duplicates are not spotted by the clean installer when you check items downloaded with a house to the compressed ones in your download folder?
About half of them seem to be spotted now. Yet when I throw them in their allocated folders, I am asked to replace or not.
I'm trying the simpe scan folder now, but that takes more than an hour. And I have to boot after re-installing the normal sized fonts, or I can't see the scanfolder controls.

I originally posted this reply in the "New DBPF library" thread, but I'm copying it here since the original message was in both threads.


SimPE uses System.IO.BinaryReader.ReadString() and its BinaryWriter counterpart for most string serialization. These do not use a single byte for the length -- they use a weird base-128 encoding documented here: http://msdn2.microsoft.com/en-us/library/system.io.binarywriter.write7bitencodedint.aspx (http://msdn2.microsoft.com/en-us/library/system.io.binarywriter.write7bitencodedint.aspx). In this encoding a length of 0x95 is serialized as 0x95 0x01. I'm almost certain that this is your problem, not anything to do with compression.

I don't know whether this is a bug in SimPE or whether it actually is the encoding used by The Sims. If the former, you may have to use an ugly workaround anyway.

Slight issue, may have been mentioned before. The Collection Creator doesn't play nice with this. In fact, when opening folders in CC, I get nothing or one gobbly-gooked named file. Ick. I really need to clean my recolors and decorative objects, too. Any short-term suggestions?

If CC = Collection Creator, I think jfade has an updated version of that on his site that plays nice with this.

Thanks. New version reads the files fine, the program just didn't do what I wanted it to. Still useful, though.

What do you mean by this? I'm slightly confused now. :P

I had wanted something that would let me see what it was and delete the file if need be. This did half, and I couldn't see the real file name to cull them. I still use it for the intended purpose of collection creating, though.

Ah -- you want Paladin's Sims 2 Categorizer.  You can find it in the programs section over on http://www.simwardrobe.com/.

My downloads folder got squished from 7.9 gigs to  3.1 gigs. This program is awesome! I just wish it wouldn't create problems with Delphy's Download Manager which was awfully useful. Is there anything I can do, so I can still run Delphys program, plus have my squished downloads folder?

From what's been told so far: bothering Delphy to update his program would be, uhm, something like a waste of time, wouldn't it?

Delphy's posted a couple weeks ago sounding like he was going to look into it, unless I'm misreading and it was an "I give up".

And thanks, jsalemi. Too many "C" Sim programs and "-izers" on my desktop. Got confuzzled.

Until this program is able to work with Delphy's download organizer, it might be a good idea to take all compressed downloads out of the folder and just use the organizer for newly downloaded files before compressing.

Some more questions to add to the FAQ ;D:

Can I run the Compressorizer on folders I've already compressed to make them even smaller?

I accidentally ran the Compressorizer on a folder that I'd already compressed! Will something bad happen?

I'm pretty sure the answers to both of these questions are "Once the file is compressed, it can't be compressed any smaller. So the utility will just ignore it.", but it would be nice to have that spelled out in idiot-friendly text in the manual.

Also, a question/bug report. I've noticed the program doesn't work with the standard Windows shortcuts of holding down CTRL to select multiple individual files, and SHIFT to select a block of files. How difficult would it be to add these in? I have all my Downloads sorted in folders depending on where they came from, and even with Recursive selected, it's still requiring something like 60 individual selections to compress everything if I don't feel brave enough (or am too impatient to wait) to select my entire Downloads folder in one chunk.

I also have a question about using this along with the Pets CC Updater. I noticed that, when updating my CC for Pets, the PCCU decompresses the files (which I've already compressed using the Compressorizor). Does this mean that I will have to then re-compress them once I'm done updating them for Pets?

I know I'm doing things backward, but I'd decided not to bother updating my CC and then changed my mind after I'd already compressed all my files.

Actually, the Compressorizer will just decompress and then recompress anything you throw at it, so if it's already compressed, it's just a waste of time, no harm done.

As for the multiple select, you mean with the open file dialog? Hmmm... I'm not sure you can toggle that to allow multiple selections since I've never tried. I'll have to experiment and see. :)

Yeah, you will have to recompress them after updating them. I'm working on updating the updater (ha) to make it have more options for updating things for Seasons, make it faster, and add auto-recompression of things once it's saved them. This is, of course, low on my to do list at this point as I'm sort of focused on the Wardrobe Wrangler. (Yes, it's actually in alpha at this point, and I'm working on getting it to the beta stage. So far so good! :) )

Thanks for the answer, jfade. I'd already discovered that I did need to re-compress the files but it's good to have confirmation. It definitely helps to do things in the correct order (update, THEN compressorize). I now have a system in place. :)

Please, please don't lose focus. Don't waste your time on boring old things like eating, sleeping or having a life either. :D

If, umm, you know, you want some beta testers... I have a truckload of crap to test it on.

Actually, I wouldn't mind a few more testers. You can hop over to http://forum.djssims.com/ (http://forum.djssims.com/) and then register and apply to join the Beta Testers group there. To do that, just register, and then under the description of the beta testing section there's a link that you click that takes you to a page where you apply. :) Anyone is welcome! :)

Cool, I'll check that out when I get home on Wednesday.

I ran the Compressorizer on my downloads folder and went from 1.83GB to 1.15GB. I randomly checked my lots in game and everything sees to be running well.  Thanks.

A thought occurs to me- any reason why compressed downloads wouldn't work in Stories? After all, if I'm putting in a set of default replacement skins, they ought to be compressed...

Should work fine, unless EA did something stupid and EA like, such as changing their own algorithm. :P

Try it and see. :)

Earlier in the thread, people talk about compressing their base game files or "EA GAMES" folders. I'm not sure if they're talking about the Program Files\ stuff (including going into each xpac's directory?), or the Documents and Settings stuff, or both. Would anyone be kind enough to tell me some good candidates for base game compression, or point me to a place I could find out?

TashaYarrr, Re-read the first post. Then head to jfade's site and read all the info there. Everything you need should be in those two places.

Tasha.
You'll read all that to learn that some folks compressed all their files and some folks did just their saved game files and some did just their downloads files. There was a variety of results ranging from some games load faster, some load slower. Most people had no problems with crashing. I'd say just about everyone was very happy with it and I don't say all were because I'm not sure--but maybe they were all happy with it! The program is great. I did just my DLs. I tried the others but they seemed to be compressed as is so I by passed that part. It shaved off 2 gigs! I load in about three minutes now.

Just wondering:

I remember reading through this before, and thought i read something about a problem with re-categorizing objects after compressorizing, but cant find it this time.

Has anybody mentioned anything about this?

Yes. And if you are using SimWardrobe's categorizer, then the algorithms are not compatible and it will not find your compressed downloads (not that I've ever actuaully tried it--second-hand knowledge for the win!). Better recategorize with that tool before compressing. You can still use SimPE, though.

okay thanks.

i just compressorized my 5 gig downloads folder after deleting my cache files, and it added 4 minutes onto the total load time.
replaced it with my old non-compressorized (trying saying that when you're drunk!)] folder and its back down to nine minutes..

Not strictly true--Categorizer finds most of my compressorized files, though it does appear to miss a few.


Deleting the cache files will also slow things down the first time you run the game afterward, since it needs to rebuild the cache.

Deleting the cache files will also slow things down the first time you run the game afterward, since it needs to rebuild the cache.
[/quote]

yeah i suspected that, however i tried it 3 times after, and it was still taking ages the third time ???

Simply put: Your results may vary.

ok, just a quick heads up:
second time i've compressorized my 7 gig sims 2 folder, and the desktop-to-lot loadtime went from 9 1/2 minutes to 14 and a 1/2.
same as last time couple weeks ago.

strange. kudos anyhow to you jfade, it seems to just clash against something in my setup i suppose.

i wonder if the people having longer load times could have something in common, such as what type of OS they run, or perhaps even how you sub-directory your downloads? Or will it be 'just one of those things?'