Merging mods to increase Performance!

[Anach]

I have 7 large official content .package files so far. I had to split the Build/Buy Mode stuff into two merged files, clothes into three, and hair into two because when I tried to import them all into one and save I got an out-of-memory error. I kept sets together, and am maintaining separate merged files for user-created objects, clothes and hair. I haven't attempted to merge any hacks.

That was my main concern - whether I'd eventually try to merge too much stuff into one package.  Do you (or anyone) happen to know what the limit is on that?

The game uses some .package files close to 2gig in size. My store cache is around 600MB, and my largest custom merge is around 250MB. I don't know what the actual limit is.

[Beehoosis]

So, probably, there is a limit on package size that can be created by the merge process in S3PE, and that limit is much lower than what the game can actually handle.

Good to know.  I think you guys have answered all my questions.  Many thanks.

[Indiasong]

Thank you for this solution. I've decided to try, but after n°3, I can't find import as dbc. All I have is import from file or import from package, and if I choose that, I have save as DBPF.
What am I doing wrong?

[Baarogue]

Thank you for this solution. I've decided to try, but after n°3, I can't find import as dbc. All I have is import from file or import from package, and if I choose that, I have save as DBPF.
What am I doing wrong?

Are you sure you're getting the latest version directly from the link in point 1 of the tutorial in the OP?

[Matildarose]

I'm getting the 'outofmemory' and 'Manual merge of XML files required'. Is the latter error normal? I get them even on store stuff, so that discounts badly made packages- maybe it's detecting the 0x73E93EEB _xml files?

It seems to pay to have smaller compressed files at this stage- I went hogwild compressing all of my store files into one huge file, and... that gave me the out of memory errors, of course, which doesn't surprise me. I've packaged a bunch of my hacks that were replacements or tweaks to the game, and they work (I used the 'nointro' hack to give me a signal that it was being read).

[AloeOwl]

Thank you for this solution. I've decided to try, but after n°3, I can't find import as dbc. All I have is import from file or import from package, and if I choose that, I have save as DBPF.
What am I doing wrong?

P.S: Next time, please read through the thread, because this was answered before.

I had the same problem. When you open the link given here you have two options: "View all files" and a box under it listing the contents of the download. You need to click the link of the s3pe download from the *box* not from the "view all files" > "download". By clicking the latter you get the new "public" version.  But when you click on the link in the box you get the the version you need. Also, you should uninstall your current version before installing the test version.

[Anach]

I'm getting the 'outofmemory' and 'Manual merge of XML files required'. Is the latter error normal? I get them even on store stuff, so that discounts badly made packages- maybe it's detecting the 0x73E93EEB _xml files?

It seems to pay to have smaller compressed files at this stage- I went hogwild compressing all of my store files into one huge file, and... that gave me the out of memory errors, of course, which doesn't surprise me. I've packaged a bunch of my hacks that were replacements or tweaks to the game, and they work (I used the 'nointro' hack to give me a signal that it was being read).

You may need to try merging smaller number of files, as it might simply be you are running out of RAM during the import. Once a successful import is complete, you will usually get that manual merge message for all those 0x73E93EEB files. For now I simply delete them, until someone comes up with a good reason I shouldnt. However, if you are getting other types of XML conflicts, then you most likely have a conflicting mod, which tries to edit the same file as another.

[Indiasong]

Thank you for the answer, they had the same name. And I read the thread, twice. When the window with all the files marked for deletion appears, does saving automatically deletes them, or do I have to do it manually?I suppose so, since nothing is crossed out when it has saved. And what is having to merge xlm manually?

[Anach]

Thank you for the answer, they had the same name. And I read the thread, twice. When the window with all the files marked for deletion appears, does saving automatically deletes them, or do I have to do it manually?I suppose so, since nothing is crossed out when it has saved. And what is having to merge xlm manually?

Yes they get deleted when it is saved. Generally merging manually would be bad. If its any other type of xml than those I described, then you should not be merging those particular mods which share xml, and not be using them together in game, unless you know how to merge the xml externally using an editor.

[Indiasong]

They were patterns, and had many crossed out, and the next line was always a 00000 instance, that I marked for deletion too.
And the other folder I did, paintings all by the same creator, had quite a few too.

[jezzer]

This is probably more of a question of s3pe, but I got an error saying: "Expected magic tag 'DBPF'.  Found 'DBPP'." while importing files as .dbc.

The program just stops there then, and the error message doesn't tell you which package causes the error either, is there any simple way to fix that?

As far as figuring out which package caused it, you have to watch the archive being built as it displays the names of the files being added.  The last file name displayed before the error halts the program?  Whatever file comes AFTER that one in your folder caused the error, and as Pescado said, it's probably going to be an uncompressorized store file that got converted to a package.

Of course, having said that, someone will come along after me and point out the giant flashing "this file caused it" button that I've missed all along. 

[simfan]

Can some one make new dcb files for the new store updates for those of us with mac? It would be much appreciated.

[Littlecat]

My thanks to you Anach.  My game had slowed to such a crawl playing was nearly impossible.  After following your instructions game performance has improved 90 percent.  The load time went from 10 to 15 minutes to 3 minutes, all of my CC is showing up and running fine and game play is on par with a fresh install without any CC at all.  To be honest, I wasn't expecting much from this experiment, only to be astounded at the difference it has made.  If you would be so good as to indulge my curiousity, I do have one question.  When merging files is there any reason one could not group them according to website or creator?  e.g., GOS.package or aikea_guinea.package.  It seems to me further precision might come in handy should one run into a problem after a merge.

 
             

[captaincaliena]

I downloaded the Fancy Fashions set that is posted here and ran it through s3rc. I then converted to .package using Delphy's tool, and loaded the newly created .packages into s3pe as per the instructions in this thread.  That all went fine up until I got the "Manual Manage of XML Files Required Message."  However, instead of it detecting the 0x73E93EEB files, it's detecting a bunch of files marked 0x0166038C.  Currently, s3pe has those files marked for deletion.  Can someone please tell me if this is ok or normal?  I'm still pretty new to learning what all these different file types mean.  

EDIT:  I went ahead and let s3pe create the file.  It saved in .package format and I plopped it in my "Packages" folder.  The game loaded up just fine, but the items were not there.  I am stumped.  I am also probably missing something really basic that I'm going to slap myself for once someone points it out to me...

[GlamGirl]

Thank you for posting this information!

 I am using it to organize my files (much like the poster above by creator) but one thing I noticed is that in some cases, the files output are larger than the files input.  For example, I had a folder of Fresh Prince cars at 38mb, the output file of the .dbc file created is 48mb.  Another folder I had was 284mb, the output file is 387mb.  Any idea why this may be happening?

[Moryrie]

Different compression rates probably.

[Anach]

Thank you for posting this information!

 I am using it to organize my files (much like the poster above by creator) but one thing I noticed is that in some cases, the files output are larger than the files input.  For example, I had a folder of Fresh Prince cars at 38mb, the output file of the .dbc file created is 48mb.  Another folder I had was 284mb, the output file is 387mb.  Any idea why this may be happening?

Make sure you are using the experimental import as .dbc option, not the import from package option. It sounds like you are using import from package without compression.

I downloaded the Fancy Fashions set that is posted here and ran it through s3rc. I then converted to .package using Delphy's tool, and loaded the newly created .packages into s3pe as per the instructions in this thread.  That all went fine up until I got the "Manual Manage of XML Files Required Message."  However, instead of it detecting the 0x73E93EEB files, it's detecting a bunch of files marked 0x0166038C.  Currently, s3pe has those files marked for deletion.  Can someone please tell me if this is ok or normal?  I'm still pretty new to learning what all these different file types mean. 

EDIT:  I went ahead and let s3pe create the file.  It saved in .package format and I plopped it in my "Packages" folder.  The game loaded up just fine, but the items were not there.  I am stumped.  I am also probably missing something really basic that I'm going to slap myself for once someone points it out to me...

The 0x0166038C are _key files, not _XML files. They are simply manifest files which are merged when using import as .dbc. So it deletes all the excess _key files that it merges into one. So nothing to worry about. The only merging of _xml's you can get away with is the deleting of those 0x73e93eeb files. Any other _xml conflicts would normally mean a mod conflict. So sounds like you did everything ok

My thanks to you Anach.  My game had slowed to such a crawl playing was nearly impossible.  After following your instructions game performance has improved 90 percent.  The load time went from 10 to 15 minutes to 3 minutes, all of my CC is showing up and running fine and game play is on par with a fresh install without any CC at all.  To be honest, I wasn't expecting much from this experiment, only to be astounded at the difference it has made.  If you would be so good as to indulge my curiousity, I do have one question.  When merging files is there any reason one could not group them according to website or creator?  e.g., GOS.package or aikea_guinea.package.  It seems to me further precision might come in handy should one run into a problem after a merge.
             

No problem. I'm honestly surprised no one else suggested it sooner.

You can create whatever packages you want and name them anything you like. I like to keep sets together as well. I've been experimenting and changing my .package categories as I go. I would recommend everyone to test mods thoroughly in procmon by keeping them in a probation folder before merging them into their main .package files though.

[GlamGirl]

Quote from: GlamGirl on Today at 08:04:34
Thank you for posting this information!

I am using it to organize my files (much like the poster above by creator) but one thing I noticed is that in some cases, the files output are larger than the files input.  For example, I had a folder of Fresh Prince cars at 38mb, the output file of the .dbc file created is 48mb.  Another folder I had was 284mb, the output file is 387mb.  Any idea why this may be happening?

Make sure you are using the experimental import as .dbc option, not the import from package option. It sounds like you are using import from package without compression.

Thanks for your response.

I am using "Import ... (EXPERIMENTAL) as .dbc" - it seems to be doing it to some degree with each folder.  I do notice I have one folder where it did exactly the opposite.  It took 72mb and turned it into 18mb.  I haven't loaded my game up yet.  I actually had no problems with slowdown whatsoever (after using Process Monitor in the past to remove problem files) but was playing around with this more out of curiosity so either way it should be fine.   Probably the compression levels being adjusted as suggested.

[Inge]

Just to let everyone know the import as dbc is now in the latest official s3pe release, in the MTS forum as usual.

[coconnor]

I downloaded the Fancy Fashions set that is posted here and ran it through s3rc. I then converted to .package using Delphy's tool, and loaded the newly created .packages into s3pe as per the instructions in this thread.  That all went fine up until I got the "Manual Manage of XML Files Required Message."  However, instead of it detecting the 0x73E93EEB files, it's detecting a bunch of files marked 0x0166038C.  Currently, s3pe has those files marked for deletion.  Can someone please tell me if this is ok or normal?  I'm still pretty new to learning what all these different file types mean.  

EDIT:  I went ahead and let s3pe create the file.  It saved in .package format and I plopped it in my "Packages" folder.  The game loaded up just fine, but the items were not there.  I am stumped.  I am also probably missing something really basic that I'm going to slap myself for once someone points it out to me...

Did you follow Anach's directions in Step 6 of his tutorial?  (See below)

"6. After the import was complete, I deleted all the type "0x73E93EEB" _xml files with a instance of "0x0000000000000000" , as these manifest files don't seem to be required by the game, but will be a main source of conflicts (possible left overs from extracting .package from .sims3pack)"

I think that's important.  I did the Fancy Fashions set and it is showing up in game.  However, previously I did some others that are not showing up and I believe this is why.  I will have to go back into those packages and check.  But, try re-importing the package to s3pe to check.

[nilniliel]

Is there a way for a Mac user to do this? I have seen two others ask this question but no one has answered them. If anyone knows how to get an .exe file running on a Mac, that'd be terrific, but as of yet, downloading it via MTS doesn't allow for any other platform than Windows.

[jesuisdvd]

When i try to import the file from resource/import/ experimental, gives me a "JIT (just-in-time) error" (exception text "System.IO.InvalidDataException: Expected magic tag 'DBPF'.  Found "), and i cant load any package file, what can i do, i need to convert all the new "simstore files " who doesnt work, to work on my game....

[Baarogue]

When i try to import the file from resource/import/ experimental, gives me a "JIT (just-in-time) error" (exception text "System.IO.InvalidDataException: Expected magic tag 'DBPF'.  Found "), and i cant load any package file, what can i do, i need to convert all the new "simstore files " who doesnt work, to work on my game....

Already answered in this thread.

[Anach]

Is there a way for a Mac user to do this? I have seen two others ask this question but no one has answered them. If anyone knows how to get an .exe file running on a Mac, that'd be terrific, but as of yet, downloading it via MTS doesn't allow for any other platform than Windows.

I don't think anyone here can turn your oversized PDA into a proper computer. Other than that, the reason there is no answer, is simply because there is no answer.

When i try to import the file from resource/import/ experimental, gives me a "JIT (just-in-time) error" (exception text "System.IO.InvalidDataException: Expected magic tag 'DBPF'.  Found "), and i cant load any package file, what can i do, i need to convert all the new "simstore files " who doesnt work, to work on my game....

Read back a few posts.

[morriganrant]

Is there a way for a Mac user to do this? I have seen two others ask this question but no one has answered them. If anyone knows how to get an .exe file running on a Mac, that'd be terrific, but as of yet, downloading it via MTS doesn't allow for any other platform than Windows.

The only way to get windows stuff running on a Mac OS is to do a boot camp arrangement and to have two OSs.