Are stub character files really harmful to your game? If so, how?

[April Black]

Hi guys,

I am investigating the subject of BFBVFS/neighborhood corruption just out of sheer curiosity. The main thing that I am trying to determine is how exactly is deleting Sims from the bin (and thus deleting gravestones in vanilla game) is supposed to be harmful to your neighborhood, or if it's even harmful at all.

And yes, I am aware of the "official" explanation (i.e. "dangling" references pointing to non-existent data), but as I am a software dev in real life, sadly this does not satisfy my curiosity

I've conducted some experiments in my own game. What I've found is that The Sims 2 went down the "soft delete" route, and reasonably so in my opinion. If I was a Sims 2 dev, I would not want to cascade delete all references along with the Sim (performance! also what if the game crashes mid-deletion?), and leaving behind a stub file with only necessary information seems like a logical thing to do.

Personally, I haven't found any evidence suggesting that deleting Sims from the bin could cause corruption. The references are not pointing to non-existent data, as Sim Description entry still exists and character file is still there, although notably smaller (since it doesn't have 3D data, clothing info and all that other crap). NID is still considered as taken, therefore new Sims will never generate with it, even if SCID is reset. The information needed to correctly display references such as family ties & memories is still there (e.g. name, thumbnail). So, the way I see it, everything is fine, no corruption. I can see why straight up deleting the character file would be a problem (null references are never good news), but not the Sim bin route.

I understrand that there is consensus on this forum that the stub files themselves are the problem. As in, the memories pointing to stub files are corrupt, even if thumbnail & name is in place, because they cause the game to access some unspecified non-existent data. This is also why moving Sims between the neighborhoods would be bad, since they generate the same stub files (for moved Sims in the source hood and for all relatives in the target hood).

Case in point - this post:
http://www.moreawesomethanyou.com/smf/index.php/topic,19966.msg566851.html#msg566851

This is just an example of course, I've seen this explanation appear here and there on MATY. My question is: do we have any evidence for that? How can I observe this in game or in SimPe? I personally don't see why the game would try to access the data of a "shredded" Sim. Unlinked Sims are not allowed to spawn on community lots, they cannot be called, don't appear in a relationship panel and are as good as inaccessible in normal gameplay. I have conducted multiple in-game experiments and the game seems to have some safety mechanism regarding bringing up references pointing to unlinked Sims. Not only memories and family trees display correctly, but also Sims don't roll wants about missing Sims. They also don't seem to be able to gossip about them. If the deleted Sim was the only person that the remaining Sim knew, daydreaming/online chat thought bubbles just have random object placeholders, not the dreaded squiggly lines. I have also contacted LazyDuchess, who told me that he created a mod that purposefully tried to access the data of unlinked Sims, and he reported no issues. He basically agrees with me that the Sim bin route is safe.

Am I missing something? Or maybe deleting Sims from the bin does not cause corruption, and this was just an assumption based on the knowledge we had back then?

I have already asked this on MTS, but I've been only told that some modders agree with me (i.e. Lazy Duchess, Chris Hatch) and I didn't find out what caused people to believe that it's harmful, which is the main thing I'm interested in. So I came here, because I believe that this discovery was first mentioned on this forum. I would appreciate your insight - as I said, I'm a dev, so the more technical you go, the better! Also I am hoping that Pescado chimes in, since he has been here from the very beginning.

[Milhouse Trixibelle Saltfucker III]

I know the correct answer to your question, but what I really want to say is

Duchess

he

lol

[J. M. Pescado]

If you mean "will doing this in very isolated incidents explode your game instantly", then no. However, once you fall into a pattern of it, it will cause unbounded growth of these stub files that are largely invisible to the player. As the game has no intrinsic garbage collection routines, this means it never, ever, gets cleaned up.

This can produce several kinds of failure and problems:

1. You can overflow the file limits of the OS. Each of these dud entries still counts against you here. This will cause total failure of the game to load or save correctly.

2. Anything which actually does have to iterate through all these stubs in script code can overflow the game's max instruction count ("too many iterations"), and thus fail.

3. If none of these conditions are reached, the game ultimately has a hard limit on the number of sim descriptors present. While this is a number that's probably too high to plausibly reach in normal play, casually spamming ghost entries into your neighborhood save can easily hit this limit, which is why transplanting households across neighborhood saves is extra bad as this explosively grows the numbers of stubs.

4. The problem can be compounded if unsafe mods attempt to interact with these stubbed entries in incorrect ways.

[April Black]

Thank you, that is very helpful and it actually confirms the theory that I've had, that the character file count is the root of all evil

I actually overpopulated one of my neighborhoods on purpose (for science, of course) and I have noticed major slow down of the save process around the 1300s territory. I have a brand new gaming PC, normally saving takes a few seconds, but in the overpopulated hood it could last for a good few minutes. It actually crashed on me once, which caused a newly created Sim to not be commited. I can definitely see how having too many character files is bad. I also get how moving the Sims between the neighborhoods is a fast track way to overpopulation because of all of these stubs. This is actually how I lost my hood in 2005 (before Nightlife; I had a brilliant idea to merge Pleasantview, Strangetown and Veronaville into one using the lot bin method )

I think you are right that it's important to educate people that deleting Sims in a bin is actually pointless, as all it really does is hide that Sim from the game and saves a miniscule amount of hard drive space, but that Sim still takes up resources. Thanks so much again!

[Milhouse Trixibelle Saltfucker III]

That is the answer that I expected, for the record. I was planning to say more, not just totally leave you hanging with my useless response like that... but then I lost power for days.

Pescado used to call the process mentioned in item 3, of counting up to overflowing the NID slot, "certain 16-bit doom". I liked that phrase so I wanted to make sure it was mentioned.