Crash on resized lots at 7PM -- any ideas?

[ingeli]

Yesterday I was working on my castle courtyard (3x3 lot, set as a park, placed in the middle with no road access) and tried to place a pedestroan portal 1 level up, on the top of the stairs. Not good. Made the game crash every time I tried to save. Moved portal to ground level, saved fine. 

[Inge]

So shrinking lots so that the walls are on the boundary is not safe to do?

This is a red herring.  All my lots with walls along their boundaries are working fine.

And I think it is safer than feared from the point of view of neighbourhood corruption.  As soon as the game detects the data is attempting to write to space it's not meant to, it crashes with an access violation.  If it was allowing corrupt data to overwrite other parts of the file, you'd be more likely to get your errors or crashes when you loaded another lot, not while you were updating or saving the faulty one.  So I think there are inbuilt defences.

[dizzy]

This is a red herring.

So, you can explain the crash?

[Inge]

No, but unless you think elimination processes are meaningless we have eliminated walls on the edge of the lot as being the culprit.  If it takes walls on the edge of the lot AND some Other Factor in order to crash, then it's the Other Factor we need to winkle out, which will leave walls on the edge of the lot innocent and cleared for use under most circumstances.

[dizzy]

* dizzy thinks the "Other Factor" is the Real Red Herring (tm).

[Inge]

Well it depends on your main reason for wanting lot shrinking.  If you dearly wanted to do the Other Factor on your lots, then you'd need to avoid the walls on the boundary (IF and only IF it turned out to be implicated).  However for many of us, the idea of proper joined up row houses was the main reason we ever asked for shrinking in the first place.  We are obviously therefore very interested in tracking down the Other Factor.   What we do know without any doubt at all is that walls and/or roofs and/or floors on boundaries are not a no-go by themselves.

[Doc Doofus]

I seem to have a reliable way of avoiding the crash on my lots. 

1. Plop down the lot.
2.  Go into build mode and modify it.  (Repainting it and changing roof color is a good idea if you're setting up a little colony of row houses).
3. Move in your sim whether  CAS or other.

I tried to repackage one of the crash-y lots (Backdoor 42) immediately after step (2).  I plopped the packaged copy of the fixed lot down and moved in a CAS sim.  It crashed.  Apparently the packaging process undoes whatever tidying-up and housecleaning that first build-mode edit does.

Whatever.  The lots looks really cool and the crashes haven't borked my neighborhood yet, so I'm happy.

You know, I've been playing this game for years now, since the first release, and I've never messed up a neighborhood so badly that I had to stop playing it or couldn't use it.  EA really improved on the lousy QA they had with Sims 1, which was not robust at all.

[Simsample]

in order to disprove the theory that previously played, "pre-crashed" lots are safe, someone could play those and see if they can get them to crash. I have attached all of my test lots to my post in the R+D thread at mts2 (link see above) -- could somebody download "lot03+04+05+05_post-play.zip" and test lot 3, 4 and 6 to see if you can get them to crash?

Plasticbox, I'm getting inconsistent results with these lots. I could make none of them crash at 7PM or by day/ night toggle, but I did have lots 3, 4 and 5 (but not 6) crash upon loading.

I placed each lot several times and in several situations in the hood, and they didn't crash all of the time- so these results aren't really conclusive. One of the placements of lot 3, I tried 4 different CAS sims to move into it- but each time it crashed. Then I moved the Ramaswamis in with no trouble. However, placing a fresh instance of that lot elsewhere in the hood allowed me to move in and play a CAS sim with no troubles. The only link I made was that the times I had a crash, the lots were placed adjoining another lot which had wall on the boundary- but just placing the lot next to a boundary wall didn't guarantee a crash. Confusing!
I've attached to logs for different lots that crashed upon loading (lots 5 and 3), and here is an image of the graphical glitches I have with the boundary walls:

That's lot 6, which I couldn't make crash!

[pbox]

Dizzy:

Can you see any significant difference in the error logs depending on what lot the error occurred on? Or can you point me at where to look in the logs, so that I can see that for myself? (it's probably not very interesting reading, going through 25 logs all over again) This could be interesting in order to see whether there's any difference in what is *on* the lots (walls, roofs, foundations) .. you said

I'm saying it's probably unsafe. If the index (-192 in my case) is coming from the lot data itself, you may be able to do something about it, but this value changes in different instances of the crashing. In some cases, it's the multiplier (81 in this instance) overflowing (the value 19496 in one of pbox's logs). In either case, it's the same EIP, so this suggests to me that the errant value is being calculated. In other words, the bad offset is probably not coming straight out of the lot file, but rather as a result of a miscalculation because of the walls and their situation.

I can see where it say's "EIP" in the logs but I have no idea where/how to look for an index or overflowing multiplyers, if you'll forgive my ignorance =).


Also:

As soon as the game detects the data is attempting to write to space it's not meant to, it crashes with an access violation.  If it was allowing corrupt data to overwrite other parts of the file, you'd be more likely to get your errors or crashes when you loaded another lot, not while you were updating or saving the faulty one.  So I think there are inbuilt defences.

I think it's not impossible that this is *exactly* what happened during my test run yesterday. I have never seen so many crashes on load/save before.

The "inbuilt defences" might vary with the game version you're running (and possibly other parameters; RAM?) .. which might explain how some users *see* less or no errors, even though they would still *exist* just the same.


(Also, could someone explain "red herring"? Wikipedia says "In literature, a red herring is a plot device intended to distract the reader from a more important event in the plot, usually a twist ending" .. this what you mean? (Non-english person here))

1. Plop down the lot.
2.  Go into build mode and modify it.  (Repainting it and changing roof color is a good idea if you're setting up a little colony of row houses).
3. Move in your sim whether  CAS or other.

I tried to repackage one of the crash-y lots (Backdoor 42) immediately after step (2).  I plopped the packaged copy of the fixed lot down and moved in a CAS sim.  It crashed.  Apparently the packaging process undoes whatever tidying-up and housecleaning that first build-mode edit does.

Did you try to see what happens when you modify the freshly re-plopped-down lot *again*? Does it work then?

In any event, I'm afraid that no crashes don't prove anything. I don't mean to scare you, but only because it doesn't crash it doesn't mean there's no data corruption going on in your hood. I'm going to take down my uploaded lots from mts2 as soon as it's back up, so if anyone still wants them you'll have to be quick =). Please do not redistribute any of my shrunk lots without a dire warning that using them may corrupt neighbourhood data. I don't want people to fuck up their games just because they think the house looks cool.


Simsample (and Zazazu and baratron), thanks for your testing. There's no need in trying to make *all* of the lots crash -- one single crash is enough to prove that we can't assume a lot is "safe" just because it did or did not crash during building/playtesting: There is no correlation. This is what I wanted to know. Thank you.

(Simsample, your screenshot doesn't show up for me, but I think I know what you mean -- vertical lines on the border walls?)


Also:

No, but unless you think elimination processes are meaningless we have eliminated walls on the edge of the lot as being the culprit.

How so?

In case you're referring to my test run: only because the wall lots didn't crash for me 3 times in a row, it does not mean there's nothing wrong with them. Again, no crash proves nothing.

The only thing this test run really showed is that there's definitely something wrong with the roofs -- but right now, we have no way of knowing whether the crash on the wall lot (lot 6, phase 5) occurred because of general data corruption of the lot/sim/neighbourhood due to the borken roof lots, because of the walls, because of the moon phase .. all I can see is that it did crash. Simsample crashed with lot 5 (walls), Zazazu with lot 3 (foundations) .. they didn't even download a roof lot; how are the walls "not the culprit" then?

[pbox]

Inge,

I'm not up to date with the portal stuff -- this:

tried to place a pedestroan portal 1 level up, on the top of the stairs. Not good. Made the game crash every time I tried to save. Moved portal to ground level, saved fine. 

is something that's already known? I believe there was some talk about z levels of portals on R+D .. not sure though. Is this a new thing that Mootilda should know about, or is it old hat?

[ladykat]

I tried pbox's BackdoorLane lot - it crashed first time but not the next 3 times.

What exactly do you mean with "3 times": 3 different inhabitants, or 3 different attempts to load/toggle nighttime/save?


I placed the lot from the bin 4 times in toal, in 2 different neighbourhoods I am using for testing,  I made 4 CAS sims, put one in each house.  Each time I got the sim to move a bit, saved, bought a chair, then toggled the day/night view.  The first house crashed, the other  3 didn't.



kat

[Inge]

The "inbuilt defences" might vary with the game version you're running (and possibly other parameters; RAM?) .. which might explain how some users *see* less or no errors, even though they would still *exist* just the same.

Game version, yes definitely that's a high possibility - as they (hopefully) improve on their code as time goes on.

(Also, could someone explain "red herring"? Wikipedia says "In literature, a red herring is a plot device intended to distract the reader from a more important event in the plot, usually a twist ending" .. this what you mean? (Non-english person here))

Pretty much, but of course in this case I am not suggesting a person is trying a distraction technique, simply that we may be distracted by the fact walls on the edge of the lot are a very obvious difference from lots before the lot expander so we keep thinking of them.

[Mootilda]

My primary concern since the beginning has been that the lot package is actually corrupted (based on the game code) and that this corruption may spread to the rest of the neighborhood.  I'm very sorry if I was unable to explain this adequately.  Sounds like dizzy has managed to explain it so that some people now understand the problem.  Thanks, dizzy!

However, people really want this feature, so I am willing to put some time into getting it to work.  The current LotExpander may be creating a lot package which the game is unable to handle, but that doesn't mean that we've run out of ideas.

My initial attempt at this feature was to shrink the various arrays without attempting to delete "off the lot" objects, or to keep objects on the shrunken lot.  When pbox put out her test lot, I was hoping that we might be able to narrow down the possible culprits (roofs, portals, foundations, ...).  Part of my problem is that I am completely new to modding and am unsure of what's possible.

Here are some of the things that I can try:
- Delete any roofs which go past the lot edge.
- Move any objects which are outside of the new shrunken lot, so that they are back on the lot.

If anyone else has any ideas, please let me know.

And I think it is safer than feared from the point of view of neighbourhood corruption.  As soon as the game detects the data is attempting to write to space it's not meant to, it crashes with an access violation.  If it was allowing corrupt data to overwrite other parts of the file, you'd be more likely to get your errors or crashes when you loaded another lot, not while you were updating or saving the faulty one.  So I think there are inbuilt defences.

I don't believe that you understand.  An access violation occurs because the game became so confused that the corruption was completely out of control.  An access violation is the operating system's way of stopping a program which has become "insane" and is trying to change memory which doesn't even belong to the program.  IE, the Sims 2 is trying to write into your open Word document and Windows won't let it.

An access violation is absolute proof that the game has no inbuilt defences.

The reason that these lots do not become more and more corrupted is that the game never gets a chance to write the file after the corruption is completely out of control.

[Inge]

Here are some of the things that I can try:
- Delete any roofs which go past the lot edge.
- Move any objects which are outside of the new shrunken lot, so that they are back on the lot.

1) would be devastating unless it was optional.  I make gable-end roofs for terraced houses (that's the normal roof type) and they always hang over the end a little.  If you got rid of those there would be holes in my roofs.

2) yes I have a hunch this would be very worth trying.  Some of those deleted objects may be holding sound and graphic information.  Missing sound and graphic information is usually what causes CTD in this game, in my personal experience.

[Mootilda]

1) would be devastating unless it was optional.  I make gable-end roofs for terraced houses (that's the normal roof type) and they always hang over the end a little.  If you got rid of those there would be holes in my roofs.

Again, I'm a newbie to modding, but are you sure that this would create holes?  My plan was to try to cut the roofs at the very edges of the lot, so that there would be no overhang, but the roofs would line up with the walls.

[Mootilda]

...and if so, what about one space from? The difference between having houses 4 spaces apart and having houses 2 spaces apart is considerable.

What about if you are shrinking pre-existing lots,and never moving them or putting them in the lot bin-does that have the same potentially damaging effect?

For both of these, the answer is the same:  Until we know what's causing the problem, we just don't know what's safe and what isn't.

[pbox]

1) would be devastating unless it was optional.  I make gable-end roofs for terraced houses (that's the normal roof type) and they always hang over the end a little.  If you got rid of those there would be holes in my roofs.

Again, I'm a newbie to modding, but are you sure that this would create holes?  My plan was to try to cut the roofs at the very edges of the lot, so that there would be no overhang, but the roofs would line up with the walls.

I don't think there would be holes -- I guess if Mootilda does that, the edges would look like they currently do on the "high side" of half shed roofs: roof stopping short at the wall edge.

This may even be a good thing for gameplay, because it would increase visibility in narrow lots that are between other lots with this type of gable roof. Right now, the neighbouring roofs visually poke a little bit into the current lot where they overhang.


However, I'm not sure if "simply" deleting the overhang would solve anything: from my experience e.g. with snow in Seasons, the game thinks of a tile under the overhang as "not under a roof" (snow will pile up there, regardless of the overhang), so I would think the overhang is purely visual and has nothing to do with the "footprint" of the roof. Other purely visual things can poke into neighbouring lots without any problem -- think of trees for example: if you put the large willow tree right next to the edge of a regular lot, it pokes into the neighbouring lot by a considerable amount. No problem at all (except visually, if the tree branches end up in the neighbour's bedroom).

Still, it may be worth a try to change the roof code, do another test run with minimal lots, and see what happens. Something about the roofs is definitely broken right now, after all.

Is there any difference between "roof code" and "roof wall code", by the way?

[Inge]

Again, I'm a newbie to modding, but are you sure that this would create holes?  My plan was to try to cut the roofs at the very edges of the lot, so that there would be no overhang, but the roofs would line up with the walls.

This type of thing is not my main area of expertise, but I would have thought the overhang was built into the mesh.  The modularity seems to be a panel of roof has a floor/ground grid intersection at its centre, so if you remove the panel that is overhanging the wall, the next panel would not begin until half a grid into the lot.   Now it is possible that you can find a panel fraction to substitute, because after all, when you put a dormer on it does cut the roof at a gridline, though it could be using masking, but this is not something I know much about.  Maybe Niol can help here.  But the overhanging roof in any case belongs to the same position as the wall it is sitting on, rather than a position off the lot, so in theory it is as much part of the shrunk lot remainder as the border wall is.

[Inge]

I don't think there would be holes -- I guess if Mootilda does that, the edges would look like they currently do on the "high side" of half shed roofs: roof stopping short at the wall edge.

I think those roofs are only like that because a roof panel/module mesh is positioned so as to line up with the grid at its top edge.  Otherwise it would intersect the panel the other side of the ridge when you are making a ridged roof making a strange teepee effect.

[pbox]

I placed the lot from the bin 4 times in toal, in 2 different neighbourhoods I am using for testing,  I made 4 CAS sims, put one in each house.  Each time I got the sim to move a bit, saved, bought a chair, then toggled the day/night view.  The first house crashed, the other  3 didn't.

Got it, thanks =).

[dizzy]

Dizzy:

Can you see any significant difference in the error logs depending on what lot the error occurred on? Or can you point me at where to look in the logs, so that I can see that for myself? (it's probably not very interesting reading, going through 25 logs all over again) This could be interesting in order to see whether there's any difference in what is *on* the lots (walls, roofs, foundations) .. you said

...

I can see where it say's "EIP" in the logs but I have no idea where/how to look for an index or overflowing multiplyers, if you'll forgive my ignorance =).

No, the logs tell you very little. In fact, I wish they had outputs on the FP registers as well as some other stuff, but you take what you can get. I did a very preliminary analysis using a debugger, and that led me to the conclusion of bad pointers due to screwy offsets. Basically, they are trying to load a value from [EAX+ECX] after doing a multiply using EDX. I assume there is a boundary error considering you are getting an Access Error from the CPU/MMU, however the code itself clearly does no bounds checking of its own (due to some compilation optimizations, no doubt).

[Mootilda]

the code itself clearly does no bounds checking of its own (due to some compilation optimizations, no doubt).

Interesting hypothesis (that the compiler optimization is removing the bounds checking).  What do you base that on?  I've been assuming that the developers are using a language without bounds checking (nobody really writes software with C#, do they?) and that they are too rushed in development to add bounds checking on their own.

[Update:]

Did a quick check with friends in compiler development and game development and they agree.  Games are likely written in C / C++ and have no bounds checking unless the developers add it themselves.  Compiler optiimizations would never remove bounds checking unless they were fundamentally broken.  Let's not blame the compilers for the fact that the game developers are working too hard fast to do proper bounds checking.

(Sorry about the rant, but I work on program development tools for a living.)

[Inge]

I suppose they never reckoned with us writing these tools.   So if it's not the game protecting its own data, I am pleased with the job my operating system is doing!  I have not yet had a totally borked hood and you should see some of the things I get up to!

[dizzy]

Let's not change the subject, Mootilda. I don't think anyone else cares about the details of how compilers work. My point is, whether you believe compilers are involved, the program is severely lacking in what I would consider common robustness that is taught even at the college level.

[Mootilda]

I suppose they never reckoned with us writing these tools.   So if it's not the game protecting its own data, I am pleased with the job my operating system is doing!  I have not yet had a totally borked hood and you should see some of the things I get up to!

Absolutely.  What we've doing with shrinking of lots to allow walls at the edges is well ouside of normal game play.

the program is severely lacking in what I would consider common robustness that is taught even at the college level.

Agreed.  However, that doesn't stop us from modding.  We just need to be careful to avoid doing things that the game can't handle.