SimSheet (a sim-tracking program) [Now with Aspiration and LTW tracking!]

[Inge]

Inge - could he just automagically replace all newlines with \n right before writing the data to file?

And what about embedded commas?  Some users may be in the habit of using commas in housenames, sim descriptions etc (I noticed the sim description wasn't being put out and asked Peter to add it)

http://www.wheresjames.com/rulib/class_c_csv_file.html

[rufio]

Inge - so just replace all commas, \s and newlines with \, \\ and \n.

I'll send you a link to SimSheet-debug.  Download, run, and screencap the latest debug message, please.

[Inge]

I am not writing, designing or testing your application.  This is an exercise in API - in managing an interface.

Also try http://www.codeproject.com/KB/database/CsvReader.aspx

[rufio]

I thought we were talking about having SimPE output things?  Anyway, it was just a suggestion Pescado made, and I was only thinking out how it might work.  We've agreed that something has to change in the output.  What would you prefer to do?  I've already been over what would be easiest for me, but I'm sure I can work around whatever you decide to do, without needing to learn some whole new API.

[J. M. Pescado]

Also, I think I've probably gotten spoiled with Qt's I/O classes, but I would really prefer not to have to go back to using C I/O.  I've forgotten a lot of those functions and never really learned printf/scanf anyway.

Pfah! C I/O offers tighter control and is superior to wussy lesser I/Os! The fact that you speak of going BACK to it proves its superiority to lesser, wussier I/Os! Return to the truth path or burn forever in the fires of HELL and DAMNATION! REPENT, SINNER!

[Inge]

Rufio, we continue to negotiate SimPE output.  My comment was in response to you giving me a link to download your program.

[rufio]

Oh, sorry, that should have been directed a coral.   

JMP - Yes, I know, in your days they had to do it all with null-terminated arrays and structs and functions with irritating numbers of underscores in their names and they liked it.  Unfortunately I like my new toys too much to repent.  I will be sure to stay off of your lawn, though.

[Inge]

So did you have a look at http://www.codeproject.com/KB/database/CsvReader.aspx ?   I feel I would like you to at least have a try at reading full CSV before we give up.

[tunaisafish]

Code:

string line = in.readLine ();
if (line[line.size () - 1] == '\\' && line[line.size () - 2] != '\\') line += "\r\n" + in.readLine ();

tunafish - That's why it checks for \ as the last character and \ as the second-to-last.  Naturally, any \ in the data itself would be escaped with another \.

Yes.  That's where your code fails. You backtrack -2 to test '!='.  So as -2 held a '\' too, you'd need an 'else if' to go back to -3... etc.
I was just trying to show you how escaping characters will add more complexities.
It's actually tricky to do properly without a backtracking regex engine.
So while you invent a new standard to fix a bug, you'll have all the bugs that come with reading this new standard too.

CSV has some flavours sure, but the libraries I've seen allow you to choose the ruleset.
That's why I suggested 'text/csv' and the link to the rfc

Any standard is good, CSV seems the logical choice here, with XML coming a distant second.
CSV will also cope with the non-ascii charsets.

[Inge]

Pete says xml would be create too huge a file for this purpose.

[rufio]

I see what you mean, tunafish.

Inge, that looks nice, but I don't use .NET.  (I learned C++ in linux using vi and the command line, and now I'm just using gvim for Windows.)  Do I need .NET to make this work?

[Inge]

Probably for that one, but Peter found it by a simple Google search on "csv C++".  Maybe you could try that and find a library function more suited to your development environment?

[rufio]

Er, well, my "development environment" is basically a text editor, so ideally it would just be a header file with a class definition in it.  I guess I'll make an account on that site and see what's in their zip.

ETA:  Apparently by "C++" they mean C#.  Or maybe it's all the same thing when you use .NET?  Who knows.

Like I said above, though, I'm pretty sure I can figure out how to read enquoted fields containing newlines if that's how we decide to do it.

[Inge]

I found a situation even Excel can't cope with.  On one of the Gavigan family - the father I think - there is a linefeed without a CR.  Excel splits the record on it, even though it was enclosed in quotes    That's an excel bug.  A quote is a quote is a quote and if it's not closed the stuff is NOT THE END OF RECORD, INNIT?

[coralleane]

I've downloaded the debug version and when I click on import Sims, it brings this up:

"C:\Documents and Settings\(username)\My Documents\EA Games\SimPE-test\SimPE-test\SimPE.exe" -rufio -hood N001 -out C:\Documents and Settings\(username)\My Documents\EA Games\SimSheet-debug

The paths are both correct.  Once I click on OK, everything proceeds exactly as it did on the previous attempt; SimPE runs through its bit, then I get exactly the same messages as before.

[Inge]

I will soon be putting up another QA of SimPE on the ambertation site.  This is advance warning, because it has the proper CSV output for SimSheet.

Now, Rufio has not yet updated SimSheet to accept this format, and we don't know for sure she will do so.

So, SimSheet users must KEEP A SIMSHEET-COMPATIBLE VERSION OF SIMPE ON YOUR SYSTEM if downloading the new QA.

[rufio]

Coral - which messages are those again?  I thought you were having trouble getting SimPE to use the command line arguments.

Inge, I will need to see a sample output so I know which fields you have enquoted before I can update and test.

[Inge]

Yes, I assume you have access to the QA?

Did you make any progress with finding out how to handle standard CSV?  Obviously there is still an option to revert to or add the output you preferred, but I am keen to make a distinction between "can't be parsed in my version of C++" and "can be done but I need time to learn how"

[coralleane]

All I'm doing is going in, setting up the settings, then going into Manage Sim Lists.  When I click on import it does the first bit correctly, then once SimPE seems to have done its bit, instead of saying it's generated a list of however many Sims, it instead gives me the "-rufio -out {outpath} {hood hood} {group group}" dialog.  Only option is "OK".  Clicking that then generates the "Cannot find output file.  Check that you are using the AL version of SimPE" error message.  Using the debug version, once I tell it to import it brings up a dialog that says: "C:\Documents and Settings\(username)\My Documents\EA Games\SimPE-test\SimPE-test\SimPE.exe" -rufio -hood N001 -out C:\Documents and Settings\(username)\My Documents\EA Games\SimSheet-debug, which is the path of SimPE and SimSheet, then runs through the SimPE step before giving me the two error messages again.  The list of exported Sims is not generated in the SimSheet folder, the TS2 folder, or anywhere else I can find on the computer. 

[rufio]

Inge, I just have the version you sent me a link to in email.  Don't I have to sign up for a list somewhere to get access to the QA?

RE: CSV - it doesn't have to do with my "version" of C++ so much as that all of the CSV-parsing stuff that I can find has IDE crap in it that I don't know what to do with because I don't know how it's supposed to work.  What I can do, as I said before, is to parse it out manually, which may not work perfectly in obscure instances of people putting weird shit in their sims' names, but I think I should be able to deal with newlines.  Alternatively, we can leave it exactly the same but kill the newlines.

Do you recognize Coral's error message?  It's not from SimSheet, at any rate.  Is there anything wrong with the command she posted?

[Inge]

Do you recognize Coral's error message?  It's not from SimSheet, at any rate.  Is there anything wrong with the command she posted?

It seems strange that SimPE is putting out a message saying ""Cannot find output file.  Check that you are using the AL version of SimPE""

a) Why would SimPE even be looking for the output file?  It would be complaining instead that it could not create it.
b) Surely SimPE would know which version it was?

[Inge]

Coralleane, make a shortcut to Simpe, and then edit the shortcut properties so that after the string already in Target, you add just -rufio

Run it.  The output folder should be in your My Documents\EA Games\The Sims 2 folder

If it got there ok, then edit the shortcut once again to add -out <pathtotestfoldersomewhere>

See if that works too.

If you like you can add -hood N00x (where x is a number) to both those tests to limit the output to one hood.

[coralleane]

Firstly, the target precisely as formatted in the shortcut properties, just in case I've made an error somewhere like not using " when I should or vice versa:

"C:\Documents and Settings\Coral\My Documents\EA Games\SimPE-test\SimPE-test\SimPe.exe" -rufio -out C:\Documents and Settings\Coral\My Documents\EA Games\SimSheet-debug -hood N001

Your first suggestion, Inge, with it outputting into the Sims 2 folder, worked fine.  Once I added the specific path and narrowed down the neighbourhood, I got the first message again (the -rufio -out {outpath} etc... one), but not the one about it being an incorrect version of SimPE.  As well as the version above, I tried using:

"C:\Documents and Settings\Coral\My Documents\EA Games\SimPE-test\SimPE-test\SimPe.exe" -rufio -out C:\Documents and Settings\Coral\My Documents\EA Games\SimSheet

"C:\Documents and Settings\Coral\My Documents\EA Games\SimPE-test\SimPE-test\SimPe.exe" -rufio -out C:\Documents and Settings\Coral\My Documents\EA Games\SimSheet\ -hood N001

"C:\Documents and Settings\Coral\My Documents\EA Games\SimPE-test\SimPE-test\SimPe.exe" -rufio -out C:\SimSheet -hood N001

..and various other similar combinations. 

I manually moved the list generated following Inge's first suggestion, by cutting and pasting it into the SimSheet folder.  Once I'd done that, I tried to create a list by telling it to use the previously generated data, and it picked up the list alright in the SimSheet folder.  So from what I can tell, the problem is that for some reason, SimPE is not creating the output into the SimSheet folder and is giving the first dialog instead.  SimSheet then presumably looks for the exported sims list, can't find it, and gives out the message about not having a current version of SimPE. 

[Inge]

If you have a path with spaces - like "My Documents" then the whole path needs to be in quotes.  Just put them round the path one quote before C: and its partner after SimSheet-debug

[coralleane]

Okay, running it through with the quotemarks now and so far it seems to be working.  Possibly this is the problem in the program itself as well, the existence of the spaces in the output path?  I have to head for work now but I'll try moving SimSheet to a space-less path either when I get back or tomorrow if the solution hasn't already been found.  Using the debug version of SimSheet, when I told it to import, it brought up:

"C:\Documents and Settings\Coral\My Documents\EA Games\SimPE-test\SimPE-test\SimPE.exe" -rufio -hood N001 -out C:\Documents and Settings\Coral\My Documents\EA Games\SimSheet-debug

...with no quote marks.  So is it just a case of forcing SimSheet to include quotes around the output path?

If you have a path with spaces - like "My Documents" then the whole path needs to be in quotes.  Just put them round the path one quote before C: and its partner after SimSheet-debug