How do you guys deal with csv values?

[jp1]

When outputting ZP info into a csv file, I use "," to isolate each value because , without the apostrophes will break every comma inside the value. Like an article with commas, will be broken at every single turn.

But when there are apostrophes inside the value, like this:

"{-Variable.producturl-}","{-Variable.movietitle-}","{-Variable.reviewtitle-}","{-Variable.review-}","http://{-Variable.tumblrpic-}","{-Variable.pinterestpic-}","{-Variable.youtubevid-}"

"Wer nur ein Menschenleben rettet, rettet die ganze Welt!",

it sees this:

"","

it seems to interpret it as a line break and it jumps onto the next row. Plus, once it's on the next row, the commas inside the value break the values, so it's very messy.
I don't understand why this happens because I thought I was successfully isolating the values with commas, and the apostrophes were protecting the value breaks from the commas inside the values. But no.

 

[hotohori]

Hi,

If your data contains comma, you should use other separators like semicolon or tab.

However, the whole line of data will fall into 1 column if you open the file in excel.
In this case, select your column, go to the Data tab, click on the Text to Columns button and follow prompt from there.

Hope this helps.

 

[jp1]

I'm not on my main pc now to test, but even if that would work, when it comes to articles there's bound to be another conflict, another ; sitting there somewhere waiting to screw me over. For articles and such, and long values like that, do you just give up on csv and use sql which works better?

 

[jp1]

thanks hotohori, I'm starting to understand more about this, but I realise now that if I export data from zennoposter to a database, I'm in trouble. A standard csv won't work, and I was relying on standard csv to import into sql to actually create the database. Even if I separate the file with values other than the comma, the sql won't understand this correctly, unless sql also has separation options I don't know?

 

[hotohori]

You're welcome

Sorry, I have no experience in sql.
If I'm not wrong, C# code should be able to write data directly to sql database.

 

[hotohori]

This may help you.

http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/

Using .sql script to initiate the import. You can determine the separator with "FIELDTERMINATOR"

 

[jp1]

Thanks, but right now I've zoned the problem down to what must be an issue with TSV or CSV not being capable of holding more than 255 characters, or large articles in any one specific value. I'm doing more split tests to try to figure this out and it's really frustrating, but people must be importing and exporting large articles from and to databases through csv somehow so it's gotta be possible.

Something in my large articles is making the row break, ruining my ability to scrape data with zp into a tsv/csv and then onto a sql that'll show on my wordpress site... I'll update this thread with any news and listen to any suggestions, thanks

 

[jp1]

Ok, the issue has to do with line breaks in regex, most probably. One value, the value with the article, is full of line breaks and it's breaking my csv and tsv files in pieces.

I haven't learn how to do line breaks in regex, so if anyone can help me isolate the article below, the line that starts with 'Schindler's list', I'd be very thankful. I only know how to isolate the file with the three first lines in it, two empty and one with the div mention, and that breaks my file. At the end of the article too, it breaks. So basically I need to know how to integrate line breaks in expressions in .net regex.

        <b><span class="h3color tiny">This review is from: </span><a href="http://www.amazon.com/Schindlers-List-VHS-Liam-Neeson/dp/6303168507/ref=cm_cr_pr_orig_subj/183-1056400-8295663">Schindler's List [VHS] (VHS Tape)</a></b>
      </div>

Schindler's List is my favorite historical drama of all-time for a number of reasons. Not only is it a masterpiece from a cinematic point of view, but it is priceless for the story it tells to the world. <p>First of all,  the acting is superb. Liam Neeson does well as Oskar Schindler, but in  particular I liked Ben Kingsley (as Istak Stern, Schindler's accountant)  and Ralph Fiennes (as Amon Goeth, the camp commandant). All of the  performances were very convincing and reflect the good  casting.</p><p></p><p>Another great feature of this film is the soundtrack. Slow,  soaring music tells of the painful circumstances of the Jews and of their  conflict with the Nazi regime. Mixed in with the instrumental pieces are  Jewish melodies which also gave me a sense of the cultural traditions of  the Jewish people. </p><p>From a technical point of view, the decision by  director Spielberg to shoot the movie is black-and-white was a good one. In  fact, I think it makes the movie better than it would have been in color.  The few color segments throughout the movie are aptly placed and help to  focus the viewer's attention on particular details through the eyes of  Schindler. The scenery and photography were excellent compared to other  movies I have seen and contribute to the whole atmosphere of the 1940s.  Some people may be put off a bit by the length (over 3 hours) but believe  me, every minute is worthwhile. Unlike other long movies, there are no  lulls or useless scenes -- everything counts. </p><p>The best part of the movie  without any doubt is the story itself, the tale of Oskar Schindler and how  he was able to save 1100 Jews from the Auschwitz gas chambers by employing  them in his enamelware factory and eventually his shelling factory.  Schindler's ambition and personal success shines through amidst the Jewish  tragedy and shows how one man, if he has the willpower, can accomplish what  appears to be impossible. Based on the novel by Thomas Keneally (which I  have not yet had the opportunity to read), this movie digs deep into the  human soul and shows how different people are able to survive. </p><p>There are  many touching moments in this film; in particular, near the end when the  war has been declared over and the Nazis must flee from the Soviet army.  This part and the modern-day segment that follows are both truly  heart-warming tributes. I finished watching this movie for about the fourth  time yesterday, and even though I didn't cry, tears welled up in my eyes  (and this rarely happens when I watch movies). </p><p>This movie is a must-see  not only for its excellence in the film genre but for the story it presents  to the viewer. Although it is not suitable for young children (due to its  violence and mature content), any mature individual should see it so they  can understand that a spark of good can still exist in a fire of evil. This  movie deserved all of the Academy Awards that it received and will likely  remain in top ten lists for at least the next fifty years. Highly  recommended.
      </p><div style="padding-top: 10px; clear: both; width: 100%;">