A csv (comma separated value) file is just a text file where the rows have notional columns separated by commas. (Or some other recognised separator character)
If you want to append to one you are just appending text as output as you have found.
You need to control the output and, at the basic level, make sure you have the right columns in the right order.
Depending on your purpose for the file you may also need to do some checking about the column widths. As a csv file anything that reads it (Excel for example) will simply adjust to the maximum width for each column (assuming it is not so wide that the target application cannot accept it) but if you are sending it to a predefined "database" of some sort you may need to control or field size as part of your export.
Does that help you at all?