
|
View Full Version : Please help - How can I put this into a database quickly?
gemasis 07-24-2009, 11:11 PM Hello again,
It`s been a while since I was last here. My question is, I have a CSV with mobile phone models and their specs. I attached a screenshot sample. The structure is not ideal to go into a database so I was hoping someone here can suggest any program or method I can use to get this into a database a.s.a.p. There are 100s of phones and manual input isnt an option at this point.
Thanx in advance
I've found Navicat (http://www.navicat.com/) to be an incredibly handy tool for that sort of thing. You might consider giving it a look. Don't let the cost sway you, it's well worth it for the time you save, IMHO.
tim2718281 07-25-2009, 10:07 AM Hello again,
It`s been a while since I was last here. My question is, I have a CSV with mobile phone models and their specs. I attached a screenshot sample. The structure is not ideal to go into a database so I was hoping someone here can suggest any program or method I can use to get this into a database a.s.a.p. There are 100s of phones and manual input isnt an option at this point.
Thanx in advance
With only a few hundred phone, probably the quickest way for non-programmers is to load the CSV file in a text editor, and go through line by line manually inserting a comma at the begining of each line then deleting the previous line end, so that you end up with one phone per line.
Then import the resulting CSV file into a spreadsheet, check that the columns containing names are correct, and then delete those columns.
That should all take you less than an hour.
Then save the result, and import it into your database.
mwatkins 07-26-2009, 08:03 AM If your data is consistent - all records have the same number of "fields" whether there is data in them or not, then the problem is simply turning a horizontal CSV into a vertical. a vim (or insert your editor here) macro could do this. Or you could write a little program - the guts of this is 20 lines long:
"""
h2v_csv.py : An example in Python which takes vertically oriented
data from a CSV file and turns it into horizontally oriented data
- one row (line) per record - to make it easier to import into a database.
"""
import csv
import re
word_re = re.compile(r'\b\w+\b')
example_csv_file_contents = """
Nokia e75,
Camera,3.2 Mp
Bluetooth,v2.0
Music Player,MP3/AAC/eAAC+ player
FM Radio,Yes
GPRS,Class 32
EDGE,Class 32
Talk Time,up to 5.30 hr
,
Nokia e71,
Camera,3.2 Mp
Bluetooth,v2.0
Music Player,MP3/AAC/eAAC+ player
FM Radio,Yes
GPRS,Class 32
EDGE,Class 32
Talk Time,"up to 3 hr, 5 hr with alternate battery"
"""
# this creates some test data
open("vertical_phones.csv", "w").write(example_csv_file_contents)
def get_data(filename):
phones = []
def read():
for row in csv.reader(open(filename)):
if len(row) == 2 and row != ['','']:
yield row
for key, value in read():
if value == '':
# a new phone encountered
model = []
model.append(key)
phones.append(model)
else:
model.append(value)
return phones
def write_as_csv(phones, filename):
writer = csv.writer(open(filename, 'w'))
for phone in phones:
writer.writerow(phone)
if __name__ == '__main__':
phones = get_data("vertical_phones.csv")
write_as_csv(phones, "phones.csv")
mwatkins 07-26-2009, 08:31 AM ps, ignore the unused import "re" and "word_re"; I dashed together a version that imported everything into a MySQL db but thought better of that, only I didn't delete all the code. With the data in rows the OP ought to get to the next step.
gemasis 07-26-2009, 09:46 AM thanx, I will try these solutions!
mwatkins 07-26-2009, 10:58 AM The output produced by running against the sample data is:
$ python csv_example.py
$ more phones.csv
Nokia e75,3.2 Mp,v2.0,MP3/AAC/eAAC+ player,Yes,Class 32,Class 32,up to 5.30 hr
Nokia e71,3.2 Mp,v2.0,MP3/AAC/eAAC+ player,Yes,Class 32,Class 32,"up to 3 hr, 5 hr with alternate battery"
If you want to use the code more or less as-is, comment out the line:
open("vertical_phones.csv", "w").write(example_csv_file_contents)
... and simply save your data to convert in a file called "vertical_phones.csv".
grahamrb 07-30-2009, 03:10 AM another consideration is to pass the work to someone offshore for a small fee, not sure about webhostingtalk but you'll certainly find someone on digitalpoint forums who will do this very cheaply. Just have them convert it to a proper tabulated spreadsheet for you which you can then convert to CSV and import into your database.
|