Web Hosting Talk







View Full Version : Please help - How can I put this into a database quickly?


gemasis
07-24-2009, 11:11 PM
Hello again,

It`s been a while since I was last here. My question is, I have a CSV with mobile phone models and their specs. I attached a screenshot sample. The structure is not ideal to go into a database so I was hoping someone here can suggest any program or method I can use to get this into a database a.s.a.p. There are 100s of phones and manual input isnt an option at this point.

Thanx in advance

bear
07-25-2009, 08:39 AM
I've found Navicat (http://www.navicat.com/) to be an incredibly handy tool for that sort of thing. You might consider giving it a look. Don't let the cost sway you, it's well worth it for the time you save, IMHO.

tim2718281
07-25-2009, 10:07 AM
Hello again,

It`s been a while since I was last here. My question is, I have a CSV with mobile phone models and their specs. I attached a screenshot sample. The structure is not ideal to go into a database so I was hoping someone here can suggest any program or method I can use to get this into a database a.s.a.p. There are 100s of phones and manual input isnt an option at this point.

Thanx in advance

With only a few hundred phone, probably the quickest way for non-programmers is to load the CSV file in a text editor, and go through line by line manually inserting a comma at the begining of each line then deleting the previous line end, so that you end up with one phone per line.

Then import the resulting CSV file into a spreadsheet, check that the columns containing names are correct, and then delete those columns.

That should all take you less than an hour.

Then save the result, and import it into your database.

mwatkins
07-26-2009, 08:03 AM
If your data is consistent - all records have the same number of "fields" whether there is data in them or not, then the problem is simply turning a horizontal CSV into a vertical. a vim (or insert your editor here) macro could do this. Or you could write a little program - the guts of this is 20 lines long:

"""
h2v_csv.py : An example in Python which takes vertically oriented
data from a CSV file and turns it into horizontally oriented data
- one row (line) per record - to make it easier to import into a database.
"""
import csv
import re

word_re = re.compile(r'\b\w+\b')

example_csv_file_contents = """
Nokia e75,
Camera,3.2 Mp
Bluetooth,v2.0
Music Player,MP3/AAC/eAAC+ player
FM Radio,Yes
GPRS,Class 32
EDGE,Class 32
Talk Time,up to 5.30 hr
,
Nokia e71,
Camera,3.2 Mp
Bluetooth,v2.0
Music Player,MP3/AAC/eAAC+ player
FM Radio,Yes
GPRS,Class 32
EDGE,Class 32
Talk Time,"up to 3 hr, 5 hr with alternate battery"
"""
# this creates some test data
open("vertical_phones.csv", "w").write(example_csv_file_contents)

def get_data(filename):
phones = []
def read():
for row in csv.reader(open(filename)):
if len(row) == 2 and row != ['','']:
yield row

for key, value in read():
if value == '':
# a new phone encountered
model = []
model.append(key)
phones.append(model)
else:
model.append(value)
return phones

def write_as_csv(phones, filename):
writer = csv.writer(open(filename, 'w'))
for phone in phones:
writer.writerow(phone)

if __name__ == '__main__':
phones = get_data("vertical_phones.csv")
write_as_csv(phones, "phones.csv")

mwatkins
07-26-2009, 08:31 AM
ps, ignore the unused import "re" and "word_re"; I dashed together a version that imported everything into a MySQL db but thought better of that, only I didn't delete all the code. With the data in rows the OP ought to get to the next step.

gemasis
07-26-2009, 09:46 AM
thanx, I will try these solutions!

mwatkins
07-26-2009, 10:58 AM
The output produced by running against the sample data is:
$ python csv_example.py
$ more phones.csv
Nokia e75,3.2 Mp,v2.0,MP3/AAC/eAAC+ player,Yes,Class 32,Class 32,up to 5.30 hr
Nokia e71,3.2 Mp,v2.0,MP3/AAC/eAAC+ player,Yes,Class 32,Class 32,"up to 3 hr, 5 hr with alternate battery"
If you want to use the code more or less as-is, comment out the line:
open("vertical_phones.csv", "w").write(example_csv_file_contents)

... and simply save your data to convert in a file called "vertical_phones.csv".

grahamrb
07-30-2009, 03:10 AM
another consideration is to pass the work to someone offshore for a small fee, not sure about webhostingtalk but you'll certainly find someone on digitalpoint forums who will do this very cheaply. Just have them convert it to a proper tabulated spreadsheet for you which you can then convert to CSV and import into your database.