Results 1 to 4 of 4
  1. #1

    C struct - how to read a line and delete it?

    I have created a simple C struct:

    typedef struct msgfmt {
    byte starthdr;
    byte dest;
    byte src;
    byte prio;
    byte error;
    byte type;
    byte pad;
    byte endhdr;
    } msgfmt_t;

    I insert data into the struct using this function:

    inline void parsehdr(byte *buf, int *dest, int *src, int *prio, int *error)
    msgfmt_t hdr;

    memcpy(&hdr, buf, HDRSIZE);
    if (hdr.starthdr != SI && hdr.endhdr != SO) {
    quit("Malformed header", 1);
    if (dest != NULL) {
    *dest = hdr.dest;
    if (src != NULL) {
    *src = hdr.src;
    if (prio != NULL) {
    *prio = hdr.prio;
    if (error != NULL) {
    *error = hdr.error;

    The problem now is how to read a line. The idea is to use key which I use to identify one row. Any idea?


  2. #2
    Join Date
    Aug 2002
    Superior, CO, USA
    You don't show how you're storing this data. When you say "line" you imply that you've got a simple text file of records. However, since you've got byte data types it may be difficult to store the data in a text file.

    An alternative is to use a fixed width struct (which you have) and write one struct after another to a file. Just like in the "line" example, the only way to search is via a linear search through the file. However, you can seek() to a particular record which may be faster for some operations.

    I have to ask though - why not use a database? It sounds like you're trying to do database-like things with this struct - split it up into columns and save it. Reading and deleting will be pretty fast (way faster if you can identify a unique identifier for each record) and you don't have to write all of this yourself.
    Need Java help? Want to help people who do? Sit down with a cup of Java at the hotjoe forums.

  3. #3
    this operation will be very I/O intensive

    Do you know how I can read the "line" based on key

  4. #4
    Join Date
    Aug 2002
    Superior, CO, USA
    Quote Originally Posted by rcbandit View Post
    this operation will be very I/O intensive
    Which is an even better reason to use a database. However, you've got other "roll your own" choices.

    The first would be a simple linear search. In this, you'll walk through every line one at a time looking for the key. If you've got a small (say less than 10000 records) this may work ok as the O/S will end up caching the file for you so that there will be minimal disk I/O. A delete or update is very expensive because you'll have to read the part before the deleted record into memory or a temporary file, write it out to a new file then read the second part into memory and write it out. You have to rewrite the entire file to remove or update a single record. An insert will be super fast though as you'll just seek() the end of the file.

    The second is to use two files. The first will be your struct written end to end. The second will be an index file of key -> file location mappings. Basically you will use the mapping file to do reads very quickly by seek()'ing to the location of the data file for a read. A delete will still be very expensive as you'll now have two files to update. Inserts will still be fast. A variation on this is to have the index table contain a flag to indicate a deleted record and that record will be skipped. Later another process can coalesce the deleted records in the background.

    Neither of these are at all efficient for a delete. The B-tree algorithm is much more efficient with all of insert, update, and delete but now you've built your own database.

    A well indexed table, backed by a database with enough memory to cache the data will still likely be faster and take you 1/10th to 1/100th the time of development.

    How many of these structs do you expect to have? At a total guess, 10000 or less will not matter much as you could cache the entire set in memory (it would only be around 80k), depending on your platform (it would not fit on an Arduino for example). Heck, 1,000,000 is only 8Mb or so. But if you're talking tens of millions or tens of billions then you're unlikely to be able to store them all in memory.

    Additionally, you have to figure out your transaction rate - will you be inserting and selecting most often? Then the index file solution will be extremely fast. If you only rarely delete then it is an appropriate solution. Or will you be deleting, inserting, and selecting with about the same frequency? Now the delete overhead could outweigh the other things.

    If you're just looking for code you can find lots of examples online or let us know. Knowing your platform would help us help you.
    Need Java help? Want to help people who do? Sit down with a cup of Java at the hotjoe forums.

Similar Threads

  1. read line-by-line text file with ASP
    By orongo in forum Programming Discussion
    Replies: 4
    Last Post: 01-05-2007, 12:31 PM
  2. read a line from a text, if exist delete it
    By UrlGuy in forum Programming Discussion
    Replies: 3
    Last Post: 04-02-2005, 04:13 PM
  3. how read a specified line from a file
    By galamagalama in forum Programming Discussion
    Replies: 3
    Last Post: 07-22-2004, 08:08 AM
  4. delete first line of text file.
    By jasong in forum Programming Discussion
    Replies: 5
    Last Post: 07-15-2004, 02:06 PM
  5. Delete files in Linux by date from Command Line?
    By KIA-Joe in forum Programming Discussion
    Replies: 4
    Last Post: 06-22-2004, 12:49 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts