Text-Delimited database - help

Results 1 to 6 of 6

Thread: Text-Delimited database - help

  1. #1
    GMAN2 Guest

    Default Text-Delimited database - help

    I have a software delivery system that sends out software on my company&#039s intranet. After the delivery the system let the pusher (user who is pushing the software) know if the software was successfully pushed or it failed. This data is coming from the unix server. Currently the data is coming in the form of text (csv) files. The data tells us that which hostname received the software and which failed<BR><BR>The management uses my asp page to access this text file and know which hostname received the software sucessfully and which failed. Looking at the no of softwares being pushed to all the servers this text file is growing like crazy. Right now it has about 30,000 rows and expected to grow more. Due to this the search becomes slow. So what should i do to make this search faster.<BR><BR>Database table?<BR>Create a table which gets the data feeded directly from the unix servers...how do I do that? Every time user pushes the software and delivery is over...I want the data to come from unix to my table and get appended. How to do that. <BR><BR>Also I am using SQL 7.0 . Will my database will really suffer if I have a table which has about 50000 rows? And more than one user are querying while system is feeding the data to it. If this is the case then would it be just better to keep the text file growing?<BR><BR>Please help!

  2. #2
    Tobey Unruh Guest

    Default RE: Text-Delimited database - help

    50000 rows in SQL is nothing. I was dealing with one db that had well over 200,000 rows. As far as the table suffereing from one person querying it and another feeding it, you shouldn&#039t have a problem unless it is a lot of people on both sides doing it (and by a lot, I mean, 100-150 at any one time).<BR><BR>Hope this helps,<BR>Tobey

  3. #3
    GMan2 Guest

    Default RE: Text-Delimited database - help

    Kool:<BR><BR>So my best option is to maintain a database table. Now how do I go about doing this? Should I use the text file to feed the data in to the table? How do I do that... Thanks Tobey

  4. #4
    Tom Crowe Guest

    Default RE: Text-Delimited database - help

    Use data transformation services to import the text file into a new table. Rewrite your code to use the table instead of the text file.

  5. #5
    GMan2 Guest

    Default RE: Text-Delimited database - help

    But I need to transfer this data periodically. Evertime the new data is added to this file from the unix server, I have to move that to the sql table. How do I do that without manual intervention? Is it possible with sql database transformation?<BR><BR>Something like running batch file from a simple command. What I think I really want to do is...I want the user to click something on the web and so the data is transfered from the text file to the database. Then the text file is deleted.

  6. #6
    Tobey Unruh Guest

    Default RE: Text-Delimited database - help

    The easiest thing to do would be to get your UNIX box setup to feed directly into the SQL table and bypass that text file altogether.<BR><BR>Unfortunately, I haven&#039t the first clue on how to do that. It can be done, I just don&#039t know how.<BR><BR>If you can&#039t figure out how to do that or simply do not want to figure it out, then you would have to set the script that runs when management activates it to update the database from that text file. This is feasable and simple and would ensure that they are looking at the most recent data.<BR><BR>The downside to that is it may be slow and the script is doing a lot of work each time it is ran. So if you have people hitting this thing 20 times an hour, your servers are going to be getting kicked quite often for little tasks.<BR><BR>Again, this isn&#039t necessarily a bad thing, just not the most efficient. If you can figure out how to get the UNIX server to feed directly into the SQL table, then every call management makes to that script will only be calling a recordset vs having to dump a text file into the db first then calling a recordset.<BR><BR>If any of this is too confusing or you would like clarification or help, please feel free to email me and I will help out the best I can.<BR><BR>Hope this helps,<BR>Tobey<BR>tobeyu@hotmail.com

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts