I.T. Discussion Community!
-Collapse +Expand
V.FoxPro
Search V.FoxPro Group:

Advanced
-Collapse +Expand V.FoxPro Store

Prestwood eMagazine

July Edition
Subscribe now! It's Free!
Enter your email:

   ► MB LobbyVisual FoxPro BoardMicrosoft Visual FoxPro (VFP) Topic   Print This     

Massive amounts of appends

Massive amounts of appends in Microsoft Visual FoxPro (VFP) topic (part of our Visual FoxPro group).

Quick Search: Massive   amounts   appends   Massive amounts   Massive amounts of  
Jeff.R
-- UNK

If I had 1000 servers scrapping (spidering) the web, at say the Amazon  Elastic Compute Cloud (Amazon EC2).

Could Foxpro append massive amounts of data?

Is there a upper limit to the amount of record locks and unlocks (with "append blank" or "insert into") that can be processed per second, minute or hour?

Has anybody written about this before?

Is there a breakdown point?

Is there a failure point? If there is, what would fail, eventually, if you continued to add servers?

Or is there a slow down point where it isn't worth having x amount of "append blanks" where they just start blocking each other?

As usual, you folks are fantastic.

Thanks
Jeff

 Posted 10 years ago (Thread Starter)
Comment Quote
About Jeff.R -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.

Post ID #14176, 2 replies
Thread Started 6/16/2010 12:03:36 PM
View Counter=4649
Last Reply Posted 6/23/2010 1:18:24 PM)
Location=-- UNK 
Joined=10 years ago   MB Posts=1  
Moderator
Mike Prestwood
Prestwood IT
Prestwood IT office in Citrus Heights, CA

Yes, there are limits to database inserts. The more robust the database product, the more it can handle. In fact, the more robust hardware it is running on, the more it can handle. For example, with FoxPro, if you're inserting into xBASE (a file based dBASE DB) then you'll be much more limited than for example to MS SQL 2008 which can handle more than SQL 2005. Anyway, I hope that gives you an idea. Follow up with an additional question if you wish.

--
Mike Prestwood
Prestwood IT Solutions

 Posted 10 years ago
Comment Quote
About Mike Prestwood -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.
About Mike Prestwood

Mike Prestwood is a drummer, an author, and creator of the PrestwoodBoards online community. He is the President & CEO of Prestwood IT Solutions. Prestwood IT provides Coding, Website, and Computer Tech services. Mike has authored 6 computer books and over 1,200 articles. As a drummer, he maintains play-drums.com and has authored 3 drum books. If you have a project you wish to discuss with Mike, you can send him a private message through his PrestwoodBoards home page or call him 9AM to 4PM PST at 916-726-5675 x205.

Web Presence
Facebook, Prestwood IT Facebook page -- fan page. (Visit Me)
Twitter, Follow Prestwood IT on Twitter. (Visit Me)
LinkedIn, Prestwood IT company page on LinkedIn. (Visit Me)
YouTube, Prestwood IT YouTube Channel (Visit My Channel)
Website, My drum website where I sell my drum books. (http://www.play-drums.com)

Post ID #14184 (Level 1.1)  Reply to 14176
Thread Started 6/20/2010 9:25:52 PM
Location=Prestwood IT office in Citrus Heights, CA 
Joined=20 years ago   MB Posts=1410   KB Posts=1805   KB Comments=69   BLOG, Topics=4  
Most Recent Post
AIBreveleri

Most "enterprise" database engines have one or more bulk load techniques.  For example, PostgreSQL has "COPY FROM", MySQL has "LOAD DATA INFILE", and Oracle has its Data Pump, its SQL*Loader, and some kind of native import utility.  Smaller "convenience" engines such as FoxPro, Access, and BDE do not have bulk insert methods.  If you absolutely must insert large amounts of data quickly, you will need to move to a database that supports bulk insertion.

I suspect that if a database engine does not allow bulk insertion, then it is also not recommended for any heavy lifting, such as large record counts, many simultaneous users, deeply nested subqueries, etc.  So that even if you could get your FoxPro installation to accept data rapidly, you would soon encounter other frustrating limitations.

-Al.

 Posted 10 years ago
Comment Quote
About AIBreveleri -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.

Post ID #14191 (Level 1.2)  Reply to 14176
Reply Posted 6/23/2010 1:17:23 PM
Location= 
Joined=19 years ago   MB Posts=286   KB Comments=8  

Revive Thread!

Add a comment to revive this old thread and make this archived thread more useful.

Write a Comment...
Full Editor
...
Sign in...

If you are a member, Sign In. Or, you can Create a Free account now.


Anonymous Post (text-only, no HTML):

Enter your name and security key.

Your Name:
Security key = P1165A1
Enter key:
Icon: A Post    Thread    Idea    Important!    Cool    Sad    No    Yes    Includes a Link...   
Thread #14176 Counter
4649
Since 6/16/2010

Follow PrestwoodBoards on: 


©1995-2020 PrestwoodBoards  [Security & Privacy]
Professional IT Services: Coding | Websites | Computer Tech