I.T. Discussion Community!
-Collapse +Expand
Windows
Search Windows Group:

Advanced
-Collapse +Expand Windows Store
PRESTWOODSTORE

Prestwood eMagazine

November Edition
Subscribe now! It's Free!
Enter your email:

   ► MB LobbyWindows Users BoardWindows 9x, ME, NT, 2000, XP Topic   Print This     

Xcopy/robocopy most efficient way to trans. files?

Xcopy/robocopy most efficient way to trans. files? in Windows 9x, ME, NT, 2000, XP topic (part of our Windows Users group).

Quick Search: Xcopy/robocopy   efficient   Xcopy/robocopy most   Xcopy/robocopy most efficient   way trans  
S0nY
Baton Rouge, LA USA

The company i work for uses xcopy to copy our server files and folders to a backup folder every night, this process takes anywhere from 45 mins to 1hr 20 mins, other than trying robocopy or xcopy does anyone have a better more efficient ( faster ) idea or way that they do things to copy large tables/files/folders to a backup folder? Please post your ideas or what you do if it is different.

 Posted 11 years ago (Thread Starter)
Comment Quote
About S0nY -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.

Post ID #12805, 10 replies
Thread Started 6/3/2008 11:25:00 AM
View Counter=14404
Last Reply Posted 10/23/2013 10:14:24 AM)
Location=Baton Rouge, LA USA 
Joined=12 years ago   MB Posts=53  
Moderator
Mike Prestwood
Prestwood IT
Prestwood IT office in Citrus Heights, CA

Every file system has overhead such as setting Last Access, etc. These overhead procedures add time to copying files. Knowing which overhead procedures are ok to skip can be difficult at times. So, my first question is why is this an issue? An hour or two doesn't seem like a big deal unless you have to baby-sit the process or the process brings down users. {#think} 

With that question asked, I have a few suggestions:

Suggestion 1: Zip First

Zip the files into a local zip file then copy the zip file and unzip if needed. This "trick" cuts the time down tremendously when copying large amounts of typical files.

Suggestion 2: Use a Better Tool

I don't know how good RoboCopy is. I've read it's about twice as fast as XCopy but I've had great success with FileBack PC to automate our backing up of files from one server to the next. Although I don't usually skip the overhead procedures, it does allow you to skip some of them, adjust buffers, schedule retries on failed files, etc. Here's a screen shot from their Advanced setup dialog:

filebackpc

--
Mike Prestwood
Prestwood IT Solutions

 Posted 11 years ago
Comment Quote
About Mike Prestwood -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.
About Mike Prestwood

Mike Prestwood is a drummer, an author, and creator of the PrestwoodBoards online community. He is the President & CEO of Prestwood IT Solutions. Prestwood IT provides Coding, Website, and Computer Tech services. Mike has authored 6 computer books and over 1,200 articles. As a drummer, he maintains play-drums.com and has authored 3 drum books. If you have a project you wish to discuss with Mike, you can send him a private message through his PrestwoodBoards home page or call him 9AM to 4PM PST at 916-726-5675 x205.

Web Presence
Facebook, Prestwood IT Facebook page -- fan page. (Visit Me)
Twitter, Follow Prestwood IT on Twitter. (Visit Me)
LinkedIn, Prestwood IT company page on LinkedIn. (Visit Me)
YouTube, Prestwood IT YouTube Channel (Visit My Channel)
Website, My drum website where I sell my drum books. (http://www.play-drums.com)

Post ID #12832 (Level 1.1)  Reply to 12805
Thread Started 6/7/2008 1:10:18 PM
Location=Prestwood IT office in Citrus Heights, CA 
Joined=19 years ago   MB Posts=1410   KB Posts=1805   KB Comments=70   BLOG, Topics=4  
Daniel Fought
Prestwood IT
Home office in Fresno, CA.

Just to clearify what we are talking about, what is the database system?  Since "copy time" is not an issue then I assume that reindexing tables is a time issue.  Why are you reindexing the tables nightly?  How exactly are you acomplishing the reindexing?

Dan Fought
Senior Programmer Analyst
Prestwood IT Solutions
http://www.prestwood.com

 Posted 11 years ago
Comment Quote
About Daniel Fought -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.
About Daniel Fought
Danial Fought is a senior programmer analyst with Prestwood IT where he develops custom Windows software and custom websites. When Dan is not coding for clients, he participates in this online community. Coding specialties include Paradox/ObjectPAL, MS Access, Visual Basic, and VS.Net/VB.Net.

Post ID #12856 (Level 1.2)  Reply to 12805
Reply Posted 6/11/2008 10:09:31 AM
Location=Home office in Fresno, CA. 
Joined=17 years ago   MB Posts=401   KB Posts=12   KB Comments=4  
Daniel Fought
Prestwood IT
Home office in Fresno, CA.

I agree with Wes, Chimney Sweep is a great product and if you are going to rebuild the tables you should look closely at it.  However, if the indexes are maintained you shouldnt have to rebuild them nightly. 

There is a table build interactive form supplied with paradox and the tUtil32.dll is also included.

Dan Fought
Senior Programmer Analyst
Prestwood IT Solutions
http://www.prestwood.com

 Posted 11 years ago
Comment Quote
About Daniel Fought -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.
About Daniel Fought
Danial Fought is a senior programmer analyst with Prestwood IT where he develops custom Windows software and custom websites. When Dan is not coding for clients, he participates in this online community. Coding specialties include Paradox/ObjectPAL, MS Access, Visual Basic, and VS.Net/VB.Net.

Post ID #12867 (Level 1.3)  Reply to 12805
Reply Posted 6/11/2008 8:40:44 PM
Location=Home office in Fresno, CA. 
Joined=17 years ago   MB Posts=401   KB Posts=12   KB Comments=4  
Wes Peterson
Prestwood IT
Prestwood IT office in Citrus Heights, CA

Dan, all:

Dan:  See the prev post.  Definitely PDOX

First a disclaimer: I am no PDOX guru.  I just work for one. {#biggrin}

I have done a fair amount of Delphi coding using PDOX back ends, however.

One does have to wonder about the reindexing.  In a properly configured enviornment, and driven by a robust application, the PDOX database is pretty darned reliable.

I'll throw out just a couple of suggestions that, perhaps taken together, will lead to a more satisfactory nightly procedure.

First, there's the old rule, "If it ain't broke, don't fix it."  Why reindex all tables if none, one, or just a few are damaged?  It would make more sense, to me, to validate each table, then "repair" only those that fail validation.  I'd bet that 27 nights out of 30, the only necessary operation is the file copy.

No matter how validation is performed, I'd want a log of those tables that failed.  Over time, a pattern will probably emerge that'll suggest places in the application(s) that may need attention.

I've heard several hard-core Paradox developers lauding Chimney Sweep.  I don't know anything about it, but it might bear investigation.  It may well be that it can:

  • Perform validations faster
  • Report tables failing validation
  • Perform table repairs faster than Borland/Corel tools.

For a nightly regimen, it may take some batch file programming and command line parameter tweaking, but it seems to me that the effort to get a selective repair, rather than a slow, shotgun approach, would be worth it.

Wes Peterson
Senior Software Engineer
Prestwood IT Solutions
http://www.prestwood.com

 Posted 11 years ago
Comment Quote
About Wes Peterson -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.
About Wes Peterson

Wes Peterson is a Senior Programmer Analyst with Prestwood IT Solutions where he develops custom Windows software and custom websites using .Net and Delphi. When Wes is not coding for clients, he participates in this online community. Prior to his 10-year love-affair with Delphi, he worked with several other tools and databases. Currently he specializes in VS.Net using C# and VB.Net. To Wes, the .NET revolution is as exciting as the birth of Delphi.


Post ID #12857 (Level 1.4)  Reply to 12805
Reply Posted 6/11/2008 12:57:59 PM
Location=Prestwood IT office in Citrus Heights, CA 
Joined=15 years ago   MB Posts=158   KB Posts=163   KB Comments=34   BLOG, Topics=20  
Wes Peterson
Prestwood IT
Prestwood IT office in Citrus Heights, CA

I'd love to give you an example in code, but:

  1. See my caveat.  I am not a Paradox programmer.
  2. I don't even know if PDOX gives you access to the low level routines necessary.

Which are reasons I suggested exploring Chimney Sweep. 

That said, when Borland still owned Paradox, they issued a nifty DLL, TUtil32.DLL, that surfaces the necessary calls.

Included are:

function TUVerifyTable(hTUSession: hTUses;
                       pszTableName,
                       pszDriverType,
                       pszErrTableName,
                       pszPassword: PChar;
                       iOptions: integer;
                       var piErrorLevel: Integer): DBIResult; stdcall;
function TURebuildTable(hTUSession: hTUses;
                        pszTableName,
                        pszDriverType,
                        pszBackupTableName,
                        pszKeyviolName,
                        pszProblemTableName: PChar;
                        pCrDesc: pCRTblDesc): DBIResult; stdcall;

Along with the DLL, Borland also provided an interacive utility program, written in Delphi, and which you can obtain by perusing the linked KB article at the top of this thread. That download's installer will also get you the TUtil32.DLL.

Now, because the above is an interactive program, it's not exactly suitable for an automated, nightly batch process.

Using the calls in TUtil32.DLL, we could create command-line programs for you that would do the trick for a "smart" nightly regimen. 

Imagine a command-line program we'll call PdoxValidate that, when run, would validate every table in your database, and emit a simple text file with a list of tables that failed - and would append the same to a date-stamped log file.

Then imagine another command line program we'll call PdoxRebuild.  When run, it would examine the short file of tables that failed validation, and rebuild only them, then delete the file so it's out of the way for the next night's run.

Your batch file, then, might look like this:

PdoxValidate
PdoxRebuild
XCopy....

But I hesitate to suggest such an effort before you've exhausted your exploration of Chimney Sweep.  I'm not a big fan of reinventing the wheel.

Wes Peterson
Senior Software Engineer
Prestwood IT Solutions
http://www.prestwood.com

 Posted 11 years ago
Comment Quote
About Wes Peterson -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.
About Wes Peterson

Wes Peterson is a Senior Programmer Analyst with Prestwood IT Solutions where he develops custom Windows software and custom websites using .Net and Delphi. When Wes is not coding for clients, he participates in this online community. Prior to his 10-year love-affair with Delphi, he worked with several other tools and databases. Currently he specializes in VS.Net using C# and VB.Net. To Wes, the .NET revolution is as exciting as the birth of Delphi.


Post ID #12861 (Level 1.5)  Reply to 12805
Reply Posted 6/11/2008 4:11:33 PM
Location=Prestwood IT office in Citrus Heights, CA 
Joined=15 years ago   MB Posts=158   KB Posts=163   KB Comments=34   BLOG, Topics=20  
Wes Peterson
Prestwood IT
Prestwood IT office in Citrus Heights, CA

Since my last post, I found a better article in our knowledge base, and linked it at the top of this thread. The Repairing Paradox Tables article still might be useful.

Wes Peterson
Senior Software Engineer
Prestwood IT Solutions
http://www.prestwood.com

 Posted 11 years ago
Comment Quote
About Wes Peterson -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.
About Wes Peterson

Wes Peterson is a Senior Programmer Analyst with Prestwood IT Solutions where he develops custom Windows software and custom websites using .Net and Delphi. When Wes is not coding for clients, he participates in this online community. Prior to his 10-year love-affair with Delphi, he worked with several other tools and databases. Currently he specializes in VS.Net using C# and VB.Net. To Wes, the .NET revolution is as exciting as the birth of Delphi.


Post ID #12869 (Level 1.6)  Reply to 12805
Reply Posted 6/12/2008 12:53:05 PM
Location=Prestwood IT office in Citrus Heights, CA 
Joined=15 years ago   MB Posts=158   KB Posts=163   KB Comments=34   BLOG, Topics=20  
S0nY
Baton Rouge, LA USA

Actually my problem really isnt the " time " with the file coping. Before my files are copied at the end of the night my paradox tables are all reindexed and the tables are large and it takes time to do this, after the reindex, the files are then copied over to a "lookups" location, SOMETIMES the tables may not get reindexed correctly or may have a size issue all kind of random problems sometimes happen, and after the file copy to the lookup folder ( this folder is the lookups the processors in the office use to work with ) the next day at work in the morning the processors cannot do lookups because some tables may be messed up, when this happens, we have them work from a day or so older back up while we reindex the tables again and make sure the tables are working, then x copy them over to the lookups folder, then switch the processors alias back to the " todaysreal" lookups for them to work.

In the morning when this happens and i have to wait on x copy to transfer the files it takes time and the processors cant see new information and just causes problems, i was just trying to think of some ways to speed up these processes...

 Posted 11 years ago (Thread Starter)
Comment Quote
About S0nY -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.

Post ID #12849 (Level 1.7)  Reply to 12805
Reply Posted 6/10/2008 3:32:25 PM
Location=Baton Rouge, LA USA 
Joined=12 years ago   MB Posts=53  
S0nY
Baton Rouge, LA USA

To Dan's reply,

I am using paradox ver. 11.

This is a small example of what code i use, this code here just reindexes a few fields in the invoices.db, the real script i have has more tables and fields: ( i copied and pasted the code in here and the code looks "jumbled" here sorry ; / )

var
  t table
  q query
  timebefore, timeafter time
  tc tcursor
  fs filesystem
  tv tableview
  app application
  src  Table
  dest String
endvar

  timebefore=time()
  app.maximize()
  sleep()
      sleep(1000)
  
  tablecount=0
  indexcount=0
  totalcount=40

  ;message(+" "+string(tablecount)+" "+string(indexcount)+" out of "+string(totalcount))

  [;_--]-----  INDEX TABLES ON ACB\ACB21 DIRECTORY
  tablecount=tablecount+1

      t.attach("\\\\acb\\acb21lookups\\invoices.db")
  if t.lock("full")=true then
  indexcount=indexcount+1
   message("CONTACTNUM of Invoices "+string(tablecount)+" "+string(indexcount)+" out of "+string(totalcount))
         sleep(100)
   t.reindex("CONTACTNUM")
   t.unlock("full")
  endif
  t.unattach()

      t.attach("\\\\acb\\acb21lookups\\invoices.db")

  if t.lock("full")=true then

   indexcount=indexcount+1
   message("CheckMoName of Invoices "+string(tablecount)+" "+string(indexcount)+" out of "+string(totalcount))
   t.reindex("CHECKMONAME")
   t.unlock("full")

  endif
  t.unattach()

      t.attach("\\\\acb\\acb21lookups\\invoices.db")

  if t.lock("full")=true then
  indexcount=indexcount+1
   message("CHECKMONUM of Invoices "+string(tablecount)+" "+string(indexcount)+" out of "+string(totalcount))
   t.reindex("CHECKMONUM")
   t.unlock("full")
  endif
  t.unattach()

      t.attach("\\\\acb\\acb21lookups\\invoices.db")

  if t.lock("full")=true then
  indexcount=indexcount+1
   message("CREDITCARDNUM of Invoices "+string(tablecount)+" "+string(indexcount)+" out of "+string(totalcount))
   t.reindex("CREDITCARDNUM")
   t.unlock("full")
  endif
  t.unattach()

      t.attach("\\\\acb\\acb21lookups\\invoices.db")

  if t.lock("full")=true then
  indexcount=indexcount+1
   message("INVOICENUM of Invoices "+string(tablecount)+" "+string(indexcount)+" out of "+string(totalcount))
   t.reindex("INVOICENUM")
   t.unlock("full")
  endif
  t.unattach()


      timeafter=time()

      tc.open(":acb:timeofupdatescript.db")
      tc.edit()
      tc.insertrecord()
      tc."date"=today()
      tc.timebefore=timebefore
  tc.timeafter=timeafter

      exit()

 Posted 11 years ago (Thread Starter)
Comment Quote
About S0nY -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.

Post ID #12858 (Level 1.8)  Reply to 12805
Reply Posted 6/11/2008 2:14:12 PM
Location=Baton Rouge, LA USA 
Joined=12 years ago   MB Posts=53  
S0nY
Baton Rouge, LA USA

Very true Wes, if it is't broken then you shouldnt fix it. You have a point. Would you mind sharing an example in code of what you mean by "to validate each table"?

 Posted 11 years ago (Thread Starter)
Comment Quote
About S0nY -Collapse +Expand
Visit Profile
Approved member.
Member subscribes to this thread with a verified email.

Post ID #12859 (Level 1.9)  Reply to 12805
Reply Posted 6/11/2008 2:24:16 PM
Location=Baton Rouge, LA USA 
Joined=12 years ago   MB Posts=53  
Most Recent Post
Anonymous Post

GSCopy Pro v6.0 (RoboCopy Alternative) with Open File Agent

GSCopyPro is a single command-line tool (CLI) that can copy, replicate and move files from one folder to another. This folder can be on the same machine/ server or another server elsewhere. What makes GSCopyPro stand out from other competitors is the fact it works on 32-bit as well as 64-bit systems and has no restrictions. It can easily be scheduled to run as a scheduled task and fully automated. GSCopyPro also comes with an open file agent which can copy files that are locked/ opened by other processes. This feature is supported in all windows versions from widows XP/ 2003 and later.

Go To:>> http://www.gurusquad.com/GSCOPYPRO

---
GSCopy Pro v6.0 (RoboCopy Alte
 Posted 6 years ago
Comment Quote

Post ID #15084 (Level 1.10)  Reply to 12805
Reply Posted 10/23/2013 10:13:32 AM

Revive Thread!

Add a comment to revive this old thread and make this archived thread more useful.

Write a Comment...
Full Editor
...
Sign in...

If you are a member, Sign In. Or, you can Create a Free account now.


Anonymous Post (text-only, no HTML):

Enter your name and security key.

Your Name:
Security key = P1215A1
Enter key:
Icon: A Post    Thread    Idea    Important!    Cool    Sad    No    Yes    Includes a Link...   
Thread #12805 Counter
14404
Since 6/3/2008

Regarding...

Linked Knowledge Base Article.

This thread is linked to the following KB article.


Mike Prestwood
1. Copy Paradox Tables with Confidence What is the best way to copy my Paradox tables?
Posted to KB Topic: Paradox Tables
11 years ago

FAQ
Nothing New Since Your Last Visit
11457
Hits

Corel Paradox / ObjectPAL Coding


Follow PrestwoodBoards on: 


©1995-2019 PrestwoodBoards  [Security & Privacy]
Professional IT Services: Coding | Websites | Computer Tech