Need Help Transfering 200 Files : Total Files Size Is 25 GB!

Portal Home > Knowledgebase > Articles Database > Need Help Transfering 200 Files : Total Files Size Is 25 GB!

Posted by InfiniteTech, 01-02-2008, 07:42 PM
Okay, so I couldn't keep my mouth shut (technically fingers). A customer wanted to upgrade servers and he needed a way to move the data across. Since I don't allow hard drives to be swapped, they have to do it manually all by themselves. I generally allow up-to 4 days for them to transfer data and make DNS changes, etc. But this time, I offered help! I agreed to move the data (darn me) and it just came out of me, involuntarily. God knows what just happened... but in a positive way, customer is extremely happy! So... Both servers are on cPanel - with root access (duh) 200 odd files which total to 25 GB 1 database about 100 MB in size (no biggie) Any ideas? I was planning on using one of my Windows 2003 servers (via remote desktop) to download the 25 GB and upload the 25 GB, but that sounds like a waste of resources and time. Any other better ideas? I am sure, it will surely come in handy for many others.
Posted by david510, 01-02-2008, 09:55 PM
Will not WHM multiple account transfer help here? Or Package all account in the source server, transfer the backup file to the remote server and restore it there. To package account /scripts/pkgacct username To restore account /scripts/restorepkg username
Posted by InfiniteTech, 01-02-2008, 09:59 PM
Hey David, Nope. WHM account transfer doesn't. It transfered all the files but while extracting them, it just stopped. If I make a tar ball of the entire public_html dir and then try to move it, how can I? I am sure wget doesn't support files larger than 2-3 GB in size. Best Regards, Ganesh Rao
Posted by david510, 01-02-2008, 10:27 PM
In the case of scripts, there is no need of this. If you specifically wish to move the public_html directory, you can tar it and scp it to the remote server. scp filename.tar.gz root@remote_server_ip:/home/ Run the above command from the source server. This will copy the tar file to the /home directory inside the remote server. You can run this in screen if you fear your net connection will be lost in between. Screen will make sure the process will continue running. To start a screen screen -S screen_name
Posted by domainworldaccess, 01-03-2008, 12:21 PM
So you have some accounts, but not all of them? Try to pick up where it left off. Delete the last account that was being worked on when it crashed, and try importing that one alone, and then import the rest after that one succeeds. How many did it extract before the crash?
Posted by InfiniteTech, 01-07-2008, 08:49 PM
Hold on... There is ONLY 1 account with a database! When cPanel performs the move it first reduces the entire account to a tar ball. Since the size of this account is too big it breaks it into smaller chunks (about 500 tar balls). On the new server, after downloading all the 500 of them when it unzips, it hangs up (stops basically) at 250~. I need to move this asap now.
Posted by foobic, 01-07-2008, 09:09 PM
If the problem is with gzipping the big files, I'd just move them somewhere else temporarily and make a full cpanel backup of the rest. Without the big files it should be fairly quick to make that, transfer it the new server and restore the account there. Then use rsync or scp to copy the big files across.
Posted by fog, 01-07-2008, 09:45 PM
Seconding foobic -- rsync and scp (both network-based file copy protocols) sound like they're exactly what you need.
Posted by astutiumRob, 01-07-2008, 10:30 PM
for most accounts a pkgacct and scp then use WHM to doa "restore full backup" is fine for very large accounts first scp over the db's then delete them from the original server then do the pkg acct, then copy them to the right place