PC hardware gurus

RatLabGuy

RatLabGuy

You look like a monkey and smell like one too
Joined
May 18, 2005
Location
Churchville, MD
Feeling too lazy to register on an actual PC forum just to ask - lets hear from the hardware nerds.

In my lab I have 2 PCs I need to transfer data between - large files, lets just say is 1 gig range.
I need to be able to make the transfer of single files as quickly as possible (targeting tens of seconds)

Unfortunately the "origin" PC is an old Optiplex 260 using IDE, I don't know whether ATA66 or what. Hard drive is about to be replaced w/ 7200 RPM at least.
Anyway - what's the best way to do the transfer, and the rate limiting step? Can get gigabit ethernet cards and crossover...or a switch...router? but will the IDE or bus rate actually limit it? E.g. do I need to also consider getting a SATA controller card instead?
 
I tried to do this a while back (in the days of dial up) in which FTPing it online would have taken a week. I went to a computer store and to transfer data from one computer would cost ya. If you have a home network, you can share the file and just drag and drop to the network drive.

If you dont have a network but have blazin fast internet, just upload to a free FTP site. Most sites give you 1-5GB range free. Sign up. Upload, then download to the other computer. It is also useful so your files will be online at your disposal.

.... I wouldnt suggest uploading to a site if you have kiddie porn er uh sensative information on there. Ha ha but that should work out.
 
BTW I have a wirele$$ ethernet hub for home networking avail if ya need one.
 
Gigabit speeds are possible on a PCI bus, but the gigabit card just about saturates the available bandwidth on the northbridge.

It should transfer a 1 gig file in under a minute, possibly in as little as 20-30 seconds. But you're definitely going to be limited by the bus speeds and/or the hard drive.

Is this one static file that needs to be moved in its entirety, or is it just pieces that need updated? If the latter, you could look at rsync or Deltacopy or something like that. It would only move the portions of the file that have changed.
 
y2kcrawler said:
If you dont have a network but have blazin fast internet, just upload to a free FTP site...
Click to expand...

This is in a lab w/o net acces; the point is simply to get a isngle file from PC A to PC B as fast as possible.

saf-t scissors said:
Gigabit speeds are possible on a PCI bus, but the gigabit card just about saturates the available bandwidth on the northbridge.

It should transfer a 1 gig file in under a minute, possibly in as little as 20-30 seconds. But you're definitely going to be limited by the bus speeds and/or the hard drive.

Is this one static file that needs to be moved in its entirety, or is it just pieces that need updated? If the latter, you could look at rsync or Deltacopy or something like that. It would only move the portions of the file that have changed.
Click to expand...

1 static file.
FYI, PC A is collecting EEG data from a study participant off of the EEG equipment, streams to a data file. 72 channels at 24 bit precision, sampled 8196 times a second - adds up fast.
After 8 minutes of work, we stop, take a break, get ready for the next run. That's when PC B gets the data, so it can start analyzing it. Since the subject wants to get back to work, we needteh transfer to go fast so we can re-start.
But it's pushing a gig in size.

So I guess I'm wondering, lets say it is an ATA66 onboard controller, is there really much advantage to trying to bipass it and go SATA, or is the PCI bus still going to be the limiter there?
 
Feeling too lazy to register on an actual PC forum just to ask - lets hear from the hardware nerds.
In my lab I have 2 PCs I need to transfer data between - large files, lets just say is 1 gig range.
I need to be able to make the transfer of single files as quickly as possible (targeting tens of seconds)

Simple, pull the IDE drive out, hook it to new machine, drag and drop. I Am assuming the new computer is of decent quality and has IDE controller. If not buy a SATA to IDE adaptor.

Unfortunately the "origin" PC is an old Optiplex 260 using IDE, I don't know whether ATA66 or what. Hard drive is about to be replaced w/ 7200 RPM at least.

RPM doesn't play that big of a deal in speed for normal users.

Anyway - what's the best way to do the transfer, and the rate limiting step?

Rate limiting what? I do this professionally and never wanted to limit the rate.

(online dictionary) "rate limiting step" - The slowest step in a metabolic pathway, or the step in an enzymatic reaction that requires the greatest amount of energy to initiate.

Can get gigabit ethernet cards and crossover...or a switch...router?

Yes, that would get the job done, but what if there is a unique network setting on one of the computers or other network problem. Definately takes too long to set up.

but will the IDE or bus rate actually limit it?

IDE is limited to a max of 133 with most from that era being 66. It's the speed on the controller on the drive.

E.g. do I need to also consider getting a SATA controller card instead?

No need to buy a SATA controller. If replacing the drive in the old computer just put in a IDE drive. Modern IDE is much faster than old IDE.



I just read more details. It seems your major thing is minimal down time for the origin computer. Use a USB flash drive. Since the 260 has USB 2.0 it should be very quick.
 
GubNi said:
I just read more details. It seems your major thing is minimal down time for the origin computer. Use a USB flash drive. Since the 260 has USB 2.0 it should be very quick.
Click to expand...

yes, zero down time - this transfer will be done many times.
Collect data on PC A, transfer to PC B. Collect more data, transfer. And so on. Only a few minutes total "off" time, have to get it all done in that time.

Right now, USB hard drive is what we are doing. Flash drives are too small, for 1 subject we get about 10 gigs.
However this is still slow - have to transfer from PC A to USB drive then from USB drive to PC B. That's 2x the data transfer time, plus even though it should technically be 480 Mb/sec, actual time seems much slower, minutes each way. Plus just a PITA.
 
Get a Cross over cat 5 cable and plug them directly in to each other.

then Share the folders you want to access on each pc

make sure you know the names of each PC and there admin login user name and password

Example PC1 and PC2 to access pc1 from pc2
Right click on the start menu and select "explore"

to access pc1 from pc2 type

\\pc2 (then hit enter)

then enter username and password if required



and I list of the shared folder will show up
then just drag and drop
 
I share an external HD between to computers....with a usb hub and transfer large files that way seems like an option...


I am tired but can go into more details later if interested dave
 
SkyHiK5 said:
I share an external HD between to computers....with a usb hub and transfer large files that way seems like an option...
Click to expand...


beat me to it. With an External HD, there is no need for a file transfer. its right there for use on any computer in as long as it takes to switch a usb cable.
 
SkyHiK5 said:
I share an external HD between to computers....with a usb hub and transfer large files that way seems like an option...


I am tired but can go into more details later if interested dave
Click to expand...
BigRedB2 said:
beat me to it. With an External HD, there is no need for a file transfer. its right there for use on any computer in as long as it takes to switch a usb cable.
Click to expand...

Do tell how you rigged this up, please.
Do you mean that both PCs can "share" (e.g. see) the same USB drive simultaneously, as though it is a file server? Like both PCs plugged into the same hub, which has the drive connected? I fail to see how that can be possible.

Simply moving the hard drive from one to the next will not solve the problem. In my situation, I have PC A "working" for 8 minutes collecting data. Then during a 2 minute break, data must go to PC B. PC B begins "working" on that same data, while PC A goes back to work collecting more data for another 8 minutes, and so on. Thus the 2 are operating at the same time.

I *think* you guys are overthinking my original question. My question is simply, what is the rate-limiting step to actually acheiving (close to) gigabit speeds of data transfer. Currently we also have them doing this via crossover cable, PC B has gigabit card but A does not - it has been ordered, so once it arives, what else might I need to replace to match the speeds, if anything.
 
Gigabit ethernet will do you faster than wireless or USB. Get two gigabit cards and make a category 5E patch cable to get the fastest trasfer speeds.
 
not sure what kind of numbers your computer is crunching, but rather than spending alot of money to make these two old pos' together, would it not be more efficient to just bite the bullet and get ONE computer with the processing power to handle the job?

i know, your dealing with grants and budgets and gov't BS, just throwing it out there though.
 
The two GIG network cards will help but if you only have IDE either 66,100 to 133 that will be your limiting factor here. With your existing hardware the fastest you will be transferring data if both if your computers have IDE at 133 would be 133 that is still faster then a 100MB NIC card so the GIG network card will help. But like was mentioned before you might end up saturating the bus since they are only probably 100 or 133 but could be as low as 66 in that old hardware.

If you really wanted to get the full speed out of the GIG network the drives on both end need to be SATA or SAS drives. Since you only have PCI connections inside the computer you will still be limiting. You need to have two computers that have SATA or SAS drives on both sides, or the need to have PCI Express slots so you can put SATA or SAS controller for the drives. The only way you are going to get those type of speeds is to have newer hardware on each end.

I work with a GIS system so I am very familiar with dealing with very large files that need to move places really fast. I deal with files like 100 to 500 gig in size and the users want them to show up instantly when they move around the map.

I hope that helps.
 
If USB will work, eSATA will be faster.

BUT...

What's your budget? With a pro budget in a real IT shop you'd get VMware or Citrix's Xen and run a SAN with fiberchannel. If I had only a couple grand, I'd get a big assed quad core box and run Ubuntu and KVM. I'd ghost the old box's image across put it on the quad core system and point it's storage drive to the local drive of that box. THEN I'd have the storage on the big box and already transferred.

OR... I'd back up the pos get a badassed server box and reconfigure the POS to use the badassed server as it's drive for storing these files in the first place. Once you store them over gigabit ethernet to a SATA RAID array with the proper RAID configuration for speed, then you should see an increase in performance over the original setup.

So, these are shots in the dark with little or no information.

J
 
RatLabGuy said:
Do tell how you rigged this up, please.
Do you mean that both PCs can "share" (e.g. see) the same USB drive simultaneously, as though it is a file server? Like both PCs plugged into the same hub, which has the drive connected? I fail to see how that can be possible.
Click to expand...


My set up...and I have no idea how it works but it does (I am an EE not a computer guy)

I have a desktop connected via dual usb to a usb hub that has an external also connected to the usb hub and laptop docking station that is alos dual usb'd to the usb hub.

I can not access the external simultaneously from both computers (actually I may be able to but I am scared of what may happen so have never tried) but I can load to it and almost instantly access it from the other.
 
hobie said:
If USB will work, eSATA will be faster.

BUT...

What's your budget? With a pro budget in a real IT shop you'd get VMware or Citrix's Xen and run a SAN with fiberchannel. If I had only a couple grand, I'd get a big assed quad core box and run Ubuntu and KVM. I'd ghost the old box's image across put it on the quad core system and point it's storage drive to the local drive of that box. THEN I'd have the storage on the big box and already transferred.

OR... I'd back up the pos get a badassed server box and reconfigure the POS to use the badassed server as it's drive for storing these files in the first place. Once you store them over gigabit ethernet to a SATA RAID array with the proper RAID configuration for speed, then you should see an increase in performance over the original setup.

So, these are shots in the dark with little or no information.

J
Click to expand...

Ok Jason now the english version

Get 2 bad ass computers and connect them with a bad ass network conection.

then you will be so Bad ass.....LOL

:gitrdun:
 
If you network the computers share the drive on the newer computer. Map it on the older computer and then just save it directly to the newer one so no data transfer is needed.
 
Ridgerunner said:
Ok Jason now the english version
Get 2 bad ass computers and connect them with a bad ass network conection.
then you will be so Bad ass.....LOL
:gitrdun:
Click to expand...


Actually, I was saying just get one bad ass... You can build a decent quad core server for under 1k if you work at it. The rest is configuration.

J
 
Back
Top