Re: Possible to bring in data stored on S3?
This archive was generated by
I agree with Nik, it seems reasonably fast in my experience. Note that
S3 doesn't have a file system, it's just an indexed object. So you can
write your object once and read it many times, but there isn't any
updating, file locking, etc. I typically use S3 for storage and
archiving or to provide a static data reference for things that aren't
going to change, but need to be widely available. I know those that use
S3 for static webpages, or for storing logs, etc. If your data is going
to change often, you might want to do things a different way.
On 10/6/14 9:19 PM, John Readey wrote:
> Is anyone pulling in data on demand using the S3 API's? I'm curious
> how the performance compares to using NFS.
> From: Jennifer Staab <jstaab_at_cs.unc.edu <mailto:jstaab_at_cs.unc.edu>>
> Date: Monday, October 6, 2014 at 6:08 PM
> To: greg <margeemail_at_gmail.com <mailto:margeemail_at_gmail.com>>,
> "starcluster_at_mit.edu <mailto:starcluster_at_mit.edu>"
> <starcluster_at_mit.edu <mailto:starcluster_at_mit.edu>>
> Subject: Re: [StarCluster] Possible to bring in data stored on S3?
> Besides the others mentioned, AWS has their own "command line
> interface" CLI see here
> for installation instructions and here
> <http://docs.aws.amazon.com/cli/latest/> for usage. AWS CLI
> provides tools for accessing S3. If you don't want to use AWS CLI,
> there are many SDKs available depending on programming language or
> platform see here <http://aws.amazon.com/tools/>.
> Good Luck.
> On 10/6/14 3:29 PM, greg wrote:
>> Hi all,
>> Is it possible to bring data into my cluster from Amazon S3? I didn't
>> see it in the manual.
>> Thanks for reading!
>> StarCluster mailing list
Received on Mon Oct 06 2014 - 21:49:58 EDT