Re: ERROR - Invalid AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY combination

From: Justin Riley <no email>
Date: Wed, 22 Dec 2010 10:12:22 -0500

Hi Stuart,

I was just about to point you here:

http://groups.google.com/group/boto-users/browse_thread/thread/c6700ddcfd55670e

and ask about your Python version :D

Glad you got it figured out and thanks for following up.

In the future, would you mind joining the mailing list and posting to
starcluster_at_mit.edu? This allows others to see your issues/questions and
will help folks that might run into similar issues.

Thanks!

~Justin

On 12/22/10 5:41 AM, Stuart Young wrote:
> Hi Justin,
>
> I figured out it was my version of Python (2.7) that handles unicode
> differently to python2.6 and was mangling my signatures so that I
> always got the 'SignatureDoesNotMatch' error. After I installed
> python2.6 it worked fine.
>
> starcluster -c /root/.starcluster/config listpublic
>
> >>> Listing all public StarCluster images...
>
> 32bit Images:
> -------------
> [0] ami-8cf913e5 us-east-1 starcluster-base-ubuntu-10.04-x86-rc3
> [1] ami-8f9e71e6 us-east-1 starcluster-base-ubuntu-9.04-x86
> [2] ami-17b15e7e us-east-1 starcluster-base-ubuntu-9.10-x86-rc7
> [3] ami-d1c42db8 us-east-1 starcluster-base-ubuntu-9.10-x86-rc8
>
> 64bit Images:
> --------------
> [0] ami-12b6477b us-east-1
> starcluster-base-centos-5.4-x86_64-ebs-hvm-gpu-rc2
> [1] ami-0af31963 us-east-1 starcluster-base-ubuntu-10.04-x86_64-rc1
> [2] ami-a19e71c8 us-east-1 starcluster-base-ubuntu-9.04-x86_64
> [3] ami-2941ad40 us-east-1 starcluster-base-ubuntu-9.10-x86_64-rc3
> [4] ami-a5c42dcc us-east-1 starcluster-base-ubuntu-9.10-x86_64-rc4
>
> total images: 9
>
>
> Thanks for your help!
>
> Stuart
>
>
>
>
> On 12/21/2010 3:46 AM, Stuart Young wrote:
>> Hi Justin,
>>
>> I'm using boto version 1.9b, as it was downloaded automatically when
>> I installed StarCluster 0.91.2.
>>
>> python -c 'import boto; print boto.Version'
>> 1.9b
>>
>> Btw, I looked into how boto 1.9b is sending out the request to
>> pinpoint exactly where the error occurs (Note the "reason:
>> 'Forbidden'" part of the response):
>>
>>
>> connection.make_request connection.make_request(self, action,
>> params, path, verb)
>> connection.get_signature connection.get_signature(self,
>> params, verb, path)
>> connection.calc_signature2 connection.calc_signature2(self,
>> params, verb, path)
>> connection.calc_signature2 Returning qs:
>> AWSAccessKeyId=AKIXXXXXXXXXXXXTHQ&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2010-12-21T08%3A27%3A08&Version=2009-11-30
>> and b64: QNqIbL9pTcv0RXXXXXXXXXXXXXXXXXXGsfZeb1afk=
>> connection.make_request Returning
>> AWSAuthConnection.make_request(...)
>> connection.AWSAuthConnection.make_request
>> connection.AWSAuthConnection.make_request(self, method, path,
>> headers, data, host, auth_path, sender)
>> connection.AWSAuthConnection.make_request Before return self._mexe
>> connection._mexe connection._mexe(method, path, data, headers,
>> host, sender)
>> Method: GET
>> Path:
>> /?AWSAccessKeyId=AKIXXXXXXXXXXXTHQ&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2010-12-21T08%3A27%3A08&Version=2009-11-30&Signature=QNqIbL9pTcvXXXXXXXXXXXXXXXXXXXGsfZeb1afk%3D
>> Data:
>> Headers: {'Date': 'Tue, 21 Dec 2010 08:27:08 GMT',
>> 'Content-Length': '0', 'Authorization': 'AWS
>> AKIAIXXXXXXXXXXTHQ:6lzoMU3zIhy9909bXDkKpB3MqVQ=', 'User-Agent':
>> 'Boto/1.9b (linux2)'}
>> Host: None
>> connection._mexe try 0
>> connection._mexe callable(sender) NOT DEFINED
>> connection._mexe response: <httplib.HTTPResponse instance at
>> 0x140f2d8>
>> connection._mexe BEFORE if response.status == 500
>> connection._mexe response.status < 300 or >=400
>> <HTTPResponse at 140f2d8>
>> _method: 'GET'
>> chunk_left: None
>> chunked: 1
>> debuglevel: 0
>> fp: <_fileobject at 0x1404050>: <socket._fileobject object at
>> 0x1404050>
>> length: None
>> msg: <HTTPMessage at 140f320: Transfer-Encoding: chunked
>> Date: Tue, 21 Dec 2010 08:27:08 GMT
>> Server: AmazonEC2
>> >
>> dict: <dictionary at 0x1496a60>: {'transfer-encoding':
>> 'chunked', 'date': 'Tue, 21 Dec 2010 08:27:08 GMT', 'server':
>> 'AmazonEC2'}
>> encodingheader: None
>> fp: None
>> headers: <list at 0x13f2368>: ['Transfer-Encoding:
>> chunked\r\n', 'Date: Tue, 21 Dec 2010 08:27:08 GMT\r\n', 'Server:
>> AmazonEC2\r\n']
>> maintype: 'text'
>> plist: <list at 0x13f23f8>: []
>> plisttext: ''
>> seekable: 0
>> startofbody: None
>> startofheaders: None
>> status: ''
>> subtype: 'plain'
>> type: 'text/plain'
>> typeheader: None
>> unixfrom: ''
>> reason: 'Forbidden'
>> status: 403
>> strict: 0
>> version: 11
>> will_close: <bool at 0x787990>: False
>> boto.exception.EC2ResponseError
>> boto.exception.EC2ResponseError(BotoServerError)
>> boto.exception.EC2ResponseError BotoServerError: <class
>> 'boto.exception.BotoServerError'>
>> <class 'boto.exception.BotoServerError'>
>> awsutils.is_valid_conn boto.exception.EC2REsponseError RAISED!!!
>> awsutils.is_valid_conn e: EC2ResponseError: 403 Forbidden
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Response><Errors><Error><Code>SignatureDoesNotMatch</Code><Message>The
>> request signature we calculated does not match the signature you
>> provided. Check your AWS Secret Access Key and signing method.
>> Consult the service documentation for
>> details.</Message></Error></Errors><RequestID>cdb73666-37a6-4dfc-9f66-2add7b04a0bf</RequestID></Response>
>> awsutils.is_valid_conn DUMP boto.exception.EC2ResponseError:
>> <class 'boto.exception.EC2ResponseError'>
>>
>> cluster._validate_credentials self.ec2.is_valid_conn NOT TRUE
>> cluster.is_valid ERROR
>> cluster.py:782 - ERROR - Invalid
>> AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY combination.
>> cli.py:243 - ERROR - settings for cluster template "smallcluster"
>> are not valid
>>
>>
>> I emailed Amazon to ask them to check the Request ID. Looking at the
>> request, is there anything that jumps out at you?
>>
>> Cheers,
>>
>> Stuart
>>
>>
>>
>> On 12/20/2010 10:05 PM, Justin Riley wrote:
>>> What version of boto are you using? Also You can determine this using:
>>>
>>> $ python -c 'import boto; print boto.Version'
>>>
>>> Also this is using StarCluster 0.91.2 correct?
>>>
>>> ~Justin
>>>
>>> On 12/20/10 8:36 PM, Stuart Young wrote:
>>>> Hi Justin,
>>>>
>>>> ElasticFox works fine with all three sets of credentials. I'm still
>>>> looking into boto's error. Will update you if I find anything.
>>>>
>>>> Cheers,
>>>>
>>>> Stuart
>>>>
>>>>
>>>> On 12/20/2010 4:21 PM, Stuart Young wrote:
>>>>> Hi Justin,
>>>>>
>>>>> Thanks for the debugging methods information - I actually got it
>>>>> working by trial and error using a permuation of method 2 but
>>>>> method 1 is so much more straightforward! (As you'll guess, I'm a
>>>>> noob to python.) I tracked the error down to boto's connect.py and
>>>>> ec2/connect.py modules:
>>>>>
>>>>> awsutils.conn self.conn()
>>>>> awsutils.conn self.aws_access_key:
>>>>> **AAAAA<no-weird-symbols>HHHHQ**
>>>>> awsutils.conn self.aws_secret_access_key:
>>>>> **4+0Ma<no-weird-symbols-except'+'>DrschU6**
>>>>> awsutils.py:57 - DEBUG - creating self._conn w/
>>>>> connection_authenticator kwargs = {'path': '/', 'region':
>>>>> None, 'port': None, 'is_secure': True}
>>>>> awsutils.conn Doing self._conn =
>>>>> self.connection_authenticator()
>>>>> awsutils.conn self.connection_authenticator: <function
>>>>> connect_ec2 at 0x9c5cf8>
>>>>> boto.__init__.connect_ec2
>>>>> boto.__init__.connect_ec2(aws_access_key_id,
>>>>> aws_secret_access_key, kwargs)
>>>>> boto.__init__.connect_ec2 aws_access_key_id:
>>>>> AAAAA<no-weird-symbols>HHHHQ
>>>>> boto.__init__.connect_ec2
>>>>> aws_secret_boto.__init__.connect_ec2access_key:
>>>>> 4+0Ma<no-weird-symbols-except'+'>DrschU6
>>>>> boto.connection.AWSQueryConnection.__init__
>>>>> boto.connection.AWSQueryConnection.__init__(self, host,
>>>>> aws_access_key_id, aws_secret_access_key, etc)
>>>>> boto.connection.AWSQueryConnection.__init__
>>>>> aws_access_key_id: AAAAA<no-weird-symbols>HHHHQ
>>>>> boto.connection.AWSQueryConnection.__init__
>>>>> aws_secret_access_key: 4+0Ma<no-weird-symbols-except'+'>DrschU6
>>>>> awsutils.conn Returning...
>>>>> awsutils.is_valid_conn boto.exception.EC2REsponseError
>>>>> RAISED!!!
>>>>> awsutils.is_valid_conn e: EC2ResponseError: 403 Forbidden
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <Response><Errors><Error><Code>SignatureDoesNotMatch</Code><Message>The
>>>>> request signature we calculated does not match the signature
>>>>> you provided. Check your AWS Secret Access Key and signing
>>>>> method. Consult the service documentation for
>>>>> details.</Message></Error></Errors><RequestID>deb3819a-d8d1-457a-a432-b6f35675be14</RequestID></Response>
>>>>> cluster._validate_credentials ERROR OCCURRED in
>>>>> self.ec2.is_valid_conn
>>>>> cluster.py:770 - ERROR - Invalid
>>>>> AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY combination.
>>>>> cli.py:253 - ERROR - settings for cluster template
>>>>> "smallcluster" are not valid
>>>>>
>>>>>
>>>>> I was able to connect and launch instances with my existing
>>>>> AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY via the perl module
>>>>> Net::Amazon::EC2 so I'm perplexed as to why I'm getting the error
>>>>> using boto. I will try elasticfox as you suggested and update you
>>>>> later.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Stuart.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 12/20/2010 4:04 PM, Justin Riley wrote:
>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>> Hash: SHA1
>>>>>>
>>>>>> On 12/20/2010 11:19 AM, Stuart Young wrote:
>>>>>>> cli.py:1083 - ERROR - SignatureDoesNotMatch: The request signature
>>>>>>> we calculated does not match the signature you provided. Check your
>>>>>>> AWS Secret Access Key and signing method. Consult the service
>>>>>>> documentation for details.
>>>>>> Hmmm this is an error reported directly from Amazon; this shouldn't be
>>>>>> caused
>>>>>> by boto and definitely not paramiko. I can reproduce this message if I
>>>>>> remove a character from my AWS_SECRET_ACCESS_KEY so it seems either
>>>>>> something's wrong with the keys in your config or wrong on Amazon's side.
>>>>>> In the past when I've seen this message it's always due to using quotes
>>>>>> or a stray
>>>>>> character some where...
>>>>>>
>>>>>> Are these recently created accounts?
>>>>>>
>>>>>>> I got the same error when I tried with two other AWS accounts. Is this
>>>>>>> maybe something to do with boto or paramiko?
>>>>>> Would you mind installing elasticfox (firefox plugin) and see if it has
>>>>>> issues with your access keys as well?
>>>>>>
>>>>>>> I was also wondering if it's possible to to run a local, unzipped copy of the StarCluster egg for
>>>>>>> debugging purposes?
>>>>>> Yes it is and there are two ways:
>>>>>>
>>>>>> 1. First uninstall starcluster (remove the egg from your site-packages).
>>>>>> Download the 0.91.2 release, unpack it somewhere, and run python
>>>>>> setup.py develop. This will 'link' the unzipped source to your python
>>>>>> install. This means if you make changes to the source code, you will be
>>>>>> able to import and see/use those changes without having to reinstall.
>>>>>> This is currently how I'm developing the code.
>>>>>>
>>>>>> OR
>>>>>>
>>>>>> 2. Simply unzip the egg in your Python's site-packages directory and
>>>>>> edit the source within your site-packages directory. You'll likely want
>>>>>> to move the egg outside of your site-packages directory after you unzip
>>>>>> it to make sure you import the unzipped code and not the egg.
>>>>>>
>>>>>> Does that make sense?
>>>>>>
>>>>>> ~Justin
>>>>>> -----BEGIN PGP SIGNATURE-----
>>>>>> Version: GnuPG v2.0.16 (GNU/Linux)
>>>>>> Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/
>>>>>>
>>>>>> iEYEARECAAYFAk0PxMwACgkQ4llAkMfDcrkylwCfQpHxxNaZsbXX3xBNe9hbRfgE
>>>>>> QAkAnR2YlcUQTH4cQTEg4M8V8Cg8MQIc
>>>>>> =1Rcm
>>>>>> -----END PGP SIGNATURE-----
>>>>>>
>>>>>
>>>>
>>>
>>
>
Received on Wed Dec 22 2010 - 10:12:05 EST