2980 Commits

Author SHA1 Message Date
Andrew Garrett
6f4cb512ac Allow CloudFormation stack tags to be updated
Limitations:

* does not update the tags of the resources in the stack. that can be
implemented later.
* does not support the supposed feature of clearing tags by passing an
empty value that boto3 mentions in its documentation. I could not find
anything in the request body to indicate when an empty value was passed.
2017-03-17 23:57:57 +00:00
Steve Pulec
a6e762340d Merge pull request #867 from 2rs2ts/flexible-route53-hostedzoneid-lookup
Be flexible with Route53 Hosted Zone IDs with /hostedzone/ prefix
2017-03-16 22:55:13 -04:00
Andrew Garrett
f2b7ba03b4 Forgot that lstrip works on character sets, not substrings
I suppose this is one way to do it. I could have also split and taken
the last element. Not sure which is best. 🤔
2017-03-17 02:45:58 +00:00
Steve Pulec
c207963a86 Cleanup SNS exceptions. Closes #751. 2017-03-16 22:28:30 -04:00
Steve Pulec
e7a3f3408e Add Lambda header for invoking error. Closes #770. 2017-03-16 22:00:57 -04:00
Steve Pulec
e3bff8b926 Fix cloudformation NoValue parsing to not add attribute. Closes #870 2017-03-16 21:20:01 -04:00
michael_lerch
83084bf2af Prevent 100% cpu usage while SQS long polling on an empty queue
While using moto server with a test SQS client, I noticed significant CPU usage while the client was long polling. I narrowed this down to the `receive_messages` call of the SQS service sitting in a `while True:` statement with no work to be done, thus looping forever.

To produce this issue, I do:
```
$ python3 -m venv venv
$ . ./venv/bin/activate
(venv) $ pip install moto moto[server] boto3
Collecting moto
  Downloading moto-0.4.31-py2.py3-none-any.whl (303kB)
  --snip--
(venv) $ moto_server sqs &
[1] 31727
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
(venv) $ python3
Python 3.6.0 (default, Dec 24 2016, 08:01:42)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto3
>>> client = boto3.client('sqs', region_name='us-east-1', endpoint_url='http://127.0.0.1:5000');
>>> client.create_queue(QueueName='testing')
127.0.0.1 - - [16/Mar/2017 13:34:20] "POST / HTTP/1.1" 200 -
{'QueueUrl': 'http://sqs.us-east-1.amazonaws.com/123456789012/testing', 'ResponseMetadata': {'RequestId': '7a62c49f-347e-4fc4-9331-6e8e7a96aa73', 'HTTPStatusCode': 200, 'HTTPHeaders': {'content-type': 'text/html; charset=utf-8', 'content-length': '343', 'server': 'Werkzeug/0.12.1 Python/3.6.0', 'date': 'Thu, 16 Mar 2017 20:34:20 GMT'}, 'RetryAttempts': 0}}
>>> client.receive_message(QueueUrl='http://sqs.us-east-1.amazonaws.com/123456789012/testing', MaxNumberOfMessages=10, WaitTimeSeconds=10)
```

At this point the moto server will run at 100% cpu for 10 seconds until the request times out waiting for a message. If multiple clients are continuously reconnected (as in mocking a normal sqs worker setup) the server will sit at 100% cpu indefinitely.

This pull request adds a simple sleep statement to the SQS `receive_messages` call when there are no messages to process. In doing so, the loop will be limited to executing once per 0.001 seconds when the queue is empty. The CPU usage is nearly 0% after this change.
2017-03-16 13:43:45 -07:00
Steve Pulec
a5da348fba Fix lint. 2017-03-15 23:43:48 -04:00
Steve Pulec
25e2af0320 Fix camelcase_to_underscore. Closes #767. 2017-03-15 23:39:36 -04:00
Steve Pulec
446843e756 Fix py3 s3 prefix decoding. 2017-03-15 23:13:09 -04:00
Steve Pulec
e25d1499c2 Update cloudformation for new list types. 2017-03-15 23:02:10 -04:00
Steve Pulec
3899eee648 Fix S3 filtering by unicode prefix. Closes #838 2017-03-15 22:53:27 -04:00
Steve Pulec
8a803cdbaf Better EC2 duplicate SG error. 2017-03-15 22:45:28 -04:00
Steve Pulec
5f3fbff627 Standardize on one account id (123456789012). 2017-03-15 22:21:04 -04:00
Steve Pulec
e9e49c5916 Fix py3 redshift encoding. 2017-03-15 22:12:16 -04:00
Steve Pulec
3cdb4afad0 Fix redshift responses to work with json or xml. 2017-03-15 21:58:37 -04:00
Steve Pulec
09a4d177f5 Add kms boto3 test. 2017-03-14 23:42:52 -04:00
Steve Pulec
0df47574ab Merge pull request #865 from adamstauffer/master
update RDS responses to return DBInstanceArn
2017-03-14 23:28:30 -04:00
Steve Pulec
2bd4567801 Do not use flask outside of server mode. 2017-03-14 23:26:31 -04:00
Steve Pulec
7eb5b60620 Merge pull request #860 from 2rs2ts/add-list-hosted-zones-by-name
Add ListHostedZonesByName
2017-03-14 23:24:03 -04:00
Andrew Garrett
6346e44c9d Be flexible with Route53 Hosted Zone IDs with /hostedzone/ prefix
We will continue to store just the unique ID, but since the AWS API
returns /hostedzone/<id>, we should accept attempts to pass that back.
For example, both just the ID as well as /hostedzone/<id> work for
specifying the HostedZoneId of a ResourceRecordSet in CloudFormation. So
we should support that too.

Signed-off-by: Scott Greene <scott.greene@getbraintree.com>
2017-03-14 19:52:36 +00:00
Adam Stauffer
c5853b48da update RDS responses to return DBInstanceArn 2017-03-13 16:48:22 -04:00
Andrew Garrett
b9ea947aa0 Add ListHostedZonesByName 2017-03-13 15:27:45 +00:00
Seamus Cawley
bd2ff89bf1 Ensure SQS property WaitTimeSeconds is an integer 2017-03-13 13:56:55 +00:00
Steve Pulec
cda553abfb Change tests to use default server port of 5000. 2017-03-12 21:04:19 -04:00
Steve Pulec
d2c56619cd Add lint to Travis. 2017-03-12 20:35:45 -04:00
Steve Pulec
5807a38092 Merge pull request #858 from spulec/dashboard
Add a dashboard
2017-03-12 20:26:42 -04:00
Steve Pulec
0e2fdf94f9 Cleanup lints. 2017-03-12 20:18:54 -04:00
Steve Pulec
3a01150154 Add about page. 2017-03-12 19:58:40 -04:00
Steve Pulec
24affc7a5f Fix py3 Model __new__. 2017-03-12 13:04:36 -04:00
Steve Pulec
4d5642ae33 Cleanup dashboard. 2017-03-12 12:59:30 -04:00
Christian Hellman
c271e31745 Fixed terraform deploy against moto fails (#857) 2017-03-12 12:40:49 -04:00
Steve Pulec
09ac3539b7 Sort dashboard attributes. 2017-03-12 12:34:54 -04:00
Steve Pulec
6d422d1f37 Add BaseModel to all models. 2017-03-11 23:41:12 -05:00
Steve Pulec
caea5f441d Fix resetting backends. 2017-03-11 23:18:58 -05:00
Steve Pulec
abe0460dc7 Merge branch 'master' into dashboard 2017-03-11 22:45:45 -05:00
Steve Pulec
1709208872 First version of dashboard. 2017-03-11 22:45:42 -05:00
Steve Pulec
cdd6e476cc If using newer dynamodb api, use version 2. 2017-03-11 20:50:24 -05:00
Steve Pulec
3b4ef2cf15 Merge in master. 2017-03-05 09:58:39 -05:00
Matt Chamberlin
0393c384ad fix etag metadata field name in key response dict (etag --> ETag) (#855) 2017-03-04 23:17:18 -05:00
Andrew Garrett
e7735c3ee1 Add event IDs to CF Stack events (#852)
So that events can be uniquely identified.

I tried to match the format documented here:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-listing-event-history.html
2017-03-04 23:12:55 -05:00
Andrew Garrett
e7ea6b350c Fix lambda stdout/stderr mocking (#851)
Originally, the code was setting sys.stdout and sys.stderr back to the
original, official forms, but this breaks idioms like mocking stdout to
capture printing output for tests. So instead, we will reset sys.stdout
and sys.stderr to what they were before running the lambda function, so
that in case someone is mocking stdout or stderr, their tests won't
break.
2017-03-04 23:01:50 -05:00
Andrew Garrett
f6465df630 Return CF Stack events in reverse chronological order (#853)
This is how the AWS API works:
http://boto3.readthedocs.io/en/latest/reference/services/cloudformation.html#CloudFormation.Client.describe_stack_events
2017-03-04 23:00:25 -05:00
Léo Cavaillé
9b6d3983d2 iam: add group policy methods (#849)
Implemented mocks for:
* get_all_group_policies
* list_group_policies (boto3)
* get_group_policy
* put_group_policy
2017-03-04 22:56:36 -05:00
Chris LaRose
56f9409ca9 Use request URL to generate SQS queue URLs; fixes #626 (#827) 2017-03-04 22:53:14 -05:00
William Richard
f46a24180f Cast desired capacity for cloudformation asg to int (#846)
Cloudformation passes MaxSize, MinSize and DesiredCapacity as strings, but
we want to store them as ints.  Also includes tests of this fix, to help
avoid regression.
2017-03-04 22:51:01 -05:00
David Wilcox
a9554924df make cloudformation update stack use parameters provided (#843) 2017-03-04 22:48:51 -05:00
Andy Freeland
783242b696 Elastic IP PhysicalResourceId should always be its public IP (#841)
According to the [CloudFormation `Ref` docs][docs], the `Ref` return
value (and physical ID of the resource) for an Elastic IP is its public
IP address.

[docs]: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html
2017-03-04 22:40:43 -05:00
Andy Freeland
a30ba2b597 EC2 tags specified in CloudFormation should be applied to the instances (#840)
Fixes #839.
2017-03-04 22:37:53 -05:00
David Wilcox
1b6007e2b2 Correct IAM list_server_certs template that was based off incorrect docs (#836)
The documentation for this method is here https://docs.aws.amazon.com/IAM/latest/APIReference/API_ListServerCertificates.html

The docs say the return type is this  ServerCertificateMetadataList.member.N

but the sample response incorrectly include a <ServerCertificateMetadata>. I've sent feedback to the AWS docs telling them to fix their stuff but this also needs to be fixed. I haven't checked other templates with <member> tags in them, as they may be prone to this same problem.
2017-03-04 22:36:25 -05:00