Commit Graph

3293 Commits

Author SHA1 Message Date
D. Ferruzzi
61e2204941
Initial EKS Implementaion (#3981)
* Implemented EKS list_clusters

* Implemented EKS create_cluster

* Implemented EKS describe_cluster

* Implemented EKS delete_cluster

* Implemented EKS list_nodegroups

* Implemented EKS create_nodegroup

* Implemented EKS describe_nodegroup

* Implemented EKS delete_nodegroup

* Implemented EKS Server Tests

* EKS - rework tests to use decorator everywhere

Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-06-20 11:34:31 +01:00
Cristopher Pinzón
e4057a34b9
Added "isTruncated" attribute for listUsersResponse (#4017)
* isTruncated attribute for listUsersResponse

* correction and test
2021-06-18 21:52:15 +01:00
Brian Pandola
a95ca81e76
Support EMR clusters with multiple master nodes (#4019)
Reference: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha-launch.html
2021-06-18 06:04:21 +01:00
Bert Blommers
2f23807a35
#3760 - DynamoDB - only raise exception when GSI key is actually empty (#3763) 2021-06-17 14:07:24 +01:00
Brian Pandola
b0e2a750dc
Disallow termination of protected EMR job flows (#4015)
Error message verified against real AWS backend.
2021-06-17 12:20:45 +01:00
Thomas Ross
67ae84e2c4
Only raise EmptyKeyAttributeException when the value node is empty (#4014)
This fixes a bug where all updates to keys (including GSIs) would raise
EmptyKeyAttributeException, even when the new value was not an empty string.
2021-06-17 12:19:47 +01:00
Jordi Alhambra
407d5c853d
Media package client error additional handling (#4011)
* Add delete container and list tags endpoints to MediaStore

* Black reformat

* Fixed Lint problems

* Check if dictionary was deleted effectively

* lint fix

* MediaPackageClientError

* Lint Fix

* Test unknown channel describe

* Concatenation Fix

* MediaPackage - fix error message

* MediaPackage Test Fix and TryError

* Lint

Co-authored-by: av <arcovoltaico@gmail.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-06-16 21:07:46 +01:00
Bert Blommers
cbeeefbec9
Prep release 2.0.9 (#4007)
* Update implementation coverage

* EC2 - Update instance type offerings

* IAM - update list of managed policies

* Changelog for release 2.0.9

* Instance Type Offerings - fix number of available offerings
2021-06-12 13:57:29 +01:00
Sahil Shah
6977bba3e1
Add capability to update AWS::ElasticLoadBalancingV2(Listener and ListenerRule) resource (#4005)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

* Added support for ListenerRule. Will remove cruft

* Pushing latest

* Something works

* Put back ripped out code

* Save point. Incase I need more validations

* Revert "Save point. Incase I need more validations"

This reverts commit dac4953335dd9335eddb7a91a63667bc3c17104c.

* Fixed validations and some refactor

* Fix formatting

* Linting

* Cannot refactor if I have to fix all tests

* Remove exceptions for now. Will do in another PR

* Remove validations. Will add in next PR

* Fix broken tests. Almost.:

* Fix all tests. Some sneaky for now.

* Python2 making me write bad code

* OrderedDict.move_to_end() does not work in python2

* Linting

* Add more checks to field in conditions later.

* Unwnated change in FakeListener

* Revert "Unwnated change in FakeListener"

This reverts commit 962c2fdfd76fce999de9feccf1dd1c3ec48c459f.

* Add back default listener rule

* Linting fix

* Fix priority sorting

* Add cloudformation test for edge case

* Add validation for ForwardConfig in Action of ListernRule CF

* use not in

* set the priority template correctly

* Check for boolean in condition

* One more check

* Implement update_from_cloudformation_json for Listener and ListenerRule

* Unwanted spaces

* Linting issues

* Add tests for code coverage

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-06-11 21:56:28 +01:00
Kyle Decot
8d4007f2b6
Escape Hyphen in Regular Expression (#4006)
* Escape Hyphen in Regular Expression

* Add Test

* Fix Linting Errors

* Use Pytest Parameterize Marker

* Fix Black Formatting Errors

* Interpolate / Use Raw String

* fixes string formatting

Co-authored-by: Correna Sprowls <correna.sprowls@joinroot.com>
2021-06-11 21:54:43 +01:00
benediktbrandt
479ce861a7
Env variable for default key buffer size (#4001)
* - introduce environment variable for DEFAULT_KEY_BUFFER_SIZE

* - prefix env variable with MOTO_S3 to avoid env variable conflicts

* - black formatting

* - fix formatting
2021-06-10 10:48:28 +01:00
Brian Pandola
c1b38be02d
Add support for DynamoDB Backup/Restore (#3995)
* Add support for DynamoDB Backup/Restore

Basic support for the following endpoints has been implemented with full test coverage:
- create_backup
- delete_backup
- describe_backup
- list_backups
- restore_table_from_backup

Behavior and error messages verified against a real AWS backend.

* Refactor test based on PR feedback
2021-06-10 07:05:07 +01:00
Sahil Shah
3f5408c9d0
Adding support for ForwardConfig property in ListernRule in CloudFormation (#3993)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

* Added support for ListenerRule. Will remove cruft

* Pushing latest

* Something works

* Put back ripped out code

* Save point. Incase I need more validations

* Revert "Save point. Incase I need more validations"

This reverts commit dac4953335dd9335eddb7a91a63667bc3c17104c.

* Fixed validations and some refactor

* Fix formatting

* Linting

* Cannot refactor if I have to fix all tests

* Remove exceptions for now. Will do in another PR

* Remove validations. Will add in next PR

* Fix broken tests. Almost.:

* Fix all tests. Some sneaky for now.

* Python2 making me write bad code

* OrderedDict.move_to_end() does not work in python2

* Linting

* Add more checks to field in conditions later.

* Unwnated change in FakeListener

* Revert "Unwnated change in FakeListener"

This reverts commit 962c2fdfd76fce999de9feccf1dd1c3ec48c459f.

* Add back default listener rule

* Linting fix

* Fix priority sorting

* Add cloudformation test for edge case

* Add validation for ForwardConfig in Action of ListernRule CF

* use not in

* set the priority template correctly

* Check for boolean in condition

* One more check

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-06-09 18:41:18 +01:00
Bert Blommers
43426c71f4
ELBv2 - Simplify tests (#3999) 2021-06-09 16:34:53 +01:00
Jordi Alhambra
ae5653b31d
Media package client error (#3983)
* Add delete container and list tags endpoints to MediaStore

* Black reformat

* Fixed Lint problems

* Check if dictionary was deleted effectively

* lint fix

* MediaPackageClientError

* Lint Fix

* Test unknown channel describe

* Concatenation Fix

* MediaPackage - fix error message

Co-authored-by: av <arcovoltaico@gmail.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-06-07 13:47:49 +01:00
usmangani1
9836985473
Fix : Logs - Add start_query functionality (#3985)
* Added start_query functionality

* fix tests
2021-06-07 10:37:11 +01:00
usmangani1
d325593e46
Fix:Add list connection API's (#3976)
* Initial commit create connection

* Added connection API's

* Add destination API's

* fixed tests

* fix tests

* Fixed tests
2021-06-07 10:35:28 +01:00
stromp
9e4972b43f
Add Tag support for other resource types then an account (#3982)
* - Adding checking for resource type in tag functions
- Adding TargetNotFoundException when no resource found
- Adding support for tags for root OU, OU and Policies
- Adding tests covering the new code
- Adding test for deletion of a tag

* fixed linting issue

* - renamed helper function to a more logical name
- added tests for helper function
- fixed bugs in tests for tag functions

Co-authored-by: Sjoerd Tromp <stromp@schubergphilis.com>
2021-06-05 15:12:17 +01:00
Sahil Shah
b82096ba37
Add support for AWS::ElasticLoadBalancingV2::ListenerRule (#3969)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

* Added support for ListenerRule. Will remove cruft

* Pushing latest

* Something works

* Put back ripped out code

* Save point. Incase I need more validations

* Revert "Save point. Incase I need more validations"

This reverts commit dac4953335dd9335eddb7a91a63667bc3c17104c.

* Fixed validations and some refactor

* Fix formatting

* Linting

* Cannot refactor if I have to fix all tests

* Remove exceptions for now. Will do in another PR

* Remove validations. Will add in next PR

* Fix broken tests. Almost.:

* Fix all tests. Some sneaky for now.

* Python2 making me write bad code

* OrderedDict.move_to_end() does not work in python2

* Linting

* Add more checks to field in conditions later.

* Unwnated change in FakeListener

* Revert "Unwnated change in FakeListener"

This reverts commit 962c2fdfd76fce999de9feccf1dd1c3ec48c459f.

* Add back default listener rule

* Linting fix

* Fix priority sorting

* Add cloudformation test for edge case

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-06-05 10:04:04 +01:00
Mike Grima
ba7a0fe3a2 Fixed AWS Config S3 ACL issues
- AWS Config appears to have changed how ACLs are returned
- This PR makes moto's behavior consistent with AWS Config's S3 representation

Fixes #3986
2021-06-04 09:23:56 -07:00
Austin Hendrix
5602c4e73e
SSM Parameter Store improvements in GetParameterHistory & GetParameters (#3984)
* Including labels and versions in SSM Get Parameters

* implementing NextToken and MaxResults into the SSM Get Parameter History functionality

* Implementing unit tests and some lint refactoring for NextToken implementation in get_parameter_history
2021-06-04 11:12:35 +01:00
Austin Hendrix
b670962c5e
SSM Parameter Store Error Message When Requesting Invalid Version (#3977)
* Implement correct error when requesting specific version of a parameter which exists but does not have this version

* removing trailing whitespace causing lint failure

* Adding unit tests and fixing linting for new error handling

* Fixing small bug in response message

* Revert change in get_parameters as versioning is not currently implemented in this method. Will fix as a separate PR
2021-06-02 16:27:23 +01:00
Brian Pandola
7947b1e001
Enforce SSM Parameter Version Limit (#3967)
Behavior verified against a real AWS backend.
2021-05-28 08:32:39 +01:00
Brian Pandola
3265e89fd3
Allow Glacier VaultName to contain period (#3964) 2021-05-27 08:44:20 +01:00
Florian Dambrine
6adb570c48
Add dot as a valid s3 metadata key (#3958) 2021-05-26 14:27:25 +01:00
Thomas Maschler
d635c78bd1
AWS Batch enhancements (#3956)
* Check exit status of container

* Added support for job dependencies

* batch container overrides

* add AWS_BATCH_JOB_ID to container env variables

* lint with black

* refactor batch dependency test

* refactor batch dependency test

* fix index

Co-authored-by: jterry64 <justin.terry@wri.org>
Co-authored-by: Daniel Mannarino <daniel.mannarino@gmail.com>
2021-05-26 08:52:09 +01:00
rajinder
fbbc8fc472
basic implementation of update rest api (#3951)
* basic implementation of update rest api

* basic implementation of update rest api

* basic implementation of update rest api

* review comments from bblommers

Co-authored-by: rajinder saini <rajinder.saini@c02vt5k2htd6.corp.climate.com>
2021-05-23 17:09:02 +01:00
Brian Pandola
16f4dc87dc
Add support for tagging Volumes in ec2:RunInstances (#3946) 2021-05-21 06:35:34 +01:00
Jordi Alhambra
7f49cd0ed6
MediaStore delete container and list tags endpoints implemented (#3938)
* Add delete container and list tags endpoints to MediaStore

* Black reformat

* Fixed Lint problems

* Check if dictionary was deleted effectively

* lint fix

Co-authored-by: av <arcovoltaico@gmail.com>
2021-05-20 16:17:31 +01:00
Brian Pandola
752b508738
Add support for KmsKeyId in ec2:RunInstances (#3943) 2021-05-20 07:11:50 +01:00
rajinder
58f7ab0d25
Refector Apigateway update_usage_plan and add all replace operation. (#3942)
* Refector Apigateway update_usage_plan and add all replace operation.

* code reformatting

Co-authored-by: rajinder saini <rajinder.saini@c02vt5k2htd6.corp.climate.com>
2021-05-20 07:10:08 +01:00
Sahil Shah
24fa8f25a1
SSM parameters in cloudformation (AWS::SSM::Parameter::) are not recognized and resolved (#3929)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-05-19 08:30:25 +01:00
usmangani1
4ad6c3932e
Fix resource group get-group-query & update-query-query (#3934)
* Fix resource group get-group-query & update-query-query

* Remove list logic

* Added tests

* Fix tests

* Fix update query tests

* fix tests
2021-05-18 20:58:21 +01:00
Giovanni Torres
867cf078de
feat: transition elbv2 state to active on first describe (#3937)
* refactor: parameterize state in elbv2 template response

* feat: transition load balancer to active on first describe

Closes #3936
2021-05-18 07:52:39 +01:00
usmangani1
31cf3c4252
Fix:SES-Added params check in template (#3753)
* Fix:SES-Added params check in template

* Added more tests and handled message

* linting

* fixed tests

* fix attribute name in message

* fix logic for exception
2021-05-18 07:51:27 +01:00
usmangani1
abc62b360d
Fix- Raise InvalidParameterValue when change visibility timeout is mo… (#3893)
* Fix- Raise InvalidParameterValue when change visibility timeout is more than permitted

* Fix review comments

* Modified tests

* Fix tests

* change

* Change client

* Fix tests

* fix tests

* fix tests

* change tests

* Fix tests

* changed tests

* Fix tests

* Fix tests

* Change message
2021-05-18 07:49:56 +01:00
usmangani1
03b2009a1c
Add unit support to cloudwatch get_metric_statistics (#3932)
* Add unit support to cloudwatch get_metric_statistics

* fix linting
2021-05-15 08:05:41 +01:00
usmangani1
58fd4aeaae
Fix tags for resource in ListTagsForResource (#3931)
* Fix tags for resource in ListTagsForResource

* Fix review comments

* Fix tests
2021-05-14 20:05:40 +01:00
usmangani1
eaac32b130
ResourceGroup - Add Support for getGroupConfiguration (#3919)
* ResourceGroup - Add Support for getGroupConfiguration

* Add tests

* Add tests

* Fix tests

* linting

* fix tests

* fix tests

* fix tests

* fixed linting

* fix tests
2021-05-14 07:40:17 +01:00
Kai
f9e0595e12
Fix sqs message retention logic (#3924)
* Fix sqs message retention logic

* Apply lint to moto/sqs/models.py

* Fix failed tests because of freezing time

* Fix freezing time in test_publish_to_sqs_in_different_region
2021-05-13 15:06:54 +01:00
Bert Blommers
9e3faf7784
Make Moto (tests) compatible with flask/werkzeug 2.x (#3923)
* Dont fail if CodeCov fails - for now

* CI - Force cache rebuild

* Bump werkzeug to latest version

* CI - Enforce cache flush

* ManagedBlockchain - fix error format

* ManagedBlockchain - Fix tests to use pytest.raises paradigm

* Revert "Lock Flask (#3925)"

This reverts commit 8bb0feb956.

* CI - Enforce cache rebuild
2021-05-13 10:36:56 +01:00
Daniel Samuels
a4b1498665
Support rotating secrets using Lambda [#3905] (#3912)
* Support rotating secrets using Lambda

The Secrets manager rotation process uses an AWS Lambda function
to perform the rotation of a secret. [1]

In fact, it's not possible to trigger rotation of a Secret
without specifying a Lambda function at some point in the life
of the secret:

```
$ aws secretsmanager rotate-secret --secret-id /rotationTest

An error occurred (InvalidRequestException) when calling the RotateSecret operation: No Lambda rotation function ARN is associated with this secret.
```

`moto` can be a little more lenient in this regard and allow
`rotate_secret` to be called without a Lambda function being
present, if only to allow simulation of the `AWSCURRENT` and
`AWSPREVIOUS` labels moving across versions.

However, if a lambda function _has_ been specified when calling
`rotate_secret`, it should be invoked therefore providing the
developer with the full multi-stage process [3] which can be
used to test the Lambda function itself and ensuring that full
end-to-end testing is performed. Without this there's no easy
way to configure the Secret in the state needed to provide the
Lambda function with the data in the format it needs to be in
at each step of the invocation process.

[1]: https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-lambda-function-overview.html
[2]: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.rotate_secret
[3]: https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-lambda-function-overview.html#rotation-explanation-of-steps

* Run `black` over `secretsmanager/models.py`

* Make `lambda_backends` import local to the condition

* Implement `update_secret_version_stage`

Allow a staging label to be moved across versions.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.update_secret_version_stage

* Add an integration test for Secrets Manager & Lambda

* Support passing `ClientRequestToken` to `put_secret_value`

By passing `ClientRequestToken` to `put_secret_value` within
the lambda function  invoked by calling `rotate_secret`, one
can update the value associated with the existing (pending)
version, without causing a new secret version to be created.

* Add application logic for `AWSPENDING`

The rotation function must end with the versions of the secret
in one of two states:

 - The `AWSPENDING` and `AWSCURRENT` staging labels are
   attached to the same version of the secret, or
 - The `AWSPENDING` staging label is not attached to any
   version of the secret.

If the `AWSPENDING` staging label is present but not attached
to the same version as `AWSCURRENT` then any later invocation
of RotateSecret assumes that a previous rotation request is
still in progress and returns an error.

* Update `default_version_id` after Lambda rotation concludes

Call `set_default_version_id` directly, rather than going 
through `reset_default_version` as the Lambda function is 
responsible for moving the version labels around, not `rotate_secret`.

* Run `black` over changed files

* Fix Python 2.7 compatibility

* Add additional test coverage for Secrets Manager

* Fix bug found by tests

AWSPENDING + AWSCURRENT check wasn't using `version_stages`.
Also tidy up the AWSCURRENT moving in `update_secret_version_stage`
to remove AWSPREVIOUS it from the new stage.

* Run `black` over changed files

* Add additional `rotate_secret` tests

* Skip `test_rotate_secret_lambda_invocations` in test server mode

* Add test for invalid Lambda ARN
2021-05-11 12:08:01 +01:00
Neil Greenwood
45167737e5
Fix separator in filter_log_events nextToken value. (#3914)
The old separator was present in some of the values being concatenated.
2021-05-07 18:49:36 +01:00
Josh McCullen
12b409e0de
Set multiple Security Groups when calling modify_network_interface_attribute (#3911) 2021-05-07 13:50:26 +01:00
Maria
cf3cf8b134
DMS - basic task replication methods implementation (#3900)
* DMS - basic task replication methods implementation

* Add ResourceNotFoundFault tests
2021-05-06 18:33:48 +01:00
George-lewis
f76571199f
Implement User Pool MFA Actions (#3903)
* implement user pool mfa actions

* Add messages to errors

Add messages to errors

Fix error message

* Change exception type

* fix validation & add more tests

Co-authored-by: George Lewis <glewis@evertz.com>
2021-05-06 17:59:04 +01:00
khneal
e5b3f4181d
EC2 describe_network_acls: add support for owner-id filter (#3898)
* add test that fails with FilterNotImplementedError

* describe_network_acls: add support for owner-id filter

Co-authored-by: Kevin Neal <Kevin_Neal@intuit.com>
2021-05-05 13:15:43 +01:00
Neil Greenwood
94a70e9ad1
nextToken value in logs:describeLogStreams response (#3896)
* `nextToken` value in `logs:describeLogStreams` response

Modified the pagination for FilterLogEvents to more closely follow
the real AWS behaviour.

* Make assertions work in py2 and py3.
2021-05-04 17:45:23 +01:00
usmangani1
0cff5458af
Fix : Remove Invalid request error when response-template is not specified (#3885)
* Fix : Remove Invalid request error

* Fix test and review commanets

* Remove duplicate put_integration_template

* API Gateway - Body can be None or empty

Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-05-01 15:45:06 +01:00
usmangani1
85d94ad6ed
Fix:SQS Added support for system attributes in sqs (#3878)
* Adding SQS system attributes

* Fix Comments

* Change template in response
2021-05-01 07:48:39 +01:00