Commit Graph

5920 Commits

Author SHA1 Message Date
Jordi Alhambra
407d5c853d
Media package client error additional handling (#4011)
* Add delete container and list tags endpoints to MediaStore

* Black reformat

* Fixed Lint problems

* Check if dictionary was deleted effectively

* lint fix

* MediaPackageClientError

* Lint Fix

* Test unknown channel describe

* Concatenation Fix

* MediaPackage - fix error message

* MediaPackage Test Fix and TryError

* Lint

Co-authored-by: av <arcovoltaico@gmail.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-06-16 21:07:46 +01:00
Moto Admin
fe7430b368 Increase version number post-release 2021-06-12 13:00:38 +00:00
Bert Blommers
cbeeefbec9
Prep release 2.0.9 (#4007)
* Update implementation coverage

* EC2 - Update instance type offerings

* IAM - update list of managed policies

* Changelog for release 2.0.9

* Instance Type Offerings - fix number of available offerings
2021-06-12 13:57:29 +01:00
Sahil Shah
6977bba3e1
Add capability to update AWS::ElasticLoadBalancingV2(Listener and ListenerRule) resource (#4005)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

* Added support for ListenerRule. Will remove cruft

* Pushing latest

* Something works

* Put back ripped out code

* Save point. Incase I need more validations

* Revert "Save point. Incase I need more validations"

This reverts commit dac4953335dd9335eddb7a91a63667bc3c17104c.

* Fixed validations and some refactor

* Fix formatting

* Linting

* Cannot refactor if I have to fix all tests

* Remove exceptions for now. Will do in another PR

* Remove validations. Will add in next PR

* Fix broken tests. Almost.:

* Fix all tests. Some sneaky for now.

* Python2 making me write bad code

* OrderedDict.move_to_end() does not work in python2

* Linting

* Add more checks to field in conditions later.

* Unwnated change in FakeListener

* Revert "Unwnated change in FakeListener"

This reverts commit 962c2fdfd76fce999de9feccf1dd1c3ec48c459f.

* Add back default listener rule

* Linting fix

* Fix priority sorting

* Add cloudformation test for edge case

* Add validation for ForwardConfig in Action of ListernRule CF

* use not in

* set the priority template correctly

* Check for boolean in condition

* One more check

* Implement update_from_cloudformation_json for Listener and ListenerRule

* Unwanted spaces

* Linting issues

* Add tests for code coverage

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-06-11 21:56:28 +01:00
Kyle Decot
8d4007f2b6
Escape Hyphen in Regular Expression (#4006)
* Escape Hyphen in Regular Expression

* Add Test

* Fix Linting Errors

* Use Pytest Parameterize Marker

* Fix Black Formatting Errors

* Interpolate / Use Raw String

* fixes string formatting

Co-authored-by: Correna Sprowls <correna.sprowls@joinroot.com>
2021-06-11 21:54:43 +01:00
benediktbrandt
479ce861a7
Env variable for default key buffer size (#4001)
* - introduce environment variable for DEFAULT_KEY_BUFFER_SIZE

* - prefix env variable with MOTO_S3 to avoid env variable conflicts

* - black formatting

* - fix formatting
2021-06-10 10:48:28 +01:00
Brian Pandola
c1b38be02d
Add support for DynamoDB Backup/Restore (#3995)
* Add support for DynamoDB Backup/Restore

Basic support for the following endpoints has been implemented with full test coverage:
- create_backup
- delete_backup
- describe_backup
- list_backups
- restore_table_from_backup

Behavior and error messages verified against a real AWS backend.

* Refactor test based on PR feedback
2021-06-10 07:05:07 +01:00
Brian Pandola
00ccce0723
Refactor listener rule ARN generation to avoid collisions (#4002)
The code that generates an ARN for a listener rule was relying on a random
number between 0 and 50 to ensure uniqueness.  As a result, there was a decent
chance of generating a collision, particularly when adding multiple rules.

When a collision occurred, the new rule simply overwrote an existing rule,
causing a test that asserts on an expected number of rules being returned
to fail.

Reference the following runs for related test failures:
https://github.com/spulec/moto/runs/2772429387
https://github.com/spulec/moto/runs/2786330037
2021-06-10 07:02:30 +01:00
Sahil Shah
3f5408c9d0
Adding support for ForwardConfig property in ListernRule in CloudFormation (#3993)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

* Added support for ListenerRule. Will remove cruft

* Pushing latest

* Something works

* Put back ripped out code

* Save point. Incase I need more validations

* Revert "Save point. Incase I need more validations"

This reverts commit dac4953335dd9335eddb7a91a63667bc3c17104c.

* Fixed validations and some refactor

* Fix formatting

* Linting

* Cannot refactor if I have to fix all tests

* Remove exceptions for now. Will do in another PR

* Remove validations. Will add in next PR

* Fix broken tests. Almost.:

* Fix all tests. Some sneaky for now.

* Python2 making me write bad code

* OrderedDict.move_to_end() does not work in python2

* Linting

* Add more checks to field in conditions later.

* Unwnated change in FakeListener

* Revert "Unwnated change in FakeListener"

This reverts commit 962c2fdfd76fce999de9feccf1dd1c3ec48c459f.

* Add back default listener rule

* Linting fix

* Fix priority sorting

* Add cloudformation test for edge case

* Add validation for ForwardConfig in Action of ListernRule CF

* use not in

* set the priority template correctly

* Check for boolean in condition

* One more check

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-06-09 18:41:18 +01:00
Bert Blommers
43426c71f4
ELBv2 - Simplify tests (#3999) 2021-06-09 16:34:53 +01:00
Jordi Alhambra
ae5653b31d
Media package client error (#3983)
* Add delete container and list tags endpoints to MediaStore

* Black reformat

* Fixed Lint problems

* Check if dictionary was deleted effectively

* lint fix

* MediaPackageClientError

* Lint Fix

* Test unknown channel describe

* Concatenation Fix

* MediaPackage - fix error message

Co-authored-by: av <arcovoltaico@gmail.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-06-07 13:47:49 +01:00
usmangani1
9836985473
Fix : Logs - Add start_query functionality (#3985)
* Added start_query functionality

* fix tests
2021-06-07 10:37:11 +01:00
usmangani1
d325593e46
Fix:Add list connection API's (#3976)
* Initial commit create connection

* Added connection API's

* Add destination API's

* fixed tests

* fix tests

* Fixed tests
2021-06-07 10:35:28 +01:00
Bert Blommers
74559f2a91
Organisations - Backport re.fullmatch for Py2 (#3990) 2021-06-05 17:53:06 +01:00
stromp
9e4972b43f
Add Tag support for other resource types then an account (#3982)
* - Adding checking for resource type in tag functions
- Adding TargetNotFoundException when no resource found
- Adding support for tags for root OU, OU and Policies
- Adding tests covering the new code
- Adding test for deletion of a tag

* fixed linting issue

* - renamed helper function to a more logical name
- added tests for helper function
- fixed bugs in tests for tag functions

Co-authored-by: Sjoerd Tromp <stromp@schubergphilis.com>
2021-06-05 15:12:17 +01:00
Bert Blommers
25bf10bf8e
Pin Jose-dependency for Py2 (#3989) 2021-06-05 15:04:50 +01:00
Sahil Shah
b82096ba37
Add support for AWS::ElasticLoadBalancingV2::ListenerRule (#3969)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

* Added support for ListenerRule. Will remove cruft

* Pushing latest

* Something works

* Put back ripped out code

* Save point. Incase I need more validations

* Revert "Save point. Incase I need more validations"

This reverts commit dac4953335dd9335eddb7a91a63667bc3c17104c.

* Fixed validations and some refactor

* Fix formatting

* Linting

* Cannot refactor if I have to fix all tests

* Remove exceptions for now. Will do in another PR

* Remove validations. Will add in next PR

* Fix broken tests. Almost.:

* Fix all tests. Some sneaky for now.

* Python2 making me write bad code

* OrderedDict.move_to_end() does not work in python2

* Linting

* Add more checks to field in conditions later.

* Unwnated change in FakeListener

* Revert "Unwnated change in FakeListener"

This reverts commit 962c2fdfd76fce999de9feccf1dd1c3ec48c459f.

* Add back default listener rule

* Linting fix

* Fix priority sorting

* Add cloudformation test for edge case

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-06-05 10:04:04 +01:00
Mike Grima
a281f9d4a6
Merge pull request #3987 from mikegrima/fixs3config
Fixed AWS Config S3 ACL issues
2021-06-04 12:01:58 -07:00
Mike Grima
ba7a0fe3a2 Fixed AWS Config S3 ACL issues
- AWS Config appears to have changed how ACLs are returned
- This PR makes moto's behavior consistent with AWS Config's S3 representation

Fixes #3986
2021-06-04 09:23:56 -07:00
Austin Hendrix
5602c4e73e
SSM Parameter Store improvements in GetParameterHistory & GetParameters (#3984)
* Including labels and versions in SSM Get Parameters

* implementing NextToken and MaxResults into the SSM Get Parameter History functionality

* Implementing unit tests and some lint refactoring for NextToken implementation in get_parameter_history
2021-06-04 11:12:35 +01:00
Austin Hendrix
b670962c5e
SSM Parameter Store Error Message When Requesting Invalid Version (#3977)
* Implement correct error when requesting specific version of a parameter which exists but does not have this version

* removing trailing whitespace causing lint failure

* Adding unit tests and fixing linting for new error handling

* Fixing small bug in response message

* Revert change in get_parameters as versioning is not currently implemented in this method. Will fix as a separate PR
2021-06-02 16:27:23 +01:00
Brian Pandola
7947b1e001
Enforce SSM Parameter Version Limit (#3967)
Behavior verified against a real AWS backend.
2021-05-28 08:32:39 +01:00
Bert Blommers
037973c8d9
Batch - remove potential race condition (#3965) 2021-05-28 06:55:01 +01:00
Brian Pandola
3265e89fd3
Allow Glacier VaultName to contain period (#3964) 2021-05-27 08:44:20 +01:00
Florian Dambrine
6adb570c48
Add dot as a valid s3 metadata key (#3958) 2021-05-26 14:27:25 +01:00
Kallan Gerard
75730e6f83
Missing code block language (#3962) 2021-05-26 12:28:22 +01:00
Thomas Maschler
d635c78bd1
AWS Batch enhancements (#3956)
* Check exit status of container

* Added support for job dependencies

* batch container overrides

* add AWS_BATCH_JOB_ID to container env variables

* lint with black

* refactor batch dependency test

* refactor batch dependency test

* fix index

Co-authored-by: jterry64 <justin.terry@wri.org>
Co-authored-by: Daniel Mannarino <daniel.mannarino@gmail.com>
2021-05-26 08:52:09 +01:00
rajinder
fbbc8fc472
basic implementation of update rest api (#3951)
* basic implementation of update rest api

* basic implementation of update rest api

* basic implementation of update rest api

* review comments from bblommers

Co-authored-by: rajinder saini <rajinder.saini@c02vt5k2htd6.corp.climate.com>
2021-05-23 17:09:02 +01:00
Moto Admin
290f6585c2 Increase version number post-release 2021-05-21 07:57:37 +00:00
Bert Blommers
171b5f2c8b
Prep 2.0.8 release (#3948) 2021-05-21 08:54:59 +01:00
Brian Pandola
16f4dc87dc
Add support for tagging Volumes in ec2:RunInstances (#3946) 2021-05-21 06:35:34 +01:00
Jordi Alhambra
7f49cd0ed6
MediaStore delete container and list tags endpoints implemented (#3938)
* Add delete container and list tags endpoints to MediaStore

* Black reformat

* Fixed Lint problems

* Check if dictionary was deleted effectively

* lint fix

Co-authored-by: av <arcovoltaico@gmail.com>
2021-05-20 16:17:31 +01:00
Brian Pandola
752b508738
Add support for KmsKeyId in ec2:RunInstances (#3943) 2021-05-20 07:11:50 +01:00
rajinder
58f7ab0d25
Refector Apigateway update_usage_plan and add all replace operation. (#3942)
* Refector Apigateway update_usage_plan and add all replace operation.

* code reformatting

Co-authored-by: rajinder saini <rajinder.saini@c02vt5k2htd6.corp.climate.com>
2021-05-20 07:10:08 +01:00
Ben Beasley
79792db4c0
Drop dependency on zipp (#3940)
A search of the repository shows zipp is not imported anywhere.
2021-05-19 19:39:04 +01:00
Sahil Shah
24fa8f25a1
SSM parameters in cloudformation (AWS::SSM::Parameter::) are not recognized and resolved (#3929)
* Add ssm parsing support for cloudformation stacks

* Start adding unit tests for ssm parameter parsing

* Add tests for code update

* Add tests to parse ssm parameters code

* Fix black lint errors

* Fix bug.

* Need to specify region_name

* region needs to be same

* Use ssm_backends[region] instead of ssm_backend

* StringList -> string

* Linting

* check if servermode tests are on

* Typo

Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
2021-05-19 08:30:25 +01:00
usmangani1
4ad6c3932e
Fix resource group get-group-query & update-query-query (#3934)
* Fix resource group get-group-query & update-query-query

* Remove list logic

* Added tests

* Fix tests

* Fix update query tests

* fix tests
2021-05-18 20:58:21 +01:00
Giovanni Torres
867cf078de
feat: transition elbv2 state to active on first describe (#3937)
* refactor: parameterize state in elbv2 template response

* feat: transition load balancer to active on first describe

Closes #3936
2021-05-18 07:52:39 +01:00
usmangani1
31cf3c4252
Fix:SES-Added params check in template (#3753)
* Fix:SES-Added params check in template

* Added more tests and handled message

* linting

* fixed tests

* fix attribute name in message

* fix logic for exception
2021-05-18 07:51:27 +01:00
usmangani1
abc62b360d
Fix- Raise InvalidParameterValue when change visibility timeout is mo… (#3893)
* Fix- Raise InvalidParameterValue when change visibility timeout is more than permitted

* Fix review comments

* Modified tests

* Fix tests

* change

* Change client

* Fix tests

* fix tests

* fix tests

* change tests

* Fix tests

* changed tests

* Fix tests

* Fix tests

* Change message
2021-05-18 07:49:56 +01:00
usmangani1
03b2009a1c
Add unit support to cloudwatch get_metric_statistics (#3932)
* Add unit support to cloudwatch get_metric_statistics

* fix linting
2021-05-15 08:05:41 +01:00
usmangani1
58fd4aeaae
Fix tags for resource in ListTagsForResource (#3931)
* Fix tags for resource in ListTagsForResource

* Fix review comments

* Fix tests
2021-05-14 20:05:40 +01:00
usmangani1
eaac32b130
ResourceGroup - Add Support for getGroupConfiguration (#3919)
* ResourceGroup - Add Support for getGroupConfiguration

* Add tests

* Add tests

* Fix tests

* linting

* fix tests

* fix tests

* fix tests

* fixed linting

* fix tests
2021-05-14 07:40:17 +01:00
Kai
f9e0595e12
Fix sqs message retention logic (#3924)
* Fix sqs message retention logic

* Apply lint to moto/sqs/models.py

* Fix failed tests because of freezing time

* Fix freezing time in test_publish_to_sqs_in_different_region
2021-05-13 15:06:54 +01:00
Bert Blommers
9e3faf7784
Make Moto (tests) compatible with flask/werkzeug 2.x (#3923)
* Dont fail if CodeCov fails - for now

* CI - Force cache rebuild

* Bump werkzeug to latest version

* CI - Enforce cache flush

* ManagedBlockchain - fix error format

* ManagedBlockchain - Fix tests to use pytest.raises paradigm

* Revert "Lock Flask (#3925)"

This reverts commit 8bb0feb956.

* CI - Enforce cache rebuild
2021-05-13 10:36:56 +01:00
Guilherme Scaranse
8bb0feb956
Lock Flask (#3925)
* Update setup.py

* Update setup.py
2021-05-12 18:41:57 +01:00
Moto Admin
f45ddf0345 Increase version number post-release 2021-05-12 08:04:55 +00:00
Bert Blommers
1fc09b5863
Prep release 2.0.7 (#3920)
* Temporarily downgrade werkzeug until we fix the tests

* ChangeLog for release 2.0.7
2021-05-12 08:55:50 +01:00
George-lewis
b4d02c922c
Update coverage (#3918) 2021-05-11 21:40:17 +01:00
Daniel Samuels
a4b1498665
Support rotating secrets using Lambda [#3905] (#3912)
* Support rotating secrets using Lambda

The Secrets manager rotation process uses an AWS Lambda function
to perform the rotation of a secret. [1]

In fact, it's not possible to trigger rotation of a Secret
without specifying a Lambda function at some point in the life
of the secret:

```
$ aws secretsmanager rotate-secret --secret-id /rotationTest

An error occurred (InvalidRequestException) when calling the RotateSecret operation: No Lambda rotation function ARN is associated with this secret.
```

`moto` can be a little more lenient in this regard and allow
`rotate_secret` to be called without a Lambda function being
present, if only to allow simulation of the `AWSCURRENT` and
`AWSPREVIOUS` labels moving across versions.

However, if a lambda function _has_ been specified when calling
`rotate_secret`, it should be invoked therefore providing the
developer with the full multi-stage process [3] which can be
used to test the Lambda function itself and ensuring that full
end-to-end testing is performed. Without this there's no easy
way to configure the Secret in the state needed to provide the
Lambda function with the data in the format it needs to be in
at each step of the invocation process.

[1]: https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-lambda-function-overview.html
[2]: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.rotate_secret
[3]: https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-lambda-function-overview.html#rotation-explanation-of-steps

* Run `black` over `secretsmanager/models.py`

* Make `lambda_backends` import local to the condition

* Implement `update_secret_version_stage`

Allow a staging label to be moved across versions.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager.html#SecretsManager.Client.update_secret_version_stage

* Add an integration test for Secrets Manager & Lambda

* Support passing `ClientRequestToken` to `put_secret_value`

By passing `ClientRequestToken` to `put_secret_value` within
the lambda function  invoked by calling `rotate_secret`, one
can update the value associated with the existing (pending)
version, without causing a new secret version to be created.

* Add application logic for `AWSPENDING`

The rotation function must end with the versions of the secret
in one of two states:

 - The `AWSPENDING` and `AWSCURRENT` staging labels are
   attached to the same version of the secret, or
 - The `AWSPENDING` staging label is not attached to any
   version of the secret.

If the `AWSPENDING` staging label is present but not attached
to the same version as `AWSCURRENT` then any later invocation
of RotateSecret assumes that a previous rotation request is
still in progress and returns an error.

* Update `default_version_id` after Lambda rotation concludes

Call `set_default_version_id` directly, rather than going 
through `reset_default_version` as the Lambda function is 
responsible for moving the version labels around, not `rotate_secret`.

* Run `black` over changed files

* Fix Python 2.7 compatibility

* Add additional test coverage for Secrets Manager

* Fix bug found by tests

AWSPENDING + AWSCURRENT check wasn't using `version_stages`.
Also tidy up the AWSCURRENT moving in `update_secret_version_stage`
to remove AWSPREVIOUS it from the new stage.

* Run `black` over changed files

* Add additional `rotate_secret` tests

* Skip `test_rotate_secret_lambda_invocations` in test server mode

* Add test for invalid Lambda ARN
2021-05-11 12:08:01 +01:00