* Fix : SQS - Added support for attribute labels for send_message function
* Add integration test on receive message function
* Add send message invalid datetype integration test and fix SQS MessageAttributesInvalid exceptions
* Implement Filter: Contains functionality for describe_params
This commit adds the Contains functionality. Tests were created
to mimic behavior in AWS/boto3, including that filters with values
in the form of `/name` will match parameters named `/name/match` but
not parameters named `match/with/other-name`. In the test example, a
Contains filter with the value `/tan` would match: `/tangent-3` and
`tangram-4` but not `standby-5`.
* Enforce parameter filter restrictions on get_parameters_by_path
According to the boto3 documentation [1], `Name`, `Path`, and `Tier` are not
allowed values for `Key` in a parameter filter for `get_parameters_by_path`.
This commit enforces this by calling `_validate_parameter_filters` from the
`get_parameters_by_path` method, and adding a check to `_validate_parameter_filters`.
I added 3 test cases to `test_get_parameters_by_path` which check for the correct
exception when calling with a parameter filter using any of these keys.
* Code formatted to match style
* Refactored logic
* FIX:IAM-Added support for pathPrefix in list_users_function
* removed changes for roles
* Added test for non decorator
* changed filter function
Co-authored-by: usmankb <usman@krazybee.com>
* Adding attchment.vpc-id, attachment.state, type, and vpn-gateway-id filters for VPN Gateways. fixes#3154
* Run formatting on tests
Co-authored-by: Alan Baldwin <alan.baldwin.jr@gmail.com>
* initial implementation of taskSets. Fixed a bug with ECS Service where task_definition was a required parameter.
* Added update_task_set and tests. DRYed up ClusterNotFoundException. General cleanup.
* Added support for filtering tags on include parameter to describe_task_sets. Added additional tests.
* Fix copy/pasta in ClusterNotFoundException
* styling updates
* Added TODO for delete_task_set force parameter
* Updated multiple function and constructor calls to use named variables. Updated tests to reference variables instead of hardcoded strings.
* Run black for formatting
* Updated create_service function call to use named variables
* First failing test, and enough framework to run it.
* Rudimentary passing test.
* Sagemaker Notebook Support, take-1: create, describe, start, stop, delete.
* Added list_tags.
* Merged in model support from https://github.com/porthunt/moto/tree/sagemaker-support.
* Re-org'd
* Fixed up describe_model exception when no matching model.
* Segregated tests by Sagemaker entity. Model arn check by regex..
* Python2 compabitility changes.
* Added sagemaker to list of known backends. Corrected urls.
* Added sagemaker special case to moto.server.infer_service_region_host due to irregular url format (use of 'api' subdomain) to support server mode.
* Changes for PR 3105 comments of July 10, 2020
* PR3105 July 10, 2020, 8:55 AM EDT comment: dropped unnecessary re-addition of arn when formulating model list response.
* PR 3105 July 15, 2020 9:10 AM EDT Comment: clean-up SageMakerModelBackend.describe_models logic for finding the model in the dict.
* Optimized imports
Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
* Make batch.utils.lowercase_first_key() recursive
* Reformat using black
* Add test of recursive lowercase_first_key()
* Fix typo in ttest_batch/test_cloud_formation.py
* Placeholder to test Application Autoscaling.
* Wire everything together and create a first passing test without any real functionality.
* Get one test working properly.
* Add some TODO items.
* Reformat code with black
* Second passing test for describe_scalable_targets.
* New test for NextToken.
* Add some tests for ParamValidationError and ValidationException.
* black
* Ensure scalable targets are being captured in an OrderedDict() for deterministic return later.
* Add validation to describe_scalable_targets and register_scalable_target.
* Fix tests.
* Add creation_time, refactor, add ECS backend, and add failing test for checking that ecs service exists.
* Add parameter validation.
* Improved documentation for CONTRIBUTING.md
Adds some details to give people an idea what's involved in adding new features/services
* Integrate with ECS.
* black
* Refactor to allow implementation of SuspendedState.
* Complete support for SuspendedState.
* Bump up implementation coverage percentage.
* Tidy up code; add comments.
* Implement suggested changes from code review.
* Minor refactorings for elegance.
* README update
Co-authored-by: Bert Blommers <bblommers@users.noreply.github.com>
This change fixes:
* Replace call to non-existent exception yaml.ParserError
* Catches yaml scanner error for valid json with tabs
* Supply yaml loader to ensure yaml loading throws exception validly for json with tabs and doesn't try to load the json incorrectly
* Several updates to the contributor documentation with extra information.
* Fix failing test by providing a region.
* Create test for issue 2210.
* Check if CreateBucketConfiguration is supplied and empty; raise MalformedXML error if so.
* change line position for uuid and last_modified because they're not input parameters
* add event_source_arn validator and setter
* refactor batch_size as setter
* add helper function to parse arn and return source service
* fix for EventSource's create_from_cfn, there was no reference in the lambda object for the esm if created by cfn
* add esm deletion by cloudformation
* remove unused variable in test
* add cfn's update
* add complete implementation of delete_from_cfn
* blacked changed files
* fix test with invalid batchsize for sqs
* Dynamodb2 Table - Bugfix for localindex and implemented get_cfn_attributes
* Dynamodb2 eventsource - fix test to use StreamArn attribute
* Lambda Test - fix test_update_event_source_mapping
* Correct group inline policy rendering in iam:GetAccountAuthorizationDetails response
* Include user inline policy if exists
* Add tests for IAM inline policies
* Remove unnecessary print stmts
* Fix: SecretsManager - Added missing pop() override to get_secret_name_from_arn (#3056)
* Added test case for delete_secret_force_with_arn (#3057)
* Fixed lint for test_delete_secret_force_with_arn (#3057)
* Enhancement : API-Gateway Put Integration Response - Adding support for contentHandling.
* Added tests where the contentHandling is None also gets tested.
* Linting
Co-authored-by: usmankb <usman@krazybee.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
* add EventBus to model's map
* add support for creation of EventBus through cloudformation's api
* add cloudformation's delete
* add cloudformation's update
* add cloudformation's attribute
* CloudWwatch delete Alarm status code handling on invalid alarm Name
* Handled cases where a mix of existent and non existent alarms are tried to delete
* Linting
Co-authored-by: usmankb <usman@krazybee.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
* Implemented IAM delete_instance_profile
* PR adjustment: positively verifying instance profile deletion in test case.
Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
* implemented s3 default encryption methods
* PR adjustments: moved logic for retrieving bucket's encrypted status to the backend.
Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
* Corrected bug in IAM delete_role() due to overloading of name 'role' in function
* PR-requested fixes: added region to tests boto client create, reformatted with black
Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
* ENH: Add unit test for propagation tags
* BUG: Add missing translation of boolean PropagateAtLaunch tag values to strings
* BUG: Should really be checking for "true" and not True
* CLN: Black formatting
* Keep order in request body to ensure auth signing works.
* Lint.
* More OrderedDict to ensure data parameter order.
* Lint.
* Improve CF test assertions.
* Fix syntax error.
* Cleanup CF test.
* ENH: Add unit test for cloudformation DependsOn
* ENH: Add implementation of retrieving list of resources that account for dependencies
* ENH: Update the name mappings so that they are consistent with the latest cloudformation names
* ENH: Add launch configuration to type names
* ENH: Create subnet for test and test creation with dependencies
* CLN: Code reformatting
* CLN: Remove print statements
* BUG: Fix error resulting in possible infinite loop
* CLN: Remove commented out fixture decorator
* BUG: Remove subnet creation
* CLN: Remove main and ec2 dependencies
* BUG: Add back in instance profile name type
* CLN: Remove print
* BUG: Fix broken unit test
* CLN: Code reformatting
* CLN: Remove main
* ENH: Add autoscaling group name to type names
* ENH: Add unit test for string only dependency and add assertions to unit tests
* ENH: Add unit test for chained depends_on in cloudformation stack
* BUG: Remove f strings for python 2.7 compatibility
* BUG: List needs to be sorted for python2.7
* CLN: Fix code formatting
* Added some member and proposal functions
* Added additional member and proposal functions
* Fixed admin password return and added update_member along with tests
* Added network removal and member removal proposal
* Fixed failing test
* Fixed Python 2.7 test
When using the ADD syntax to sum up different components
the path that is provided is allowed to be non-existent.
In such a case DynamoDB will initialize it depending on
the type of the value.
If it is a number it will be initialized with 0.
If it is a set it will be initialized with an empty set.
When an instance is running, OpsWorks reports its status as "online"
[1], while EC2 reports it as "running". Until now, moto copied the EC2
instance's status as is. This commit introduces the converts the running
status to online when returned by OpsWorks.
[1]: https://docs.aws.amazon.com/cli/latest/reference/opsworks/describe-instances.html
Part of structured approach for UpdateExpressions:
1) Expression gets parsed into a tokenlist (tokenized)
2) Tokenlist get transformed to expression tree (AST)
3) The AST gets validated (full semantic correctness)
4) AST gets processed to perform the update -> this commit
This commit uses the AST to execute the UpdateExpression.
All the existing tests pass. The only tests that have been
updated are in test_dynamodb_table_with_range_key.py because
they wrongly allow adding a set to a path that doesn't exist.
This has been alligend to correspond to the behavior of AWS
DynamoDB.
This commit will resolve https://github.com/spulec/moto/issues/2806
Multiple tests have been implemented that verify this.
The new version of botcore adds this region: f7dc4730ad (diff-9dfab05d4ba739e097a193e8b5fa61caR13)
Which in turn, breaks moto:
```
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/__init__.py:3: in <module>
from .acm import mock_acm # noqa
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/acm/__init__.py:2: in <module>
from .models import acm_backends
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/acm/models.py:7: in <module>
from moto.ec2 import ec2_backends
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/__init__.py:2: in <module>
from .models import ec2_backends
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:5169: in <module>
ec2_backends = {
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:5170: in <dictcomp>
region.name: EC2Backend(region.name) for region in RegionsAndZonesBackend.regions
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:5083: in __init__
super(EC2Backend, self).__init__()
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:852: in __init__
super(InstanceBackend, self).__init__()
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:1127: in __init__
super(TagBackend, self).__init__()
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:2368: in __init__
super(EBSBackend, self).__init__()
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:1904: in __init__
super(SecurityGroupBackend, self).__init__()
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:1355: in __init__
self._load_amis()
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:1362: in _load_amis
self.amis[ami_id] = Ami(self, **ami)
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:1313: in __init__
volume = self.ec2_backend.create_volume(15, region_name)
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:2372: in create_volume
zone = self.get_zone_by_name(zone_name)
/moto-1.3.15.dev640-py2.py3-none-any.whl/moto/ec2/models.py:1713: in get_zone_by_name
for zone in self.zones[self.region_name]:
E KeyError: 'af-south-1'
```
The "request" object in CallbackResponse is the PreparedRequest send by
whatever client is used to contact the mocked moto service. This can end
up with unparsed bodies, as we added for processing presigned post
requests in #2155. This will make sure that all of the requests comming
in from mocked functions also get processed by werkzeug as if it was
running a live server.
This commit puts AST validation on the execution path. This means updates get
validated prior to being executed. There were quite a few tests that were not
working against Amazon DDB. These tests I considered broken and as such this
commit adapts them such that they pass against Amazon DDB.
test_update_item_on_map()
=> One of the SET actions would try to set a nested element by specifying the nesting on the path
rather than by putting a map as a value for a non-existent key. This got changed.
test_item_size_is_under_400KB
=> Used the keyword "item" which DDB doesn't like. Change to cont in order to keep the same sizings.
=> Secondly the size error messages differs a bit depending whether it is part of the update or part
of a put_item. For an update it should be:
Item size to update has exceeded the maximum allowed size
otherwise it is
Item size has exceeded the maximum allowed size'
test_remove_top_level_attribute
=> Used a keyword item. Use ExpressionAttributeNames
test_update_item_double_nested_remove
=> Used keywords name & first. Migrated to non-deprecated API and use ExpressionAttributeNames
test_update_item_set &
test_boto3_update_item_conditions_pass &
test_boto3_update_item_conditions_pass_because_expect_not_exists &
test_boto3_update_item_conditions_pass_because_expect_not_exists_by_compare_to_null &
test_boto3_update_item_conditions_pass_because_expect_exists_by_compare_to_not_null &
test_boto3_update_item_conditions_fail &
test_boto3_update_item_conditions_fail_because_expect_not_exists &
test_boto3_update_item_conditions_fail_because_expect_not_exists_by_compare_to_null
=> Were broken tests which had string literal instead of value placeholder
Part of structured approach for UpdateExpressions:
1) Expression gets parsed into a tokenlist (tokenized)
2) Tokenlist get transformed to expression tree (AST)
3) The AST gets validated (full semantic correctness) -> this commit
4) AST gets processed to perform the update
This commit uses the AST to perform validation. Validation makes sure the
nodes encounterd have valid values and they will also resolve values for
references that refer to item state or values passed into the expression.
Part of structured approach for UpdateExpressions:
1) Expression gets parsed into a tokenlist (tokenized)
2) Tokenlist get transformed to expression tree (AST) -> This commit
3) The AST gets validated (full semantic correctness)
4) AST gets processed to perform the update
This commit uses the tokenlist to build an expression tree. This tree is not
yet used. Still it allows to raise additional Validation Exceptions which
previously were missed silently therefore it allows tests to catch these type of
ValidationException. For that reason DDB UpdateExpressions will be parsed
already. It also makes sure we won't break existing tests.
One of the existing tests had to be changed in order to still pass:
- test_dynamodb_table_with_range_key.test_update_item_with_expression
This test passed in a numeric literal which is not supported by DynamoDB
and with the current tokenization it would get the same error as in AWS
DynamoDB.
Currently the mock for DynamoDB has adhoc code to implement
its updateExpression functionality. This series will
transform the logic such that Update Expressions are processed
as follows:
1) Expression gets parsed into a tokenlist (tokenized) -> This commit
2) Tokenlist get transformed to expression tree (AST)
3) The AST gets validated (full semantic correctness)
4) AST gets processed to perform the update
This alows for a more realistic mocking. It will throw exceptions much
more aggressively avoiding situations where a test passes against the
mock but fails with an exception when running against AWS.
Introduction of step 3 also allows to have the update expression as an
atomic unit of work. So updates at the start of the expression cannot
be performed if there is an error further down the expression.
This specific commit will tokenize expressions but the tokenlist is not
yet used. It is purely to keep clear boundaries. It does do a minor
refactoring of the exceptions to allow more re-use and to ease testing.
This series of changes is to aid providing a long-term solution for
https://github.com/spulec/moto/issues/2806.
The support in this patch is preliminary and may or may not be feature complete.
It provides the basic support for update_secret so that future work can build
on it as needed.
By having models.py as one big file it causes to
easily create circular dependencies. With the current
setup it is not possible to re-use DynamoType. This
refactor moves it out to its own file while trying to
keep the structure as much as it is.
AWSEvents.DescribeRule throws an error that references a rule named "test" rather than the specified rule name when a rule with the specified name does not exist. It has been fixed to reference the specified rule name.
