240 Commits

Author SHA1 Message Date
Bert Blommers
f923d0d1e0
Feature: Custom cloudformation resources (#4512) 2021-11-03 20:00:42 -01:00
Bert Blommers
14a69c7524
Techdebt: Enable pylint rules (#4432) 2021-10-18 19:44:29 +00:00
Bert Blommers
8526013e61
Parallelize tests - Part 1 (#4368) 2021-10-05 17:11:07 +00:00
kbalk
3a203d35c9
Implement EC2 describe_vpc_endpoint_services() (#4322) 2021-09-24 16:01:09 +00:00
Bert Blommers
d87a2eab49
SQS - Allow XML content to be received as a message attr (#4337) 2021-09-23 18:55:22 +00:00
Silvio
79f70aa0c8
Change message deletion behavior of SQS queue to use discard (#4287) 2021-09-15 20:05:45 +00:00
Miguel Gagliardo
85f38859c3
Fix: Issue with empty tags created on SQS queues (#4254) 2021-09-01 11:57:45 +01:00
Bert Blommers
07bb843214
Fix regex URLs so that a dot is actually interpreted as a dot (#4110) 2021-08-28 13:42:45 +01:00
Bert Blommers
ec70d3cd14
SQS - get_queue_attributes() might not have AttributeNames-parameter (#4236) 2021-08-28 09:34:33 +01:00
Bert Blommers
6f4b6080f8
SQS - Raise error message when providing an invalid value for the queue_url parameter (#3657) 2021-08-28 06:19:01 +01:00
Anton Grübel
e865362791
Add fifo high throughput (#4224) 2021-08-26 16:23:17 +01:00
Bert Blommers
11883a1fda
Remove unnecessary dependencies EC2/SQS (#4094)
* Add missing dependencies for EFS
2021-07-29 06:38:16 +01:00
Miguel Gagliardo
f9f93531e8
Fix: Adding empty tags to SQS queue on creation time (#4087)
* Fix: Adding tags to SQS queue on creation time
2021-07-27 08:24:38 +01:00
Miguel Gagliardo
44624593f1
Fix: Adding fix when setting empty policy attr to queue (#4091) 2021-07-27 06:04:24 +01:00
Waldemar Hummer
f4f8527955
Merge LocalStack changes into upstream moto (#4082)
* fix OPTIONS requests on non-existing API GW integrations

* add cloudformation models for API Gateway deployments

* bump version

* add backdoor to return CloudWatch metrics

* Updating implementation coverage

* Updating implementation coverage

* add cloudformation models for API Gateway deployments

* Updating implementation coverage

* Updating implementation coverage

* Implemented get-caller-identity returning real data depending on the access key used.

* bump version

* minor fixes

* fix Number data_type for SQS message attribute

* fix handling of encoding errors

* bump version

* make CF stack queryable before starting to initialize its resources

* bump version

* fix integration_method for API GW method integrations

* fix undefined status in CF FakeStack

* Fix apigateway issues with terraform v0.12.21
* resource_methods -> add handle for "DELETE" method
* integrations -> fix issue that "httpMethod" wasn't included in body request (this value was set as the value from refer method resource)

* bump version

* Fix setting http method for API gateway integrations (#6)

* bump version

* remove duplicate methods

* add storage class to S3 Key when completing multipart upload (#7)

* fix SQS performance issues; bump version

* add pagination to SecretsManager list-secrets (#9)

* fix default parameter groups in RDS

* fix adding S3 metadata headers with names containing dots (#13)

* Updating implementation coverage

* Updating implementation coverage

* add cloudformation models for API Gateway deployments

* Updating implementation coverage

* Updating implementation coverage

* Implemented get-caller-identity returning real data depending on the access key used.

* make CF stack queryable before starting to initialize its resources

* bump version

* remove duplicate methods

* fix adding S3 metadata headers with names containing dots (#13)

* Update amis.json to support EKS AMI mocks (#15)

* fix PascalCase for boolean value in ListMultipartUploads response (#17); fix _get_multi_param to parse nested list/dict query params

* determine non-zero container exit code in Batch API

* support filtering by dimensions in CW get_metric_statistics

* fix storing attributes for ELBv2 Route entities; API GW refactorings for TF tests

* add missing fields for API GW resources

* fix error messages for Route53 (TF-compat)

* various fixes for IAM resources (tf-compat)

* minor fixes for API GW models (tf-compat)

* minor fixes for API GW responses (tf-compat)

* add s3 exception for bucket notification filter rule validation

* change the way RESTErrors generate the response body and content-type header

* fix lint errors and disable "black" syntax enforcement

* remove return type hint in RESTError.get_body

* add RESTError XML template for IAM exceptions

* add support for API GW minimumCompressionSize

* fix casing getting PrivateDnsEnabled API GW attribute

* minor fixes for error responses

* fix escaping special chars for IAM role descriptions (tf-compat)

* minor fixes and tagging support for API GW and ELB v2 (tf-compat)

* Merge branch 'master' into localstack

* add "AlarmRule" attribute to enable support for composite CloudWatch metrics

* fix recursive parsing of complex/nested query params

* bump version

* add API to delete S3 website configurations (#18)

* use dict copy to allow parallelism and avoid concurrent modification exceptions in S3

* fix precondition check for etags in S3 (#19)

* minor fix for user filtering in Cognito

* fix API Gateway error response; avoid returning empty response templates (tf-compat)

* support tags and tracingEnabled attribute for API GW stages

* fix boolean value in S3 encryption response (#20)

* fix connection arn structure

* fix api destination arn structure

* black format

* release 2.0.3.37

* fix s3 exception tests

see botocore/parsers.py:1002 where RequestId is removed from parsed

* remove python 2 from build action

* add test failure annotations in build action

* fix events test arn comparisons

* fix s3 encryption response test

* return default value "0" if EC2 availableIpAddressCount is empty

* fix extracting SecurityGroupIds for EC2 VPC endpoints

* support deleting/updating API Gateway DomainNames

* fix(events): Return empty string instead of null when no pattern is specified in EventPattern (tf-compat) (#22)

* fix logic and revert CF changes to get tests running again (#21)

* add support for EC2 customer gateway API (#25)

* add support for EC2 Transit Gateway APIs (#24)

* feat(logs): add `kmsKeyId` into `LogGroup` entity (#23)

* minor change in ELBv2 logic to fix tests

* feat(events): add APIs to describe and delete CloudWatch Events connections (#26)

* add support for EC2 transit gateway route tables (#27)

* pass transit gateway route table ID in Describe API, minor refactoring (#29)

* add support for EC2 Transit Gateway Routes (#28)

* fix region on ACM certificate import (#31)

* add support for EC2 transit gateway attachments (#30)

* add support for EC2 Transit Gateway VPN attachments (#32)

* fix account ID for logs API

* add support for DeleteOrganization API

* feat(events): store raw filter representation for CloudWatch events patterns (tf-compat) (#36)

* feat(events): add support to describe/update/delete CloudWatch API destinations (#35)

* add Cognito UpdateIdentityPool, CW Logs PutResourcePolicy

* feat(events): add support for tags in EventBus API (#38)

* fix parameter validation for Batch compute environments (tf-compat)

* revert merge conflicts in IMPLEMENTATION_COVERAGE.md

* format code using black

* restore original README; re-enable and fix CloudFormation tests

* restore tests and old logic for CF stack parameters from SSM

* parameterize RequestId/RequestID in response messages and revert related test changes

* undo LocalStack-specific adaptations

* minor fix

* Update CodeCov config to reflect removal of Py2

* undo change related to CW metric filtering; add additional test for CW metric statistics with dimensions

* Terraform - Extend whitelist of running tests

Co-authored-by: acsbendi <acsbendi28@gmail.com>
Co-authored-by: Phan Duong <duongpv@outlook.com>
Co-authored-by: Thomas Rausch <thomas@thrau.at>
Co-authored-by: Macwan Nevil <macnev2013@gmail.com>
Co-authored-by: Dominik Schubert <dominik.schubert91@gmail.com>
Co-authored-by: Gonzalo Saad <saad.gonzalo.ale@gmail.com>
Co-authored-by: Mohit Alonja <monty16597@users.noreply.github.com>
Co-authored-by: Miguel Gagliardo <migag9@gmail.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-07-26 15:21:17 +01:00
Bert Blommers
7693d77333
Remove Py2 support (#3915) 2021-07-26 07:40:39 +01:00
Grégoire Charvet 黑瓜
00be464c05
Fix #3996 (#4051)
* Fix #3996

* Don't delete any message when duplicate handles
2021-07-07 08:59:16 +01:00
Grégoire Charvet 黑瓜
b44ab23c63
Fix #4009 (#4052)
Correctly raises an error when MessageGroupId is missing when sending a
batch of messages to a FIFO queue.
2021-07-04 07:40:39 +01:00
usmangani1
abc62b360d
Fix- Raise InvalidParameterValue when change visibility timeout is mo… (#3893)
* Fix- Raise InvalidParameterValue when change visibility timeout is more than permitted

* Fix review comments

* Modified tests

* Fix tests

* change

* Change client

* Fix tests

* fix tests

* fix tests

* change tests

* Fix tests

* changed tests

* Fix tests

* Fix tests

* Change message
2021-05-18 07:49:56 +01:00
Kai
f9e0595e12
Fix sqs message retention logic (#3924)
* Fix sqs message retention logic

* Apply lint to moto/sqs/models.py

* Fix failed tests because of freezing time

* Fix freezing time in test_publish_to_sqs_in_different_region
2021-05-13 15:06:54 +01:00
usmangani1
85d94ad6ed
Fix:SQS Added support for system attributes in sqs (#3878)
* Adding SQS system attributes

* Fix Comments

* Change template in response
2021-05-01 07:48:39 +01:00
Anton Grübel
6da4905da9
Fix AttributeNames for sqs.receive_message (#3736)
* Fix AttributeNames for sqs.receive_message

* Fix Lambda issue

* Change to parametrized tests

* Simplify attribute logic
2021-03-05 10:42:07 +00:00
usmangani1
d8097b24dc
Fix:SQS-message retention period consideration (#3642)
* Fix:SQS-message retention period consideration

* Fix:SQS-message retention period consideration

* Fix:SQS-message retention period consideration

* Added comments &linting

* Fixed tests
2021-02-14 13:04:55 +00:00
Bert Blommers
9ecea2012a
Fix Blockchain to work with new API (#3575)
* Fix Blockchain to work with new API

* Improve Lambda->SQS error handling
2021-01-07 11:18:50 +00:00
irahulranjan
54bd336457
Fix: FIFO DLQ locks message group id (#3537)
* fix https://github.com/localstack/localstack/issues/3319

* fix review comments
2020-12-14 09:19:22 +00:00
irahulranjan
7b97141184
fix Fifo queue delivers duplicate message(s) (#3538)
* fix https://github.com/localstack/localstack/issues/3339

* fixe lint issues

* Fix review comments
- move deduplication time to constants
- make tests parameterized
- update tests as per review comments

* change variable name expectedCount => expected_count

* fix tests for python 2.7
increase deduplication mock config to account for delays

* ignore time mocking test in server mode
2020-12-13 14:39:10 +00:00
Artem
53cc3dd67a
Fix SQS md5 attribute hashing. (#3403)
* Fix sqs md5 attribute hashing.

* Fix test name.

* Fix format.
2020-10-27 12:13:47 +00:00
usmangani1
fcc85a9645
Including Message attributes when ALL is passed (#3393)
* Including Message attributes when ALL is passes

* Added tests

Co-authored-by: usmankb <usman@krazybee.com>
2020-10-19 09:04:38 +01:00
Brian Pandola
a2bd4515eb
Add better support for SQS MaximumMessageSize attribute (#3374)
Closes #3205
2020-10-10 17:54:36 +01:00
usmangani1
a65c0f004c
Fix:SQS Receive Message (MessageAttributes) in response (#3303)
* Fix:SQS Receive Message (MessageAttributes) in response

* Fixed tests

Co-authored-by: usmankb <usman@krazybee.com>
2020-10-05 10:10:24 +01:00
usmangani1
f9ce99f0d1
Fix:SQS md5 calculation for custom string data type. (#3346)
* Fix:SQS md5 calculation for custom string data type.

* Linting

Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-09-30 18:40:00 +01:00
usmangani1
1c3ba83fc2
Fix : SQS Create Queue with attributes (#3345)
* Considering only new attributes while queue creation

* Linting

Co-authored-by: usmankb <usman@krazybee.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-09-30 16:34:15 +01:00
Karthikeyan Singaravelan
7054143701
Fix deprecation warnings due to invalid escape sequences. (#3273)
* Fix deprecation warnings due to invalid escape sequences.

* Fix linter error.
2020-09-10 09:20:26 +01:00
usmangani1
3ea46617d9
Fix:sqs get-queue-attributes response template (#3255)
* Fix:sqs get-queue-attributes response template

* Fix:removed debug statements

* Modified the template

* "fixed build issues"

* Linting

Co-authored-by: usmankb <usman@krazybee.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-09-01 18:05:25 +01:00
usmangani1
0a89f9d1df
Fix:SQS:Added Non existent queue name in ERROR RESPONSE (#3261)
* Fix:SQS:Added Non existent queue name in ERROR RESPONSE

* Linting

Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-08-27 17:31:20 +01:00
jweite
49d92861c0
Iam cloudformation update, singificant cloudformation refactoring (#3218)
* IAM User Cloudformation Enhancements: update, delete, getatt.

* AWS::IAM::Policy Support

* Added unit tests for AWS:IAM:Policy for roles and groups.  Fixed bug related to groups.

* AWS:IAM:AccessKey CloudFormation support.

* Refactor of CloudFormation parsing.py methods to simplify and standardize how they call to the models.  Adjusted some models accordingly.

* Further model CloudFormation support changes to align with revised CloudFormation logic.  Mostly avoidance of getting resoure name from properties.

* Support for Kinesis Stream RetentionPeriodHours param.

* Kinesis Stream Cloudformation Tag Support.

* Added omitted 'region' param to boto3.client() calls in new tests.

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-08-27 10:11:47 +01:00
Yuuki Takahashi
99736c3101
fix clear pending messages when call purge_queue (#3208) 2020-08-03 15:09:25 +01:00
Adam Richie-Halford
9a9a1d8413
Decentralize cloudformation naming responsibilities (#3201)
* #3127 - Decentralize CF naming responsibilities

* Decentralize CloudFormation naming responsibilities

* Update URLs in cloudformation_resource_type functions

* Fix flake8 errors

* Black formatting

* Add a bunch of imports to populate CloudFormationModel.__subclasses__

* Add noqa to s3 models import statement in cloudformation/parsing.py

* Black formatting

* Remove debugging print statement

Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-08-01 15:23:36 +01:00
Waldemar Hummer
08a08b6af8
Fix SQS tag list from CloudFormation resource creation (#3197)
* fix sqs tag list from cloudformation resource creation

the method `create_from_cloudformation_json` of the Sqs resource
does not handle the difference of format of the Tags field in the
resource template and the format expected in Sqs resource class.

In cfn resource template Tags is specified as a list of dicts. But
the Sqs resource expects that the tags field be a single dict.
This behaviour causes a crash when a queue is created with tags
from `create_from_cloudformation_json` and later the list_queue_tags
is called because it tries to call `items` from `queue.tags` but
tags is actually a list of dicts.

* fix comment

* fix linter

* minor

Co-authored-by: Hudo Assenco <hudo.assenco@gmail.com>
2020-07-29 11:44:02 +01:00
Ninh Khong
97139d4253
Fix : SQS - Added support for attribute labels for send_message function (#3181)
* Fix : SQS - Added support for attribute labels for send_message function

* Add integration test on receive message function

* Add send message invalid datetype integration test and fix SQS MessageAttributesInvalid exceptions
2020-07-28 14:34:26 +01:00
ryanlchandler
bf8eb11dc3
Adding MessageGroupId and MessageDeduplicationId (#3163)
Adding MessageGroupId and MessageDeduplicationId when sent from batch (#3101)
2020-07-22 13:09:12 +01:00
Bert Blommers
1ef3094e45 SQS - Return multiple group-messages in the same request 2020-05-24 12:12:35 +01:00
Kevin Neal
52cbdd72e7 update SQS MaximumMessageSize from 64K to 256K 2020-05-02 16:40:52 -07:00
usmankb
1431348161 Fix SQS send_message_batch empty array Exception handling 2020-05-02 01:33:58 +05:30
jmsanders
0b7e990bbf Limit SQS list_queues response to 1000 queues
The maximum number of queues that the ListQueues API can return is 1000:

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ListQueues.html
2020-02-07 15:54:42 -06:00
gruebel
44024ab74b Fix sqs permission handling & add more error handling 2020-01-30 22:42:27 +01:00
Steve Pulec
a32b3c4b59 Fix SQS get_queue_attributes to allow RedrivePolicy. Closes #2682. 2020-01-22 19:38:07 -06:00
gruebel
9d9b620878 Fixed linter errors 2019-12-26 21:03:49 +01:00
gruebel
d7ba355a65 Add missing regions to all services 2019-12-26 17:12:22 +01:00
Fabio Dias
743dd46399 black linting 2019-12-16 21:25:20 -05:00