165 Commits

Author SHA1 Message Date
kbalk
3a203d35c9
Implement EC2 describe_vpc_endpoint_services() (#4322) 2021-09-24 16:01:09 +00:00
Bert Blommers
728c0c91b4
AWSLambda:list_functions() should only return the latest version by default (#4047) 2021-08-28 15:26:44 +01:00
Bert Blommers
027d05e21c
Improve logs when Docker is not running (#3711) 2021-08-28 10:48:28 +01:00
Bert Blommers
0e302a97cb
Streamline success/failure response from AWSLambda (#4000) 2021-08-28 09:41:05 +01:00
Bert Blommers
cc50acc8b5
AWSLambda - Allow ARN as parameter (#3740) 2021-08-28 07:23:44 +01:00
Bert Blommers
a1905ad584
Improve test coverage (#4098) 2021-07-28 11:17:15 +01:00
Waldemar Hummer
f4f8527955
Merge LocalStack changes into upstream moto (#4082)
* fix OPTIONS requests on non-existing API GW integrations

* add cloudformation models for API Gateway deployments

* bump version

* add backdoor to return CloudWatch metrics

* Updating implementation coverage

* Updating implementation coverage

* add cloudformation models for API Gateway deployments

* Updating implementation coverage

* Updating implementation coverage

* Implemented get-caller-identity returning real data depending on the access key used.

* bump version

* minor fixes

* fix Number data_type for SQS message attribute

* fix handling of encoding errors

* bump version

* make CF stack queryable before starting to initialize its resources

* bump version

* fix integration_method for API GW method integrations

* fix undefined status in CF FakeStack

* Fix apigateway issues with terraform v0.12.21
* resource_methods -> add handle for "DELETE" method
* integrations -> fix issue that "httpMethod" wasn't included in body request (this value was set as the value from refer method resource)

* bump version

* Fix setting http method for API gateway integrations (#6)

* bump version

* remove duplicate methods

* add storage class to S3 Key when completing multipart upload (#7)

* fix SQS performance issues; bump version

* add pagination to SecretsManager list-secrets (#9)

* fix default parameter groups in RDS

* fix adding S3 metadata headers with names containing dots (#13)

* Updating implementation coverage

* Updating implementation coverage

* add cloudformation models for API Gateway deployments

* Updating implementation coverage

* Updating implementation coverage

* Implemented get-caller-identity returning real data depending on the access key used.

* make CF stack queryable before starting to initialize its resources

* bump version

* remove duplicate methods

* fix adding S3 metadata headers with names containing dots (#13)

* Update amis.json to support EKS AMI mocks (#15)

* fix PascalCase for boolean value in ListMultipartUploads response (#17); fix _get_multi_param to parse nested list/dict query params

* determine non-zero container exit code in Batch API

* support filtering by dimensions in CW get_metric_statistics

* fix storing attributes for ELBv2 Route entities; API GW refactorings for TF tests

* add missing fields for API GW resources

* fix error messages for Route53 (TF-compat)

* various fixes for IAM resources (tf-compat)

* minor fixes for API GW models (tf-compat)

* minor fixes for API GW responses (tf-compat)

* add s3 exception for bucket notification filter rule validation

* change the way RESTErrors generate the response body and content-type header

* fix lint errors and disable "black" syntax enforcement

* remove return type hint in RESTError.get_body

* add RESTError XML template for IAM exceptions

* add support for API GW minimumCompressionSize

* fix casing getting PrivateDnsEnabled API GW attribute

* minor fixes for error responses

* fix escaping special chars for IAM role descriptions (tf-compat)

* minor fixes and tagging support for API GW and ELB v2 (tf-compat)

* Merge branch 'master' into localstack

* add "AlarmRule" attribute to enable support for composite CloudWatch metrics

* fix recursive parsing of complex/nested query params

* bump version

* add API to delete S3 website configurations (#18)

* use dict copy to allow parallelism and avoid concurrent modification exceptions in S3

* fix precondition check for etags in S3 (#19)

* minor fix for user filtering in Cognito

* fix API Gateway error response; avoid returning empty response templates (tf-compat)

* support tags and tracingEnabled attribute for API GW stages

* fix boolean value in S3 encryption response (#20)

* fix connection arn structure

* fix api destination arn structure

* black format

* release 2.0.3.37

* fix s3 exception tests

see botocore/parsers.py:1002 where RequestId is removed from parsed

* remove python 2 from build action

* add test failure annotations in build action

* fix events test arn comparisons

* fix s3 encryption response test

* return default value "0" if EC2 availableIpAddressCount is empty

* fix extracting SecurityGroupIds for EC2 VPC endpoints

* support deleting/updating API Gateway DomainNames

* fix(events): Return empty string instead of null when no pattern is specified in EventPattern (tf-compat) (#22)

* fix logic and revert CF changes to get tests running again (#21)

* add support for EC2 customer gateway API (#25)

* add support for EC2 Transit Gateway APIs (#24)

* feat(logs): add `kmsKeyId` into `LogGroup` entity (#23)

* minor change in ELBv2 logic to fix tests

* feat(events): add APIs to describe and delete CloudWatch Events connections (#26)

* add support for EC2 transit gateway route tables (#27)

* pass transit gateway route table ID in Describe API, minor refactoring (#29)

* add support for EC2 Transit Gateway Routes (#28)

* fix region on ACM certificate import (#31)

* add support for EC2 transit gateway attachments (#30)

* add support for EC2 Transit Gateway VPN attachments (#32)

* fix account ID for logs API

* add support for DeleteOrganization API

* feat(events): store raw filter representation for CloudWatch events patterns (tf-compat) (#36)

* feat(events): add support to describe/update/delete CloudWatch API destinations (#35)

* add Cognito UpdateIdentityPool, CW Logs PutResourcePolicy

* feat(events): add support for tags in EventBus API (#38)

* fix parameter validation for Batch compute environments (tf-compat)

* revert merge conflicts in IMPLEMENTATION_COVERAGE.md

* format code using black

* restore original README; re-enable and fix CloudFormation tests

* restore tests and old logic for CF stack parameters from SSM

* parameterize RequestId/RequestID in response messages and revert related test changes

* undo LocalStack-specific adaptations

* minor fix

* Update CodeCov config to reflect removal of Py2

* undo change related to CW metric filtering; add additional test for CW metric statistics with dimensions

* Terraform - Extend whitelist of running tests

Co-authored-by: acsbendi <acsbendi28@gmail.com>
Co-authored-by: Phan Duong <duongpv@outlook.com>
Co-authored-by: Thomas Rausch <thomas@thrau.at>
Co-authored-by: Macwan Nevil <macnev2013@gmail.com>
Co-authored-by: Dominik Schubert <dominik.schubert91@gmail.com>
Co-authored-by: Gonzalo Saad <saad.gonzalo.ale@gmail.com>
Co-authored-by: Mohit Alonja <monty16597@users.noreply.github.com>
Co-authored-by: Miguel Gagliardo <migag9@gmail.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-07-26 15:21:17 +01:00
Chih-Hsuan Yen
2000f6654f
Support Podman for mocking Lambda (#3702)
* Support Podman for mocking Lambda

Podman supports all Docker APIs used in moto since version 3.0. Note
that Podman requires pulling the image before creating a container
using a fully-qualified image name (e.g., "docker.io/library/busybox"
instead of "busybox").

Test plan:
$ podman system service -t 0
$ DOCKER_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock" pytest

Fixes https://github.com/spulec/moto/issues/3276

* Run black

* Python 2 compatibility

* Address review comments and improve parse_image_ref
2021-02-18 08:58:20 +00:00
Bert Blommers
f64532ed40
Cloudformation - Create ApiGateway resources (#3659)
* Cloudformation - Create ApiGateway resources

* Cleanup

* Linting
2021-02-15 10:31:33 +00:00
Brian Pandola
f4b81e69b8
Address pytest warnings (#3629)
* Address `boto` deprecation warnings

This commit eliminates the following warning:

../boto/ec2/connection.py:582:
  PendingDeprecationWarning: The current get_all_instances implementation will be replaced with get_all_reservations.

`boto` isn't likely to ever make good on this warning, but doing the replacement will
declutter the `moto` test output.

* Remove `invoke_lambda` tracebacks from unit test logging

If an exception is encountered, the details are returned in the response payload.
Printing the traceback was just adding noise to the pytest output.

* Use known AMIs in unit tests

This commit eliminates the following warning in the pytest output:

`PendingDeprecationWarning: Could not find AMI with image-id:ami-123456, in the near future this will cause an error.`

Known, pre-loaded AMI image ids are used instead of random ids that don't actually
exist in the moto backend.  The integrity of the tests is unaffected by this change.

A test has been added to provide explicit coverage of the PendingDeprecationWarning
raised when an invalid AMI image id is passed to moto.
2021-01-29 11:31:56 +00:00
Laurie O
ddd3c0edc4
Assume synchronous Lambda invocation by default (#3615)
* Assume synchronous Lambda invocation by default

* Support Python 2 with dict-unpacking
2021-01-26 13:28:01 +00:00
Erez Freiberger
9784e1b487
adding layers support for lambda backend (#3563)
* adding layers support for lambda backend

* improving lambda layer tests

* adding lambda list_layers and fixing tests

* make format
2021-01-17 15:28:49 +00:00
Oide Brett
83507fbc37
fixed issue in update_configuration for lambda when setting VPC config property (#3479) 2020-11-18 08:45:31 +00:00
Bert Blommers
b5b1c45d68 Rename DockerUtilities to differentiate from docker-dependency 2020-11-09 16:31:18 +00:00
Bert Blommers
8d3cc3ef32 #3359 - Reuse Docker-on-request for AWSLambda and Batch 2020-11-08 15:16:53 +00:00
Bert Blommers
390a4d5510 #3359 - Only initiate Docker when invoking Lambdas 2020-11-08 14:18:49 +00:00
jweite
49d92861c0
Iam cloudformation update, singificant cloudformation refactoring (#3218)
* IAM User Cloudformation Enhancements: update, delete, getatt.

* AWS::IAM::Policy Support

* Added unit tests for AWS:IAM:Policy for roles and groups.  Fixed bug related to groups.

* AWS:IAM:AccessKey CloudFormation support.

* Refactor of CloudFormation parsing.py methods to simplify and standardize how they call to the models.  Adjusted some models accordingly.

* Further model CloudFormation support changes to align with revised CloudFormation logic.  Mostly avoidance of getting resoure name from properties.

* Support for Kinesis Stream RetentionPeriodHours param.

* Kinesis Stream Cloudformation Tag Support.

* Added omitted 'region' param to boto3.client() calls in new tests.

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-08-27 10:11:47 +01:00
Guilherme Martins Crocetti
f744356da7
Lambda reserved concurrency (#3215)
* lambda-responses: add method to dispatch concurrency calls

* lambda-resources: add route to handle concurrency requests

* lambda-model: implement put_function_concurrency and concurrency attribute

* put-concurrency-tests: add one simple test

* get_function: add concurrency entry - with test

* lambda-reserved-concurrency: cloudformation support

* lambda-concurrency: implement delete_reserved with tests

* lambda-concurrency: implement get_reserved with tests

* lint

* implementation-cov: mark delete_function_concurrency, put_function_concurrency and get_function_concurrency

* botocore doesn't display concurrency entry for lambdas without it

* lambda(refactor): improvements on response's handler
2020-08-26 11:06:53 +01:00
Kamil Mankowski (kam193)
c917ac6ecb
Set more lambda runtime variables 2020-08-19 23:18:32 +02:00
Adam Richie-Halford
9a9a1d8413
Decentralize cloudformation naming responsibilities (#3201)
* #3127 - Decentralize CF naming responsibilities

* Decentralize CloudFormation naming responsibilities

* Update URLs in cloudformation_resource_type functions

* Fix flake8 errors

* Black formatting

* Add a bunch of imports to populate CloudFormationModel.__subclasses__

* Add noqa to s3 models import statement in cloudformation/parsing.py

* Black formatting

* Remove debugging print statement

Co-authored-by: Bert Blommers <info@bertblommers.nl>
2020-08-01 15:23:36 +01:00
Bert Blommers
f2f25060fe #3150 - AWSLambda - Verify Docker is running 2020-07-20 11:37:45 +01:00
Guilherme Martins Crocetti
0dd41d4c32
Cloudformation support for EventSourceMapping (#3045)
* change line position for uuid and last_modified because they're not input parameters

* add event_source_arn validator and setter

* refactor batch_size as setter

* add helper function to parse arn and return source service

* fix for EventSource's create_from_cfn, there was no reference in the lambda object for the esm if created by cfn

* add esm deletion by cloudformation

* remove unused variable in test

* add cfn's update

* add complete implementation of delete_from_cfn

* blacked changed files

* fix test with invalid batchsize for sqs

* Dynamodb2 Table - Bugfix for localindex and implemented get_cfn_attributes

* Dynamodb2 eventsource - fix test to use StreamArn attribute

* Lambda Test - fix test_update_event_source_mapping
2020-06-14 16:03:00 +01:00
Bert Blommers
20784a2d67 Improve implementation coverage 2020-06-06 13:15:50 +01:00
Steve Pulec
464fea84f9
Merge pull request #2973 from bblommers/master
Lambda - Return LogResult only if requested
2020-05-14 17:53:49 -05:00
Anton Grübel
e73a694219
Add CloudWatch logs subscription filters (#2982)
* Add logs.describe_subscription_filters

* Add logs.put_subscription_filter

* Add logs.delete_subscription_filter

* Change to usage of ACCOUNT_ID
2020-05-12 13:34:10 +01:00
Bert Blommers
f82e834225 Lambda - Only return Logs if LogType=Tail 2020-05-07 10:55:15 +01:00
Bert Blommers
be5b1c592f Lambda - Add actual logs to LogResult, instead of replicating the response 2020-05-07 10:40:24 +01:00
Bert Blommers
9ab02e17d5 #883 - Lambda - Add test to verify remove_permission functinonality 2020-04-03 10:30:05 +01:00
Bert Blommers
038ff620b2 DDB Streams - Bugfix where processed items are resend every time 2020-02-24 09:28:52 +00:00
Bert Blommers
2f9b259cc1
Merge pull request #2701 from kislyuk/patch-2
awslambda: explicitly specify json-file log driver
2020-01-30 07:52:23 +00:00
Bert Blommers
da40c7125a
Merge pull request #2703 from brady-gsa/fix_lambda_config
Fixes awslambda policy management
2020-01-29 15:30:25 +00:00
Brady
2a2ff32dec improves support for AWS lambda policy management 2020-01-24 14:53:16 -05:00
Andrey Kislyuk
e3906043d7
Fix linter error 2020-01-22 16:58:25 -08:00
Andrey Kislyuk
5ba786d97a
awslambda: return actual result in sync invoke
Return actual output of the Lambda instead of echoing the input.
2020-01-22 16:39:11 -08:00
Andrey Kislyuk
ed81e36faf
awslambda: explicitly specify json-file log driver
This is analogous to #2635.
2020-01-22 16:08:42 -08:00
Bert Blommers
000cb968a4 #2623 - Only return response from lambda, skip log output 2020-01-05 11:36:51 +00:00
gruebel
9d9b620878 Fixed linter errors 2019-12-26 21:03:49 +01:00
gruebel
d7ba355a65 Add missing regions to all services 2019-12-26 17:12:22 +01:00
Fabio Dias
fd9ac4b9a5 ACCOUNT_ID moved to moto.core, MOTO_ACCOUNT_ID env var override 2019-12-16 21:05:29 -05:00
Fabio Dias
fb9ebe5a47 linting fixes 2019-12-16 19:56:11 -05:00
Fabio Dias
b83a750630 Added references to moto.iam.models.ACCOUNT_ID instead of hardcoded id 2019-12-16 19:56:11 -05:00
Mike Grima
76aaa2df0f
Merge pull request #2595 from bblommers/feature/2317
Add CF update/delete methods for Lambda
2019-12-09 14:04:49 -08:00
Bert Blommers
791c25b51c #2317 - Add CF Update/Delete methods for Lambda 2019-11-24 14:54:38 +00:00
Bert Blommers
0ea98f22ee Merge branch 'master' into feature/2546 2019-11-19 08:00:59 +00:00
Mike Grima
27f36a7514 Fixed the Lambda invocation due to lambci changes.
It looks like lambci is pre-pending the "lambda" responses with an
escape character `\033`. This was breaking the unit tests.
2019-11-17 18:40:15 -08:00
Bert Blommers
c6dd3c96ea AWSLambda - refactor exception handling 2019-11-17 10:59:20 +00:00
Bert Blommers
42311c40ad Add/improve Lambda coverage - no functionality changes 2019-11-15 16:34:14 +00:00
Bert Blommers
40aa73a12b #2546 - AWS Lambda: Add Role validation when creating functions 2019-11-07 17:11:13 +00:00
Ian Yon
9e2b47e82b Applied black formatter 2019-11-04 12:22:03 -03:00
Ian Yon
577419798d Merge from master to current branch 2019-11-04 12:14:43 -03:00