Commit Graph

314 Commits

Author SHA1 Message Date
FloatingGhost
3f8895a9a3
[s3] Allow KMS SSE parameters in copy_object (#4593) 2021-11-18 19:57:12 -01:00
Bert Blommers
a912fc4cac
Feature: Allow custom endpoints for S3 (#4562) 2021-11-17 20:02:14 -01:00
Jordan Sanders
7664cab828
Set version_id to null without bucket versioning (#4574) 2021-11-15 19:25:34 -01:00
Bert Blommers
60d9667da2
S3 - Fix NoSuchKey-format (#4572) 2021-11-14 16:16:58 -01:00
Bert Blommers
dfe01dad17
S3 - Remove double quotes on ETag after copy (#4561) 2021-11-11 10:51:09 -01:00
Bert Blommers
dfb380d887
S3 - PutObject cant specify ACL and Grant (#4550) 2021-11-09 21:49:37 -01:00
Bert Blommers
0bad68f9f0
S3 - create_multipart_upload - support tags (#4548) 2021-11-09 18:49:29 -01:00
Bert Blommers
eb1d127851
Feature: S3 - enable encryption when copying keys (#4535) 2021-11-06 22:12:01 -01:00
Vincent Barbaresi
33ad777f71
add back OPTIONS key response after implementing CORS in #4497 (#4528) 2021-11-05 12:19:06 -01:00
Bert Blommers
ccd216dfe5
S3 - get_object() should error for Glacier-class (#4523) 2021-11-02 22:02:14 -01:00
Brandon Bodnar
6264fb292c
fix: S3 CopyObjects with embedded percent encoding (#4514) 2021-11-01 21:17:06 -01:00
Vincent Barbaresi
03c170e206
S3: Implement CORS headers in OPTIONS requests (#4497) 2021-10-30 10:02:30 +00:00
Vincent Barbaresi
ecdd395bec
Fix range headers: use response_headers parameters instead of creating a new dict (#4502) 2021-10-30 09:49:42 +00:00
Bert Blommers
e388a25141
S3 - Return default content type (#4439) 2021-10-19 16:46:23 +00:00
Bert Blommers
14a69c7524
Techdebt: Enable pylint rules (#4432) 2021-10-18 19:44:29 +00:00
Bert Blommers
135edda994
S3 - get/put/delete replication config (#4421) 2021-10-16 17:26:09 +00:00
Shreesha Addala
c62bd5ca41
Enforce tagging restrictions for S3 object (#4417) 2021-10-15 19:10:28 +00:00
Bert Blommers
d916fd636f
S3:list_object_versions() - Implement Delimiter + KeyMarker (#4413) 2021-10-14 18:13:40 +00:00
Bert Blommers
df1732c737
S3 - list_objects(MaxKeys=0) (#4411) 2021-10-14 09:49:50 +00:00
Bert Blommers
52df393b5a
S3 - Move ACL tests to dedicated file (#4397) 2021-10-12 17:50:36 +00:00
Bert Blommers
773e9a9f79
S3 - Allow for keyname that is just an empty space (#4244) 2021-10-09 10:12:26 +00:00
Bert Blommers
c3503081bc
S3: HeadBucket requests should be authenticated (#4346) 2021-09-25 17:25:27 +00:00
Bert Blommers
6dfb539a97
Rewrite deprecated S3 tests (#3889) 2021-09-21 20:06:42 +00:00
Bert Blommers
c53183db70
S3 - Correct error when creating a bucket that already exists (#4257) 2021-09-01 16:30:01 +01:00
Bert Blommers
dc49232734
S3 - HeadObject should authenticate requests (#4240) 2021-08-29 14:49:05 +01:00
Bert Blommers
532386327d
S3 - Add RequestId to responses (#3836) 2021-08-28 08:32:14 +01:00
Bert Blommers
0317c502f0
S3 ListParts API: use MaxParts parameter (#3658) 2021-08-28 07:38:16 +01:00
Bert Blommers
684cafa2b8
S3 - Allow uploads using a PUT request without content-type (#3699) 2021-08-28 06:10:16 +01:00
Bert Blommers
8d4596a366
Prep Release 2.2.4 (#4205)
* S3 - Refactor logic to models.py, out of Response-class

* Changelog for release 2.2.4
2021-08-21 15:05:40 +01:00
Łukasz
73368863eb
Added handling for V4 signatures in PutObject body (#4201)
Co-authored-by: Łukasz Nowak <lukasz.nowak@idemia.com>
2021-08-19 15:06:43 +01:00
Hasanul Islam
492484be20
Add missing default regiono in s3 client (#4194) 2021-08-18 20:25:33 +01:00
Cristopher Pinzón
1d90946072
Feature S3 Object Lock (#4174) 2021-08-17 06:16:59 +01:00
Bert Blommers
79f0cc9e9e
Tech debt - remove dead DynamoDB code and add RDS tests (#4136) 2021-08-04 17:24:26 +01:00
Bert Blommers
11883a1fda
Remove unnecessary dependencies EC2/SQS (#4094)
* Add missing dependencies for EFS
2021-07-29 06:38:16 +01:00
Waldemar Hummer
f4f8527955
Merge LocalStack changes into upstream moto (#4082)
* fix OPTIONS requests on non-existing API GW integrations

* add cloudformation models for API Gateway deployments

* bump version

* add backdoor to return CloudWatch metrics

* Updating implementation coverage

* Updating implementation coverage

* add cloudformation models for API Gateway deployments

* Updating implementation coverage

* Updating implementation coverage

* Implemented get-caller-identity returning real data depending on the access key used.

* bump version

* minor fixes

* fix Number data_type for SQS message attribute

* fix handling of encoding errors

* bump version

* make CF stack queryable before starting to initialize its resources

* bump version

* fix integration_method for API GW method integrations

* fix undefined status in CF FakeStack

* Fix apigateway issues with terraform v0.12.21
* resource_methods -> add handle for "DELETE" method
* integrations -> fix issue that "httpMethod" wasn't included in body request (this value was set as the value from refer method resource)

* bump version

* Fix setting http method for API gateway integrations (#6)

* bump version

* remove duplicate methods

* add storage class to S3 Key when completing multipart upload (#7)

* fix SQS performance issues; bump version

* add pagination to SecretsManager list-secrets (#9)

* fix default parameter groups in RDS

* fix adding S3 metadata headers with names containing dots (#13)

* Updating implementation coverage

* Updating implementation coverage

* add cloudformation models for API Gateway deployments

* Updating implementation coverage

* Updating implementation coverage

* Implemented get-caller-identity returning real data depending on the access key used.

* make CF stack queryable before starting to initialize its resources

* bump version

* remove duplicate methods

* fix adding S3 metadata headers with names containing dots (#13)

* Update amis.json to support EKS AMI mocks (#15)

* fix PascalCase for boolean value in ListMultipartUploads response (#17); fix _get_multi_param to parse nested list/dict query params

* determine non-zero container exit code in Batch API

* support filtering by dimensions in CW get_metric_statistics

* fix storing attributes for ELBv2 Route entities; API GW refactorings for TF tests

* add missing fields for API GW resources

* fix error messages for Route53 (TF-compat)

* various fixes for IAM resources (tf-compat)

* minor fixes for API GW models (tf-compat)

* minor fixes for API GW responses (tf-compat)

* add s3 exception for bucket notification filter rule validation

* change the way RESTErrors generate the response body and content-type header

* fix lint errors and disable "black" syntax enforcement

* remove return type hint in RESTError.get_body

* add RESTError XML template for IAM exceptions

* add support for API GW minimumCompressionSize

* fix casing getting PrivateDnsEnabled API GW attribute

* minor fixes for error responses

* fix escaping special chars for IAM role descriptions (tf-compat)

* minor fixes and tagging support for API GW and ELB v2 (tf-compat)

* Merge branch 'master' into localstack

* add "AlarmRule" attribute to enable support for composite CloudWatch metrics

* fix recursive parsing of complex/nested query params

* bump version

* add API to delete S3 website configurations (#18)

* use dict copy to allow parallelism and avoid concurrent modification exceptions in S3

* fix precondition check for etags in S3 (#19)

* minor fix for user filtering in Cognito

* fix API Gateway error response; avoid returning empty response templates (tf-compat)

* support tags and tracingEnabled attribute for API GW stages

* fix boolean value in S3 encryption response (#20)

* fix connection arn structure

* fix api destination arn structure

* black format

* release 2.0.3.37

* fix s3 exception tests

see botocore/parsers.py:1002 where RequestId is removed from parsed

* remove python 2 from build action

* add test failure annotations in build action

* fix events test arn comparisons

* fix s3 encryption response test

* return default value "0" if EC2 availableIpAddressCount is empty

* fix extracting SecurityGroupIds for EC2 VPC endpoints

* support deleting/updating API Gateway DomainNames

* fix(events): Return empty string instead of null when no pattern is specified in EventPattern (tf-compat) (#22)

* fix logic and revert CF changes to get tests running again (#21)

* add support for EC2 customer gateway API (#25)

* add support for EC2 Transit Gateway APIs (#24)

* feat(logs): add `kmsKeyId` into `LogGroup` entity (#23)

* minor change in ELBv2 logic to fix tests

* feat(events): add APIs to describe and delete CloudWatch Events connections (#26)

* add support for EC2 transit gateway route tables (#27)

* pass transit gateway route table ID in Describe API, minor refactoring (#29)

* add support for EC2 Transit Gateway Routes (#28)

* fix region on ACM certificate import (#31)

* add support for EC2 transit gateway attachments (#30)

* add support for EC2 Transit Gateway VPN attachments (#32)

* fix account ID for logs API

* add support for DeleteOrganization API

* feat(events): store raw filter representation for CloudWatch events patterns (tf-compat) (#36)

* feat(events): add support to describe/update/delete CloudWatch API destinations (#35)

* add Cognito UpdateIdentityPool, CW Logs PutResourcePolicy

* feat(events): add support for tags in EventBus API (#38)

* fix parameter validation for Batch compute environments (tf-compat)

* revert merge conflicts in IMPLEMENTATION_COVERAGE.md

* format code using black

* restore original README; re-enable and fix CloudFormation tests

* restore tests and old logic for CF stack parameters from SSM

* parameterize RequestId/RequestID in response messages and revert related test changes

* undo LocalStack-specific adaptations

* minor fix

* Update CodeCov config to reflect removal of Py2

* undo change related to CW metric filtering; add additional test for CW metric statistics with dimensions

* Terraform - Extend whitelist of running tests

Co-authored-by: acsbendi <acsbendi28@gmail.com>
Co-authored-by: Phan Duong <duongpv@outlook.com>
Co-authored-by: Thomas Rausch <thomas@thrau.at>
Co-authored-by: Macwan Nevil <macnev2013@gmail.com>
Co-authored-by: Dominik Schubert <dominik.schubert91@gmail.com>
Co-authored-by: Gonzalo Saad <saad.gonzalo.ale@gmail.com>
Co-authored-by: Mohit Alonja <monty16597@users.noreply.github.com>
Co-authored-by: Miguel Gagliardo <migag9@gmail.com>
Co-authored-by: Bert Blommers <info@bertblommers.nl>
2021-07-26 15:21:17 +01:00
Bert Blommers
7693d77333
Remove Py2 support (#3915) 2021-07-26 07:40:39 +01:00
benediktbrandt
eb6515cf50
Reduce default value for DEFAULT_KEY_BUFFER_SIZE (#4003)
* - introduce environment variable for DEFAULT_KEY_BUFFER_SIZE

* - prefix env variable with MOTO_S3 to avoid env variable conflicts

* - reduce the DEFAULT_KEY_BUFFER_SIZE to be less than the S3_UPLOAD_PART_MIN_SIZE to prevent in memory caching of multi part uploads

* - black formatting

* - fix formatting

* - fix missing import
2021-07-07 16:38:50 +01:00
Bert Blommers
a230e2a7e3
#3113 - S3#list_object_versions - verify prefix argument (#4041) 2021-06-28 16:22:08 +01:00
Bert Blommers
059c36b694
#3593 - S3 - Return appropriate error when provided with invalid multipart ID (#4040) 2021-06-28 15:57:07 +01:00
Bert Blommers
0f4ab4da1c
S3 - Dont error when deleting unknown keys (#4036)
* #1581 - S3 - Dont error when deleting unknown keys

* Update test_s3bucket_path.py
2021-06-25 10:26:03 +01:00
Bert Blommers
4778377e8e
#4018 - Raise NoSuchUpload when listing a non-existing multipart upload (#4031) 2021-06-24 08:14:28 +01:00
benediktbrandt
479ce861a7
Env variable for default key buffer size (#4001)
* - introduce environment variable for DEFAULT_KEY_BUFFER_SIZE

* - prefix env variable with MOTO_S3 to avoid env variable conflicts

* - black formatting

* - fix formatting
2021-06-10 10:48:28 +01:00
Mike Grima
ba7a0fe3a2 Fixed AWS Config S3 ACL issues
- AWS Config appears to have changed how ACLs are returned
- This PR makes moto's behavior consistent with AWS Config's S3 representation

Fixes #3986
2021-06-04 09:23:56 -07:00
Florian Dambrine
6adb570c48
Add dot as a valid s3 metadata key (#3958) 2021-05-26 14:27:25 +01:00
amar jandu
58381cce8f
S3 - Fix exception for missing versionID (#3887) 2021-04-30 12:36:08 +01:00
Codeglitches
9f9716ee01
Check S3_IGNORE_SUBDOMAIN_BUCKETNAME environment variable (#3796)
* Check S3_IGNORE_SUBDOMAIN_BUCKETNAME environment variable

* move S3_IGNORE_SUBDOMAIN_BUCKETNAME environment variable to settings
2021-03-26 16:51:19 +00:00
Tibor Djurica Potpara
b06e77b604
Make put-object-acl return 404 if the key does not exist (#3777)
Co-authored-by: Tibor Djurica Potpara <tibor.djurica@reddit.com>
2021-03-16 22:24:41 +00:00
Bert Blommers
d7b8419791
Fix Mock 4.0.3 compatibility - Unpatch only once during teardown (#3541)
* #3535 - Unpatch only once during teardown

* EnvVar patching - Fix Python2 bug

* Allow latest version of mock-library
2021-02-15 08:42:16 +00:00
wynnw
891118a7c7
#3667: Update the s3 post functionality to better support success_action_redirect (#3668)
* Update the s3 post functionality to better support success_action_redirect

- Add the bucket/key values to the redirect url like s3 does, which
supports code that relies on the key value being there on the
redirect.
- Add support for replacing ${filename} in the key value with the actual
filename from the form upload.

See Issue #3667

* Update s3 tests for changed success_action_redirect behavior

- Adds a new test called test_s3_server_post_to_bucket_redirect that
tests both the ${filename} replacement and the key/value addition to the
redirect query args
- Updated the test_creating_presigned_post checks to handle the
key/value additions to the redirect url.

* Fix test updates to work with python2.7

- remove f-string usage
- fix urllib.parse imports to use six

Co-authored-by: Wynn Wilkes <wynn@leading2lean.com>
2021-02-10 09:06:03 +00:00
Rahul Ranjan
7d066cea2f
Add versionid header in complete multipart if enabled (#3643)
* Add versionid header in complete multipart if enabled
- addresses https://github.com/localstack/localstack/issues/3514

* remove unnecessary code

* fix lint

* fix lint test

* add test case for boto3
2021-02-02 09:51:17 +00:00