331 Commits

Author SHA1 Message Date
Diego Argueta
f15f006f78 Hack around text problem in unit tests.
Now that payloads are not allowed to be text, some unit tests will cause crashes
on Python 3 because the payload sent by requests gets passed to FakeKey as a
string instead of raw bytes.

I haven't been able to figure out a way around the issue that doesn't get super
messy inside s3/responses.py so I'm just converting the value to bytes using the
system's default encoding.
2018-12-20 00:34:39 -08:00
Diego Argueta
b0a280bde2 Move S3 storage to SpooledTemporaryFile 2018-12-18 14:20:48 -08:00
Taro Sato
2038fa92be Activate proper pagination for S3 common prefixes 2018-12-05 16:17:28 -08:00
Jon Michaelchuck
d29869bf9b flake8 fix 2018-11-27 08:32:30 -08:00
Jon Michaelchuck
009b02bcd5 Raise a client error if PutBucketTags request contains duplicate keys
A PutBucketTags request with duplicate keys will raise a ClientError
with code InvalidTag and message 'Cannot provide multiple Tags with the
same key'.
2018-11-26 15:56:46 -08:00
Jon Michaelchuck
69e093fcea flake8 indentation fix 2018-11-22 06:16:37 -06:00
Jon Michaelchuck
fb7e52becc Check bucket name length at CreateBucket
Check that s3 bucket names follow the documented length restriction:

'Bucket names must be at least 3 and no more than 63 characters long.'

See https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
2018-11-22 06:08:03 -06:00
Grégory Bataille
34ac5c72b9 feat(#1959): bucket in us-east-1 return None as LocationConstraint 2018-11-20 19:50:42 +01:00
Grégory Bataille
437eb892e2 feat(#1959): LocationConstraint us-east-1 is not accepted by the CreateBucket operation 2018-11-20 18:57:05 +01:00
Jordan Guymon
8e909f580a MockAWS implementation using botocore event hooks 2018-10-29 09:12:22 +01:00
Steve Pulec
27ca96519b Fix extra whitespace in s3. Closes #1844. 2018-10-15 01:37:38 -04:00
Jon Beilke
5b3b52752d explicitly check that lifecycle actions are not None when setting lifecycle 2018-10-04 10:25:16 -05:00
Jon Beilke
a1a8ac7286 check for None in lifecycle actions 2018-10-03 01:26:09 -05:00
Jon Beilke
9b5f983cb5 add action validation to set_lifecycle() 2018-10-03 01:11:11 -05:00
Jon Beilke
691a8722a8 formatting fix for flake8 due to extra spaces 2018-10-03 00:45:47 -05:00
Jon Beilke
ea4fcaa82a add support for NoncurrentVersionTransition, NoncurrentVersionExpiration, and AbortIncompleteMultipartUpload actions to s3 lifecycle rules 2018-10-03 00:40:28 -05:00
Jordan Upiter
756b5d6671 Add support for multiple delete markers on an s3 object 2018-09-11 10:41:01 -04:00
Steve Pulec
0309d1ec39
Merge pull request #1707 from Whamied/etag-no-quote-patch
Added support for multipart upload completion with unquoted etags
2018-07-19 10:05:07 -04:00
Waleed Hamied
f8fdd439ad Added support for multipart upload confirmation with unquoted etags 2018-07-03 15:36:41 -04:00
Sanjeev Suresh
29da006f78 changed the getList default to an empty list instead of None, because otherwise an exception is raised when trying to iterate over an empty list 2018-06-21 15:26:27 -07:00
Alec Rajeev
6c10dc0403 Fix Storage Class error handling (#1653)
Added check for valid storage classes in set_key and copy_key
added unit test for standard storage and infrequent access
2018-06-07 08:09:36 +01:00
Alexander Mohr
93a404ec37 pep 2018-05-03 02:10:17 -07:00
Alexander Mohr
7a57dc2034 fix errors 2018-05-03 01:40:49 -07:00
Steve Pulec
2ecb04d6e0 Revert errant change to S3 urls in cd1c6d3e6c3c80bf55fbfd6a657ba642c6fb4e3f. 2018-04-18 16:15:47 -04:00
Steve Pulec
cd1c6d3e6c Unvendor responses, move back to upstream. 2018-04-05 16:57:43 -04:00
Mike Grima
4184acc0d2 Added Filtering support for S3 lifecycle (#1535)
* Added Filtering support for S3 lifecycle

Also added `ExpiredObjectDeleteMarker`.

closes #1533
closes #1479

* Result set no longer contains "Prefix" if "Filter" is set.
2018-04-02 14:19:14 -07:00
Colin Jones
1a8a4a084d S3 Ignore Subdomain for Bucketname Flag (#1419)
* Some circumstances need subdomains to be ignored rather that interpreted as bucketname, this patch allows such behaviour to be configured

* Adding helper case whereby localstack features as path based exception

* Remove whitespace :(
2018-03-21 09:33:09 -07:00
Mike Grima
cc14114afe Implemented S3 get/put_notification_configuration (#1516)
closes #973
2018-03-21 09:11:24 -07:00
Steve Pulec
0414cc51c1
Merge pull request #1499 from srikiraju/fix/s3-list-deleted
S3: Do not attempt to return deleted files in bucket listing
2018-03-07 07:08:07 -05:00
Steve Pulec
ece7507eec
Merge pull request #1430 from OwnZones/urldecode_fix
Url decode x-amz-copy-source for S3 copyObject requests
2018-03-06 22:26:15 -05:00
Steve Pulec
326698f3fa
Merge pull request #1460 from okomestudio/s3-region-from-url
Support both virtual-hosted and path-style URLs for region name parsing
2018-03-06 21:57:57 -05:00
Srikanth Raju
4f05aa725c S3: Do not attempt to return deleted files in bucket listing 2018-03-06 01:55:14 -08:00
Taro Sato
5e70d0ce4c Support both virtual-hosted-style and path-style URLs for region name parsing 2018-01-30 16:10:43 -08:00
Taro Sato
c7bcbadc6e Fix the S3 HEAD response body 2018-01-30 13:48:04 -08:00
ciprianaradulescu
c431a3a774
Merge branch 'master' into urldecode_fix 2018-01-14 19:58:48 +02:00
Ciprian Radulescu
ef9b229acc url decode x-amz-copy-source as per s3 nodejs documentation 2018-01-14 18:49:47 +02:00
Mike Grima
770281aef2 Added put_bucket_logging support (#1401)
- Also added put acl for XML
- Put logging will also verify that the destination bucket exists in the same region with the proper ACLs attached.
2018-01-02 20:47:57 -08:00
Waldemar Hummer
cdb7305dac use S3 path style addressing when running in kubernetes (#1315) 2017-11-15 19:37:39 +00:00
Terry Cain
d447929d29 utf 8 in key deletions V2 (#1321)
* supporting utf-8 in key deletions

* Fixed decoding of version body when regexing

* Fixed some more random errors

* Possible fix

* Fixed unused import

* Added UTF comment Py2
2017-11-06 13:39:08 -08:00
Brian Pandola
68a1e41286 Fix #1318 2017-11-02 23:03:54 -07:00
Terry Cain
b40c5e557e Fixed S3 versioning bug + minor cleanup (#1272)
* Fixed S3 versioning bug + minor cleanup

Fixes 1271

* flake8
2017-10-18 16:22:35 -07:00
Alexander Mohr
9008b85299 lambda + SNS enhancements (#1048)
* updates

- support lambda messages from SNS
- run lambda in docker container

* decode output

* populate timeout

* simplify

* whoops

* skeletons of cloudwatchlogs

* impl filter log streams

* fix logging

* PEP fixes

* PEP fixes

* fix reset

* fix reset

* add new endpoint

* fix region name

* add docker

* try to fix tests

* try to fix travis issue with boto

* fix escaping in urls

* fix environment variables

* fix PEP

* more pep

* switch back to precise

* another fix attempt

* fix typo

* fix lambda invoke

* fix more unittests

* work on getting this to work in new scheme

* fix py2

* fix error

* fix tests when running in server mode

* more lambda fixes

* try running with latest docker

adapted from aiodocker

* switch to docker python client

* pep fixes

* switch to docker volume

* fix unittest

* fix invoke from sns

* fix zip2tar

* add hack impl for get_function with zip

* try fix

* fix for py < 3.6

* add volume refcount

* try to fix travis

* docker test

* fix yaml

* try fix

* update endpoints

* fix

* another attempt

* try again

* fix recursive import

* refactor fix

* revert changes with better fix

* more reverts

* wait for service to come up

* add back detached mode

* sleep and add another exception type

* put this back for logging

* put back with note

* whoops :)

* docker in docker!

* fix invalid url

* hopefully last fix!

* fix lambda regions

* fix protocol

* travis!!!!

* just run lambda test for now

* use one print

* fix escaping

* another attempt

* yet another

* re-enable all tests

* fixes

* fix for py2

* revert change

* fix for py2.7

* fix output ordering

* remove this given there's a new unittest that covers it

* changes based on review

- add skeleton logs test file
- switch to docker image that matches test env
- fix mock_logs import

* add readme entry
2017-09-27 16:04:58 -07:00
Mike Grima
7167e09ca5 Fixed bug in S3 put_bucket_tagging. Closes #1181. 2017-09-22 13:35:00 -07:00
Jack Danger
390fe85137 supporting httpretty requests 2017-09-22 10:44:55 -07:00
Jack Danger
ea66a71765 supporting signed urls for private keys 2017-09-22 10:44:37 -07:00
Jack Danger
ca7661bc1e S3 keys inherit bucket ACL 2017-09-20 19:04:23 -07:00
Jack Danger
2055bb62f5 enforce s3 acls 2017-09-16 12:49:07 -07:00
Julien Duchesne
83dd9559db Handle "max-keys" in list-objects 2017-09-10 10:42:17 -07:00
Julien Duchesne
14dec68f15 Remove superfluous space 2017-09-09 00:27:54 -04:00
Julien Duchesne
0ddbcb5464 Handle 'marker' paginating parameter in the list-objects query 2017-09-09 00:25:02 -04:00