Commit Graph

266 Commits

Author SHA1 Message Date
Steve Pulec
e681f55ba1
Merge branch 'master' into bucket-name-length-limit 2018-12-28 21:04:10 -05:00
Steve Pulec
65364a8e71
Merge pull request #1961 from gbataille/fix_1959_location_constraint_us_east_1
Fix 1959 location constraint us east 1
2018-12-28 21:00:29 -05:00
Diego Argueta
1998d59cfc Add more tests to please Coveralls 2018-12-20 11:40:13 -08:00
Diego Argueta
191ad6d778 Make keys pickleable 2018-12-20 11:15:15 -08:00
Diego Argueta
2cc8784e5c Restore files modified in non-working fix. 2018-12-18 14:53:52 -08:00
Diego Argueta
b0a280bde2 Move S3 storage to SpooledTemporaryFile 2018-12-18 14:20:48 -08:00
Taro Sato
2038fa92be Activate proper pagination for S3 common prefixes 2018-12-05 16:17:28 -08:00
Jon Michaelchuck
009b02bcd5 Raise a client error if PutBucketTags request contains duplicate keys
A PutBucketTags request with duplicate keys will raise a ClientError
with code InvalidTag and message 'Cannot provide multiple Tags with the
same key'.
2018-11-26 15:56:46 -08:00
Jon Michaelchuck
fb7e52becc Check bucket name length at CreateBucket
Check that s3 bucket names follow the documented length restriction:

'Bucket names must be at least 3 and no more than 63 characters long.'

See https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
2018-11-22 06:08:03 -06:00
Grégory Bataille
b0eb7b263e test(#1959): us-east-1 located bucket should return a None LocationConstraint 2018-11-20 19:42:51 +01:00
Grégory Bataille
9291ff533a test(#1959): LocationConstraint us-east-1 should be refused 2018-11-20 18:56:56 +01:00
Jon Beilke
12e0a38b56 add TODO for testing exceptions with aimu and nve 2018-10-05 11:04:55 -05:00
Jon Beilke
9b5f983cb5 add action validation to set_lifecycle() 2018-10-03 01:11:11 -05:00
Jon Beilke
ea4fcaa82a add support for NoncurrentVersionTransition, NoncurrentVersionExpiration, and AbortIncompleteMultipartUpload actions to s3 lifecycle rules 2018-10-03 00:40:28 -05:00
Jordan Upiter
756b5d6671 Add support for multiple delete markers on an s3 object 2018-09-11 10:41:01 -04:00
TheDooner64
e67a8c6f1b Revert minor changes to s3 tests 2018-08-05 19:18:11 -04:00
TheDooner64
77f0a61c9f Add scaffolding for Glue service, including create_database and get_database for the Glue Data Catalog 2018-08-05 19:18:11 -04:00
Steve Pulec
0309d1ec39
Merge pull request #1707 from Whamied/etag-no-quote-patch
Added support for multipart upload completion with unquoted etags
2018-07-19 10:05:07 -04:00
Waleed Hamied
f8fdd439ad Added support for multipart upload confirmation with unquoted etags 2018-07-03 15:36:41 -04:00
Sanjeev Suresh
2ebd5f7359 tests for prefixes that return empty result sets 2018-06-22 11:59:01 -07:00
Alec Rajeev
6c10dc0403 Fix Storage Class error handling (#1653)
Added check for valid storage classes in set_key and copy_key
added unit test for standard storage and infrequent access
2018-06-07 08:09:36 +01:00
Alexander Mohr
07540a35fe add unittest 2018-05-03 02:30:29 -07:00
Mike Grima
4184acc0d2 Added Filtering support for S3 lifecycle (#1535)
* Added Filtering support for S3 lifecycle

Also added `ExpiredObjectDeleteMarker`.

closes #1533
closes #1479

* Result set no longer contains "Prefix" if "Filter" is set.
2018-04-02 14:19:14 -07:00
Colin Jones
1a8a4a084d S3 Ignore Subdomain for Bucketname Flag (#1419)
* Some circumstances need subdomains to be ignored rather that interpreted as bucketname, this patch allows such behaviour to be configured

* Adding helper case whereby localstack features as path based exception

* Remove whitespace :(
2018-03-21 09:33:09 -07:00
Mike Grima
cc14114afe Implemented S3 get/put_notification_configuration (#1516)
closes #973
2018-03-21 09:11:24 -07:00
Steve Pulec
0414cc51c1
Merge pull request #1499 from srikiraju/fix/s3-list-deleted
S3: Do not attempt to return deleted files in bucket listing
2018-03-07 07:08:07 -05:00
Srikanth Raju
4f05aa725c S3: Do not attempt to return deleted files in bucket listing 2018-03-06 01:55:14 -08:00
Taro Sato
5e70d0ce4c Support both virtual-hosted-style and path-style URLs for region name parsing 2018-01-30 16:10:43 -08:00
Mike Grima
770281aef2 Added put_bucket_logging support (#1401)
- Also added put acl for XML
- Put logging will also verify that the destination bucket exists in the same region with the proper ACLs attached.
2018-01-02 20:47:57 -08:00
Terry Cain
d447929d29 utf 8 in key deletions V2 (#1321)
* supporting utf-8 in key deletions

* Fixed decoding of version body when regexing

* Fixed some more random errors

* Possible fix

* Fixed unused import

* Added UTF comment Py2
2017-11-06 13:39:08 -08:00
Brian Pandola
68a1e41286 Fix #1318 2017-11-02 23:03:54 -07:00
Terry Cain
b40c5e557e Fixed S3 versioning bug + minor cleanup (#1272)
* Fixed S3 versioning bug + minor cleanup

Fixes 1271

* flake8
2017-10-18 16:22:35 -07:00
Mike Grima
7167e09ca5 Fixed bug in S3 put_bucket_tagging. Closes #1181. 2017-09-22 13:35:00 -07:00
Jack Danger
eeda0cd28e re-enabling tests 2017-09-22 10:57:06 -07:00
Jack Danger
ea66a71765 supporting signed urls for private keys 2017-09-22 10:44:37 -07:00
Jack Danger
ca7661bc1e S3 keys inherit bucket ACL 2017-09-20 19:04:23 -07:00
Jack Danger
c8f6fb7738 Creating server-safe anonymous clients for testing 2017-09-16 15:48:20 -07:00
Jack Danger
e33702fbac using deprecated mock just to patch requests library 2017-09-16 12:49:07 -07:00
Jack Danger
802279d7c4 Authenticating to S3 in tests 2017-09-16 12:49:07 -07:00
Jack Danger
2055bb62f5 enforce s3 acls 2017-09-16 12:49:07 -07:00
Mike Grima
5d8cd22b01 Fixed S3 lifecycle error message. (#1110)
Fixes #1109
Also added PutBucketTagging support
Also added Bucket CORS support
2017-09-07 11:30:05 -07:00
Alexander Mohr
2447725e98 fix list-object-versions with prefix (#1045)
fix list-object-versions with prefix
2017-08-10 17:14:31 -07:00
Waldemar Hummer
24d1562d2f allow non-ascii characters in request URLs 2017-08-05 21:03:50 +10:00
Jack Danger
3eef3c23b1 Updating examples in README to latest API 2017-07-26 22:57:55 -07:00
Taro Sato
38fa6809c0 Make HEAD bucket throw ClientError instead of NoSuchBucket on boto3 2017-07-20 15:39:45 -07:00
eric-weaver
abf3078c28 implement s3 object tagging 2017-07-15 22:36:12 -04:00
Steve Pulec
a956c3a85c Fix tests for py3. 2017-06-03 19:35:23 -04:00
Steve Pulec
113bfcb4ea Fix duplicate bucket creation with LocationConstraint. Closes #970. 2017-06-03 19:29:59 -04:00
Steve Pulec
49c947ece7 Stop autodecoding content so we can mimic requests. Closes #963. 2017-06-03 19:06:55 -04:00
Jack Danger
6163363c15 Merge pull request #960 from spg/feat-s3-if-modified-since
feat(s3) HeadObject: honor If-Modified-Since header
2017-06-01 09:10:20 -07:00
Simon-Pierre Gingras
a2a6514936 attempt at fixing tests 2017-05-23 11:29:01 -07:00
Kate Heddleston
a21413f4ea NoSuchKey error in S3 is actually '404'
Fixes #571 and #953
2017-05-22 19:08:12 -07:00
Simon-Pierre Gingras
517416c4d9 feat(s3) HeadObject: honor If-Modified-Since header 2017-05-19 15:59:25 -07:00
georgepsarakis
ac8b8c9a36 Add tests for DeleteMarker support 2017-05-14 20:00:26 +03:00
Steve Pulec
152d71e4f3 Merge pull request #922 from georgepsarakis/bug/s3_list_versions_valueerror
ValueError when calling boto3 list_object_versions
2017-05-10 22:06:26 -04:00
Steve Pulec
ff4649eebd Merge pull request #924 from JackDanger/require-content-length-header
Require content-length header
2017-05-10 22:02:21 -04:00
Jack Danger Canty
a2fd72d2f8 Require content-length header
fixes #908
2017-05-01 12:13:12 -07:00
georgepsarakis
2714fb76f1 Python 2/3 compatibility fixes 2017-04-30 08:03:46 +03:00
georgepsarakis
819a308e2b Add failing test for S3 list_object_versions 2017-04-29 21:56:48 +03:00
Hugo Picado
0945765537
Fixing metadata key on s3 operation response
ETag metadata key is being returned as "Etag" instead of "ETag". This leads to issues in some AWS SDKs using MotoServer. This change fixes the issue by updating the key to the correct format.

This closes #920
2017-04-27 13:57:18 +01:00
Dmytro Milinevskyy
6e61ee4caa s3: handle WebsiteRedirectLocation
spulec/moto#821
2017-04-14 13:32:52 +02:00
Steve Pulec
3c0c4c2996 Fix tests for py3. 2017-02-23 22:28:09 -05:00
Steve Pulec
f37bad0e00 Lints. 2017-02-23 21:37:43 -05:00
Steve Pulec
1433f28846 Update s3 test. 2017-02-23 19:50:34 -05:00
Steve Pulec
81836b6981 Get standalone server mode working for all tests. 2017-02-23 19:43:48 -05:00
Steve Pulec
fde721bed7 Testing new version of decorator. 2017-02-15 22:35:45 -05:00
Stefan Nordhausen
1a01bae74e Implement list_objects_v2() for S3 buckets (#814)
This adds/fixes the following things:
- Add missing KeyCount in result (fixes #734).
- Do not hard code MaxKeys to 1000.
- Truncate result if it has more than MaxKeys items. Set IsTruncated
    and NextContinuationToken accordingly.
- Support the StartAfter parameter.
- Return Owner information only when FetchOwner=True is given.
- "Prefix" in response is now "" instead of None when omitted in request.
- "Delimiter" is now omitted from response when not given in request.
2017-02-08 21:21:43 -05:00
Tom V
c54985a39f Fix for #748. Turn on autoescape for S3 templates. (#779) 2016-12-03 18:13:24 -05:00
Steve Pulec
38021c6ef5 Test Boto 2.43.0 . 2016-11-06 10:29:50 -05:00
Taro Sato
bdb9d59e46 Update response headers with response_dict from key; add tests 2016-09-20 15:42:21 -07:00
Steve Pulec
fca0d3885b Merge pull request #698 from SpoonMeiser/fix-trailing-slash
Stop stripping the trailing slash off keys listed from s3 buckets
2016-09-13 21:57:03 -04:00
Oliver Jeeves
b297b1410c Stop stripping the trailing slash off keys listed from s3 buckets
fixes #684
2016-09-13 12:21:22 +01:00
imran
478e791c39 Fix unicode S3 key listing in Python 2 2016-09-04 04:31:03 +06:00
Jesse Szwedko
ef10df0636 Update format of S3 client errors
To match
http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html#RESTErrorResponses
which documents that it should be at the top level rather than nested
under `Errors`.
2016-07-08 19:32:34 +00:00
Steve Pulec
bd7327fca2 Fix issues around S3 deletion. Closes #606. 2016-05-08 17:14:35 -04:00
Steve Pulec
79addb8926 Add S3 functionality to copy range. Closes #526. 2016-05-05 22:52:12 -04:00
Steve Pulec
0a10b2f738 Fix issues with unicode and slashes for S3. Closes #521. 2016-05-01 22:03:52 -04:00
Dan Felix
ebb00a3bfe Test key head version_id support using boto3's head_object. 2016-04-28 13:17:33 -07:00
George Leslie-Waksman
0ed7d1bb71 Add support for src_version_id to bucket copy_key op. 2016-04-27 17:08:44 -07:00
Steve Pulec
0df03ba409 cleanup code. 2015-11-27 14:43:03 -05:00
Steve Pulec
c38731ecbb Add ACL support for S3 buckets. 2015-11-11 20:26:29 -05:00
Steve Pulec
18d63a6cfe Add basics of S# website configuration. Closes #442. 2015-11-04 18:55:41 -05:00
Mike Kaplinskiy
131d44f0ad Add S3 ACL supprt. 2015-10-07 00:14:37 -07:00
Adam DePue
a6c6edbca4 Add more boto3 s3 tests 2015-08-13 21:18:25 +00:00
Steve Pulec
323f720cb5 Fix S3 bucket policy delete. Closes #396. 2015-08-07 22:43:24 -04:00
Steve Pulec
dd092fa9dc Fix s3 py3 test for unicode. 2015-08-02 09:59:16 -04:00
Steve Pulec
ff8dd7530d Add boto3 s3 test. 2015-08-02 09:54:23 -04:00
Simon-Pierre Gingras
dc0557205d S3: Bucket policy 2015-07-23 17:33:52 -04:00
MurphyMarkW
5b9c874a23 Ensures all tests compare using bytes / bytearrays in 3.x. 2015-06-30 13:02:38 -05:00
MurphyMarkW
0fcbdd52e0 Adds / moves around tests for different ranges. 2015-06-30 02:08:45 -05:00
MurphyMarkW
10c3a37613 Adds test for explicit last-byte-pos > length. 2015-06-30 00:27:10 -05:00
MurphyMarkW
f64d0b99ac Tests for last-byte-pos behavior. 2015-06-29 20:26:00 -05:00
Steve Pulec
5ee1525cd8 Fix S3 missing key error. Closes #364. 2015-06-27 17:56:37 -04:00
Steve Pulec
b296294086 Add missing test from previous commit. 2015-06-02 23:13:51 -04:00
Steve Pulec
bf32bdc042 Merge pull request #326 from jraby/patch-2
Add test_multipart_duplicate_upload
2015-03-05 19:30:40 -05:00
Steve Pulec
3ac97318e1 Fix etag for reduced min part size. 2015-03-05 19:05:06 -05:00
jraby
4a14d8d3b3 Add test_multipart_duplicate_upload
Test to make sure we do not duplicate data when uploading the same part twice
2015-02-27 18:48:51 -05:00
jraby
5ba8421242 Fix reduced_min_part_size so that tests run 2015-02-27 18:42:37 -05:00
John Lynn
19d12d3bc3 Add failing test for bucket versioning 2015-02-22 22:22:32 +07:00
Steve Pulec
53a6d2466b Merge pull request #298 from kouk/ranged_get
support ranged get in S3
2015-02-14 08:35:23 -05:00
Konstantinos Koukopoulos
b2904fe849 add byte specifier for py3 2015-02-11 11:08:00 +02:00
Konstantinos Koukopoulos
297c730932 speed up multipart tests by temporarily reducing min part size 2015-02-11 11:05:26 +02:00
Konstantinos Koukopoulos
e1163e5223 fix some boundary conditions 2015-02-10 19:43:24 +02:00
Konstantinos Koukopoulos
261328d449 set content-range header so boto knows this is a ranged response 2015-02-10 19:15:10 +02:00
Konstantinos Koukopoulos
418a6a118e return proper status codes in range get, fix suffix range 2015-02-10 19:14:47 +02:00
Konstantinos Koukopoulos
9efd12c43c support more range specifiers in ranged gets 2015-02-10 17:55:44 +02:00
Konstantinos Koukopoulos
27a022f695 fix field names for py26 in s3 test 2015-02-10 17:31:40 +02:00
Konstantinos Koukopoulos
dbf1e64d44 support ranged get in S3 2015-02-10 17:28:18 +02:00
Konstantinos Koukopoulos
8ad39449be add a test for completing an upload with an invalid part order 2015-02-10 16:55:32 +02:00
Konstantinos Koukopoulos
4ef484d908 add test for uploading parts out of order in S3 multipart upload 2015-02-10 16:42:50 +02:00
Jeff Balogh
c7bf6ffc91 Folder results should include the delimiter. 2015-01-29 09:20:40 -08:00
Steve Pulec
4ab3b318eb Add S3 location response. Closes #279. 2014-12-10 20:44:00 -05:00
Steve Pulec
a2e56afef8 Add special S3 bucket create logic for us-east-1. Closes #273. 2014-12-10 20:20:51 -05:00
Steve Pulec
3a82f089a2 Cleanup metadata setting and add fix for multipart uploads. Closes #274. 2014-12-07 12:43:14 -05:00
Steve Pulec
1a015d0e39 Cleanup some incorrect date formats. 2014-11-29 23:34:40 -05:00
Steve Pulec
e88e3d5372 S3 Fixes for special metadata headers. 2014-11-27 10:43:10 -05:00
Steve Pulec
ad80eba311 Decode S3 value manually instead of using new boto feature. 2014-11-22 10:42:43 -05:00
Steve Pulec
9024066957 Fixes for some S3 Unicode issues. Closes #158. 2014-11-22 10:19:56 -05:00
Steve Pulec
8d6868f9d3 For S3 Key Head requests, just return the real body for now so that key length works. Closes #131. 2014-11-19 21:06:23 -05:00
David Baumgold
eedb4c4b73 Support Python 3 using six 2014-08-28 10:57:43 -04:00
David Baumgold
b4205cf267 Fix tests that fail due to bad unicode support 2014-08-27 11:58:03 -04:00
David Baumgold
4201e6b430 from __future__ import unicode_literals 2014-08-27 11:33:55 -04:00
Steve Pulec
64ed605c14 Ignore S3 ACL for now since we do not implement it. 2014-08-13 22:13:23 -04:00
Steve Pulec
5e35348c0d Cleanup S3 model methods to better deal with missing buckets. 2014-07-08 21:20:29 -04:00
Steve Pulec
be25a2ba99 Fix S3 to work with other regions. 2014-07-08 20:35:48 -04:00
Richard Eames
4cc45c3ac5 Implementation of bucket.list_versions 2014-06-27 16:21:32 -06:00
Richard Eames
e972000bb4 Keep track of previous versions of keys 2014-06-27 15:37:51 -06:00
Richard Eames
5409d99ca2 Added basic implementation of key/bucket versioning 2014-06-27 11:34:00 -06:00
Gabe Rives-Corbett
9f912e7a1f Implemeted bucket.delete_keys closes #124 2014-05-06 17:21:33 -04:00
Konstantinos Koukopoulos
d4602b9cae support listing all multipart uploads 2014-04-02 19:09:48 +03:00
Konstantinos Koukopoulos
84ec801a52 refactor etag calculation to support a correct multipart key etag 2014-04-02 14:40:04 +03:00
Steve Pulec
1767d17018 Cleanup S3 a bit and add more coverage 2014-03-30 11:50:36 -04:00
Konstantinos Koukopoulos
8ada1d7829 support restore operation on s3 keys 2014-03-26 19:15:08 +02:00
Konstantinos Koukopoulos
c9d5dffa24 support reduced redundancy storage 2014-03-26 17:52:31 +02:00
Konstantinos Koukopoulos
2ba64e1322 support replacing key metadata when copying 2014-03-26 16:02:14 +02:00
Steve Pulec
60cd79c6e2 S3 Mock should return an HTTP 409 if trying to create a bucket that
already exists. Closes #94.
2014-03-16 21:25:54 -04:00
Steve Pulec
8b278eb05d Clean up querystring logic. 2013-12-29 21:50:25 -05:00
Konstantinos Koukopoulos
85e32102fa break multipart test in two 2013-11-15 11:59:30 +02:00
Mike Attili
b64dbcaa12 Remove ()'s on complete_upload since should.throw requires a 'callable'. 2013-11-07 17:07:56 -05:00
Konstantinos Koukopoulos
6e65b5f6d4 fix part numbering in multipart tests 2013-09-30 16:37:50 +03:00
Konstantinos Koukopoulos
f25caa872d Merge branch 'master' of github.com:spulec/moto
Conflicts:
	moto/s3/models.py
	moto/s3/responses.py
2013-09-30 11:02:25 +03:00
Sylvain Fraïssé
0fd4a5a6f8 Updates a test to show the ltrip use problem 2013-09-24 00:00:52 +02:00
Steve Pulec
59fe1abfdf Cleanup flake8 2013-08-03 17:21:25 -04:00
Steve Pulec
3bc975188f more s3 tests for posting 2013-05-17 19:41:39 -04:00
Dan Berglund
7de4399b93 Added tests and made current tests pass 2013-05-17 11:43:09 +02:00
Steve Pulec
4c8244b755 Clean up code for listing s3 keys. Fix #14. 2013-05-07 00:03:05 -04:00
Steve Pulec
caf73557cd Fix issue with large S3 values. Closes #11. 2013-05-06 23:33:59 -04:00
Steve Pulec
47bd4c49a3 attempting to move to upstream httpretty 2013-05-03 19:33:13 -04:00
Steve Pulec
be26daaff4 Fix S3 bucket list objects order and delimiters. Closes #14. 2013-04-13 19:23:32 -04:00
Steve Pulec
98d7a50790 Unquote s3 key names. Closes #13. 2013-04-13 19:00:37 -04:00