126 Commits

Author SHA1 Message Date
Mike Grima
84ccdbd1cd Implemented S3 Public Access Block 2019-12-12 17:45:31 -08:00
Asher Foa
96e5b1993d Run black on moto & test directories. 2019-10-31 10:36:05 -07:00
Mike Grima
503bc333ca Small fix for S3-AWS Config compatibility
- Small bug in tags with AWS Config
- Aggregated results lack "tags" in the result set
- Buckets also add a supplementary configuration of
"BucketTaggingConfiguration"
2019-10-29 14:35:13 -07:00
Mike Grima
e9dc0c9a3a Fixed bugs in AWS Config Querying
- Fixed some log bugs in the Config querying APIs
- Fixed an issue with S3 ACLs when described from Config (it's actually
a double-wrapped JSON)
2019-10-23 22:32:45 -07:00
Mike Grima
9273db7e61 Fixed a bug with S3 bucket policies for AWS Config 2019-10-22 15:01:39 -07:00
Harrison Termotto
df2de373b2 Add tagging to versioned s3 objects. 2019-10-17 00:16:16 -04:00
koshigoe
98f33740e1
fix(s3): check whether key is None or not to avoid exception.
```
AttributeError: 'NoneType' object has no attribute 'multipart'
```
2019-10-16 18:13:59 +09:00
Mike Grima
2a2c3e80f0 Adding support for querying the AWS Config resource for S3.
- Need to add support still for batch requests and batch aggregation requests in a future PR
2019-10-13 10:14:15 -07:00
Mike Grima
ab0853cddc Made fixes to the S3 Lifecycle mocks to be more consistent with the API. 2019-10-05 12:29:09 -07:00
Berislav Kovacki
b7884ef903 Add S3 support for INTELLIGENT_TIERING, GLACIER and DEEP_ARCHIVE storage
* Add INTELLIGENT_TIERING, GLACIER and DEEP_ARCHIVE as valid storage
classes for objects
* Add ObjectNotInActiveTierError error on PUT object copy for GLACIER
and DEEP_ARCHIVE storage class objects
2019-08-05 17:34:39 +02:00
Steve Pulec
e4a4e61835 Allow passing of encoding-type for s3 get_bucket_versions without throwing error.
This was a change made in https://github.com/boto/botocore/pull/1794.
2019-07-29 22:52:29 -05:00
Ruslan Kuprieiev
2c2dff22bc moto: s3: support partNumber for head_object
To support it, we need to keep multipart info in the key itself when
completing multipart upload.

Fixes #2154

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>
2019-07-15 20:47:29 +03:00
Alexander Mohr
7271fb9391 Fix S3 backend operations with VersionId (#2055)
* fix s3 issues

* fix merge conflict

* fix and add test cases
2019-05-25 20:19:33 +01:00
Gregory Bataille
d952410965 1987 support transfer acceleration (#2018)
* chore(): remove executable flag on moto/s3/response.py

* chore(): ignore .eggs temp file

* feat(#1987): get bucket acceleration support

* feat(#1987): put bucket acceleration support

* feat(#1987): suspend undefined bucket is a no-op

* feat(#1987): validate accelerate_configuration status

* feat(#1987): bucket containing dots do not support acceleration

* doc(#1987): update implementation coverage
2019-05-25 11:19:00 +01:00
Yaroslav Admin
b85d21b8fe Fixed copy-object from unversioned bucket to versioned bucket
The response of the copy-object operation was missing VersionId property
when source bucket is not versioned.
2019-04-02 15:30:01 +02:00
Gilbert Gilb's
dedb53371e [S3] Support null version ids for buckets with versioning disabled 2019-03-09 10:21:49 +01:00
Steve Pulec
97d9d46770
Merge pull request #1999 from dargueta/s3-spooling
Don't store S3 entirely in memory
2018-12-28 21:29:19 -05:00
Steve Pulec
e681f55ba1
Merge branch 'master' into bucket-name-length-limit 2018-12-28 21:04:10 -05:00
Diego Argueta
191ad6d778 Make keys pickleable 2018-12-20 11:15:15 -08:00
Diego Argueta
f15f006f78 Hack around text problem in unit tests.
Now that payloads are not allowed to be text, some unit tests will cause crashes
on Python 3 because the payload sent by requests gets passed to FakeKey as a
string instead of raw bytes.

I haven't been able to figure out a way around the issue that doesn't get super
messy inside s3/responses.py so I'm just converting the value to bytes using the
system's default encoding.
2018-12-20 00:34:39 -08:00
Diego Argueta
b0a280bde2 Move S3 storage to SpooledTemporaryFile 2018-12-18 14:20:48 -08:00
Jon Michaelchuck
009b02bcd5 Raise a client error if PutBucketTags request contains duplicate keys
A PutBucketTags request with duplicate keys will raise a ClientError
with code InvalidTag and message 'Cannot provide multiple Tags with the
same key'.
2018-11-26 15:56:46 -08:00
Jon Michaelchuck
69e093fcea flake8 indentation fix 2018-11-22 06:16:37 -06:00
Jon Michaelchuck
fb7e52becc Check bucket name length at CreateBucket
Check that s3 bucket names follow the documented length restriction:

'Bucket names must be at least 3 and no more than 63 characters long.'

See https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
2018-11-22 06:08:03 -06:00
Jon Beilke
5b3b52752d explicitly check that lifecycle actions are not None when setting lifecycle 2018-10-04 10:25:16 -05:00
Jon Beilke
a1a8ac7286 check for None in lifecycle actions 2018-10-03 01:26:09 -05:00
Jon Beilke
9b5f983cb5 add action validation to set_lifecycle() 2018-10-03 01:11:11 -05:00
Jon Beilke
691a8722a8 formatting fix for flake8 due to extra spaces 2018-10-03 00:45:47 -05:00
Jon Beilke
ea4fcaa82a add support for NoncurrentVersionTransition, NoncurrentVersionExpiration, and AbortIncompleteMultipartUpload actions to s3 lifecycle rules 2018-10-03 00:40:28 -05:00
Jordan Upiter
756b5d6671 Add support for multiple delete markers on an s3 object 2018-09-11 10:41:01 -04:00
Steve Pulec
0309d1ec39
Merge pull request #1707 from Whamied/etag-no-quote-patch
Added support for multipart upload completion with unquoted etags
2018-07-19 10:05:07 -04:00
Waleed Hamied
f8fdd439ad Added support for multipart upload confirmation with unquoted etags 2018-07-03 15:36:41 -04:00
Sanjeev Suresh
29da006f78 changed the getList default to an empty list instead of None, because otherwise an exception is raised when trying to iterate over an empty list 2018-06-21 15:26:27 -07:00
Alec Rajeev
6c10dc0403 Fix Storage Class error handling (#1653)
Added check for valid storage classes in set_key and copy_key
added unit test for standard storage and infrequent access
2018-06-07 08:09:36 +01:00
Alexander Mohr
7a57dc2034 fix errors 2018-05-03 01:40:49 -07:00
Mike Grima
4184acc0d2 Added Filtering support for S3 lifecycle (#1535)
* Added Filtering support for S3 lifecycle

Also added `ExpiredObjectDeleteMarker`.

closes #1533
closes #1479

* Result set no longer contains "Prefix" if "Filter" is set.
2018-04-02 14:19:14 -07:00
Mike Grima
cc14114afe Implemented S3 get/put_notification_configuration (#1516)
closes #973
2018-03-21 09:11:24 -07:00
Srikanth Raju
4f05aa725c S3: Do not attempt to return deleted files in bucket listing 2018-03-06 01:55:14 -08:00
Mike Grima
770281aef2 Added put_bucket_logging support (#1401)
- Also added put acl for XML
- Put logging will also verify that the destination bucket exists in the same region with the proper ACLs attached.
2018-01-02 20:47:57 -08:00
Terry Cain
b40c5e557e Fixed S3 versioning bug + minor cleanup (#1272)
* Fixed S3 versioning bug + minor cleanup

Fixes 1271

* flake8
2017-10-18 16:22:35 -07:00
Jack Danger
2055bb62f5 enforce s3 acls 2017-09-16 12:49:07 -07:00
Mike Grima
5d8cd22b01 Fixed S3 lifecycle error message. (#1110)
Fixes #1109
Also added PutBucketTagging support
Also added Bucket CORS support
2017-09-07 11:30:05 -07:00
Alexander Mohr
2447725e98 fix list-object-versions with prefix (#1045)
fix list-object-versions with prefix
2017-08-10 17:14:31 -07:00
eric-weaver
abf3078c28 implement s3 object tagging 2017-07-15 22:36:12 -04:00
Kate Heddleston
a21413f4ea NoSuchKey error in S3 is actually '404'
Fixes #571 and #953
2017-05-22 19:08:12 -07:00
georgepsarakis
9a861a367d Enable DeleteMarker support for versioned buckets
- Add object version deletion
- Add DeleteMarker in versioned key store as latest version
- GetObject returns NoSuchKey for objects with DeleteMarker as latest
  version
- Enable IsLatest in response when listing object versions
2017-05-14 19:56:25 +03:00
Steve Pulec
2adc5f2ace Prefix should not be required for S3 lifecycle config. Closes #930. 2017-05-10 22:45:01 -04:00
Hugo Picado
0945765537
Fixing metadata key on s3 operation response
ETag metadata key is being returned as "Etag" instead of "ETag". This leads to issues in some AWS SDKs using MotoServer. This change fixes the issue by updating the key to the correct format.

This closes #920
2017-04-27 13:57:18 +01:00
Dmytro Milinevskyy
6e61ee4caa s3: handle WebsiteRedirectLocation
spulec/moto#821
2017-04-14 13:32:52 +02:00
Steve Pulec
6d422d1f37 Add BaseModel to all models. 2017-03-11 23:41:12 -05:00