Yasuke/moto
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,629 Commits 1 Branch 0 Tags 43 MiB
6dfd64ff3c
Commit Graph

363 Commits

Author SHA1 Message Date
Bert Blommers
8dbfd43c5c Use TaggingService for S3 - Cleanup 2020-04-01 15:35:25 +01:00
Bert Blommers
f7ad4cbc09 Use TaggingService for S3 Objects 2020-03-31 12:04:04 +01:00
Bert Blommers
6dd6686afc Use TaggingService for S3 Buckets 2020-03-31 11:10:38 +01:00
Mike Grima
0c191ac33b Raise errors on tagging buckets with aws:* 
Cannot tag S3 buckets with reserved tag key space `aws:`
 2020-03-30 17:27:06 -07:00
Bert Blommers
c3865532f9 #2711 - Register default S3 metrics in CloudWatch 2020-03-23 15:53:39 +00:00
Steve Pulec
94f85902bc
Merge pull request #2821 from bblommers/feature/2255 
CloudFormation - Add Support for FN:Transform and AWS:Include
 2020-03-19 19:40:46 -05:00
Steve Pulec
c13aadf75b
Merge pull request #2816 from bblommers/bugfix/2789 
S3 - Bugfix - Remove possibilty for concurrent access to file content
 2020-03-19 19:37:14 -05:00
Bert Blommers
cbf0397953 #2255 - CF - Implement FN::Transform and AWS::Include 2020-03-18 13:02:07 +00:00
Bert Blommers
5e4736e233 Remove unnecessary print-statements 2020-03-17 09:19:57 +00:00
Bert Blommers
e2434cbf6f Remove unnecessary lock 2020-03-17 09:19:02 +00:00
Bert Blommers
d8423b5de0 Optimize content length for large files 2020-03-17 09:16:12 +00:00
Asher Foa
374b623e1d Fix some 'DeprecationWarning: invalid escape sequence' warnings and use str.format for string interpolation. 
I am seeing a lot of deperecation warnings when I use moto for my tests (running under pytest), so I figured I'll clean up some of them.
 2020-03-12 13:50:59 -07:00
Bert Blommers
3802767817 S3 - Add test case to showcase bug when downloading large files 2020-03-12 12:25:31 +00:00
Steve Pulec
6b6a059350
Merge pull request #2132 from Projectplace/tagging-directive 
Support x-amz-tagging-directive in s3 copy_object
 2020-02-17 20:00:30 -06:00
Steve Pulec
a1ffb47ae4
Merge pull request #2751 from mikegrima/s3control 
Implemented S3 Account-level public access block.
 2020-02-17 19:53:15 -06:00
Mike Grima
11b7be0e85 Implemented S3 Account-level public access block. 
- Also added AWS Config listing and fetching support
- Also fixed Lambda test breakage
 2020-02-17 15:41:27 -08:00
Daniel Wallace
01f3b60c09
Allow ports in k8s service urls for s3 mock 
If there is a port in the host for the request, then this if statement
is not tripped.
 2020-02-17 17:38:53 -06:00
Bert Blommers
f74f08581a S3 - Add IllegalLocationConstraint validation when creating buckets 2020-02-02 10:36:51 +00:00
Steve Pulec
19bf8bf762 Change S3 S3_ALL_BUCKETS response to return bucket creation_date in iso format. 2020-01-22 20:43:45 -06:00
Steve Pulec
d73a548bb0 Remove duplicate StorageClass in S3_MULTIPART_LIST_RESPONSE. 2020-01-22 19:45:27 -06:00
Waldemar Hummer
bcc4a74869 avoid rendering "None" as S3 Prefix value 2019-12-21 12:08:13 +01:00
Fabio Dias
743dd46399 black linting 2019-12-16 21:25:20 -05:00
Fabio Dias
fd9ac4b9a5 ACCOUNT_ID moved to moto.core, MOTO_ACCOUNT_ID env var override 2019-12-16 21:05:29 -05:00
Fabio Dias
fb9ebe5a47 linting fixes 2019-12-16 19:56:11 -05:00
Fabio Dias
b83a750630 Added references to moto.iam.models.ACCOUNT_ID instead of hardcoded id 2019-12-16 19:56:11 -05:00
Mike Grima
84ccdbd1cd Implemented S3 Public Access Block 2019-12-12 17:45:31 -08:00
Niklas Janlert
b51d5ad65f Support x-amz-tagging-directive in s3 copy_object 2019-11-23 00:38:26 +01:00
Asher Foa
96e5b1993d Run black on moto & test directories. 2019-10-31 10:36:05 -07:00
Mike Grima
503bc333ca Small fix for S3-AWS Config compatibility 
- Small bug in tags with AWS Config
- Aggregated results lack "tags" in the result set
- Buckets also add a supplementary configuration of
"BucketTaggingConfiguration"
 2019-10-29 14:35:13 -07:00
Mike Grima
e9dc0c9a3a Fixed bugs in AWS Config Querying 
- Fixed some log bugs in the Config querying APIs
- Fixed an issue with S3 ACLs when described from Config (it's actually
a double-wrapped JSON)
 2019-10-23 22:32:45 -07:00
Mike Grima
57dc6522f2
Merge pull request #2505 from mikegrima/configFix 
Fixed a bug with S3 bucket policies for AWS Config
 2019-10-22 15:53:18 -07:00
Mike Grima
9273db7e61 Fixed a bug with S3 bucket policies for AWS Config 2019-10-22 15:01:39 -07:00
Mike Grima
1cde5d37eb
Merge pull request #2483 from thincal/fix/list-objects-with-truncation 
s3 list_object: return the NextMarker info if it's truncated
 2019-10-22 09:57:15 -07:00
LS
d3ed7ee240 s3 list_object: return the NextMarker info if it's truncated 2019-10-21 09:40:05 +08:00
Harrison Termotto
df2de373b2 Add tagging to versioned s3 objects. 2019-10-17 00:16:16 -04:00
koshigoe
98f33740e1
fix(s3): check whether key is None or not to avoid exception. 
```
AttributeError: 'NoneType' object has no attribute 'multipart'
```
 2019-10-16 18:13:59 +09:00
Mike Grima
2a2c3e80f0 Adding support for querying the AWS Config resource for S3. 
- Need to add support still for batch requests and batch aggregation requests in a future PR
 2019-10-13 10:14:15 -07:00
Mike Grima
ab0853cddc Made fixes to the S3 Lifecycle mocks to be more consistent with the API. 2019-10-05 12:29:09 -07:00
Mike Grima
d925335f05
Merge pull request #2443 from mikegrima/configquery 
Adding support for querying AWS Config
 2019-10-03 13:38:43 -07:00
Mike Grima
c4b310d7a5 Adding support for querying AWS Config for supported configurations. 
At this time, only adding support for S3.
 2019-10-03 11:43:56 -07:00
Chih-Hsuan Yen
6c0a2dc374
Fix copying S3 objects with question marks in keys 
PR #2062 fixes copy for unicode keys. However, the change breaks keys
with question marks. Here is an alternative fix that works for both
emojis and question marks.

I notice this when running the test suite of aiobotocore. One of the
tests [1] is broken with moto >= 1.3.8.

[1] https://github.com/aio-libs/aiobotocore/blob/0.10.3/tests/test_basic_s3.py#L314-L329
 2019-09-29 23:55:52 +08:00
Jesse Vogt
3b4cd1c27b switch from calling clean in loop to undoing clean in delete_keys 2019-09-24 17:07:58 -05:00
Jesse Vogt
2df0309db5 unquote key name multiple times until stable value 2019-09-24 16:10:08 -05:00
Gapex
1c36e1e2c5 update unit test and fix StartAfter 2019-09-17 10:42:10 +08:00
Gapex
4946f8b853 'lint' 2019-09-16 15:31:57 +08:00
Gapex
59f87e30ba split truncated keys by type 2019-09-16 15:20:24 +08:00
Gapex
47635dc82e update key of sort 2019-09-16 13:33:53 +08:00
Gapex
a36b84b3aa fix MaxKeys in list_objects_v2 2019-09-16 11:35:36 +08:00
Gapex
d6ef01b9fd lint 2019-09-12 18:40:07 +08:00
Gapex
eea67543d1 MaxKeys limits the sum of folders and keys 2019-09-12 17:54:02 +08:00
Łukasz Sznuk
e0304bc500 Allows leading // for mocked s3 paths (#1637). 2019-08-28 16:17:45 +02:00
Vury Leo
71241f1c3f
make linter happy 2019-08-20 15:17:17 +08:00
Vury Leo
1249ba8d3b
fix KeyCount in s3.list_objects_v2 2019-08-20 15:01:37 +08:00
Berislav Kovacki
b7884ef903 Add S3 support for INTELLIGENT_TIERING, GLACIER and DEEP_ARCHIVE storage 
* Add INTELLIGENT_TIERING, GLACIER and DEEP_ARCHIVE as valid storage
classes for objects
* Add ObjectNotInActiveTierError error on PUT object copy for GLACIER
and DEEP_ARCHIVE storage class objects
 2019-08-05 17:34:39 +02:00
Steve Pulec
e4a4e61835 Allow passing of encoding-type for s3 get_bucket_versions without throwing error. 
This was a change made in https://github.com/boto/botocore/pull/1794.
 2019-07-29 22:52:29 -05:00
Steve Pulec
69d86cbd54
Merge pull request #2300 from acsbendi/master 
Basic IAM authentication, authorization
 2019-07-28 17:22:52 -05:00
acsbendi
f3f47d44ac Fixed error in python 2 and did some refactoring. 2019-07-26 21:05:04 +02:00
acsbendi
290f8f9fd5 Fixed host header not included in S3 requests sometimes. 2019-07-26 19:50:24 +02:00
acsbendi
45a380a807 Fixed host not present in headers for S3 requests. 2019-07-24 21:29:00 +02:00
acsbendi
d428acdb7c Separate SignatureDoesNotMatchError for S3. 2019-07-24 21:01:11 +02:00
Steve Pulec
5abe56fd2b
Merge pull request #2250 from szczeles/s3_delete_objects_boto3 
Raising MalformedXML exception when using boto3 client and s3.delete_objects()
 2019-07-23 22:41:45 -05:00
acsbendi
95799b99bc Fixed incorrect authentication error handling in S3. 2019-07-16 16:27:50 +02:00
Ruslan Kuprieiev
2c2dff22bc moto: s3: support partNumber for head_object 
To support it, we need to keep multipart info in the key itself when
completing multipart upload.

Fixes #2154

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>
 2019-07-15 20:47:29 +03:00
acsbendi
de01adec57 Fixed linting errors. 2019-07-13 14:58:42 +02:00
acsbendi
419fcf2ee9 Merge branch 'master' of https://github.com/spulec/moto into spulec-master 2019-07-13 13:43:19 +02:00
Mariusz Strzelecki
f32db6e64a Raising MalformedXML exception when using boto3 client and s3.delete_objects() 2019-07-11 09:53:23 +02:00
acsbendi
23957fe940 Implemented finding credentials from already created IAM users and roles. 2019-07-08 19:57:14 +02:00
acsbendi
86758182a7 Removed print. 2019-07-04 16:42:11 +02:00
acsbendi
5dbec8aee5 Implemented checking if S3 action is permitted. 2019-07-04 16:38:53 +02:00
Aden Khan
b2adbf1f48 Adding the functionality and test so that the If-Modified-Since header is honored in GET Object 
Signed-off-by: Aden Khan <aakhan@sep.com>
 2019-07-03 14:25:08 -04:00
Alexander Mohr
7271fb9391 Fix S3 backend operations with VersionId (#2055) 
* fix s3 issues

* fix merge conflict

* fix and add test cases
 2019-05-25 20:19:33 +01:00
Gregory Bataille
d952410965 1987 support transfer acceleration (#2018) 
* chore(): remove executable flag on moto/s3/response.py

* chore(): ignore .eggs temp file

* feat(#1987): get bucket acceleration support

* feat(#1987): put bucket acceleration support

* feat(#1987): suspend undefined bucket is a no-op

* feat(#1987): validate accelerate_configuration status

* feat(#1987): bucket containing dots do not support acceleration

* doc(#1987): update implementation coverage
 2019-05-25 11:19:00 +01:00
Yaroslav Admin
b85d21b8fe Fixed copy-object from unversioned bucket to versioned bucket 
The response of the copy-object operation was missing VersionId property
when source bucket is not versioned.
 2019-04-02 15:30:01 +02:00
Gilbert Gilb's
dedb53371e [S3] Support null version ids for buckets with versioning disabled 2019-03-09 10:21:49 +01:00
Domenico Testa
74ff2ccc95 Reformatting to get better coverage results 2019-03-04 09:16:43 +01:00
Domenico Testa
6897a118ed Fixing S3 copy_key error when using unicode. 2019-02-16 16:42:00 +01:00
Steve Pulec
9f3ae31a53 Fix bad merge. 2018-12-29 06:33:55 -05:00
Steve Pulec
97d9d46770
Merge pull request #1999 from dargueta/s3-spooling 
Don't store S3 entirely in memory
 2018-12-28 21:29:19 -05:00
Steve Pulec
dec665be43
Merge pull request #1986 from okomestudio/ts/fix-s3-common-prefix-pagination 
Activate proper pagination for S3 common prefixes
 2018-12-28 21:22:07 -05:00
Steve Pulec
e681f55ba1
Merge branch 'master' into bucket-name-length-limit 2018-12-28 21:04:10 -05:00
Steve Pulec
65364a8e71
Merge pull request #1961 from gbataille/fix_1959_location_constraint_us_east_1 
Fix 1959 location constraint us east 1
 2018-12-28 21:00:29 -05:00
Diego Argueta
191ad6d778 Make keys pickleable 2018-12-20 11:15:15 -08:00
Diego Argueta
f15f006f78 Hack around text problem in unit tests. 
Now that payloads are not allowed to be text, some unit tests will cause crashes
on Python 3 because the payload sent by requests gets passed to FakeKey as a
string instead of raw bytes.

I haven't been able to figure out a way around the issue that doesn't get super
messy inside s3/responses.py so I'm just converting the value to bytes using the
system's default encoding.
 2018-12-20 00:34:39 -08:00
Diego Argueta
b0a280bde2 Move S3 storage to SpooledTemporaryFile 2018-12-18 14:20:48 -08:00
Taro Sato
2038fa92be Activate proper pagination for S3 common prefixes 2018-12-05 16:17:28 -08:00
Jon Michaelchuck
d29869bf9b flake8 fix 2018-11-27 08:32:30 -08:00
Jon Michaelchuck
009b02bcd5 Raise a client error if PutBucketTags request contains duplicate keys 
A PutBucketTags request with duplicate keys will raise a ClientError
with code InvalidTag and message 'Cannot provide multiple Tags with the
same key'.
 2018-11-26 15:56:46 -08:00
Jon Michaelchuck
69e093fcea flake8 indentation fix 2018-11-22 06:16:37 -06:00
Jon Michaelchuck
fb7e52becc Check bucket name length at CreateBucket 
Check that s3 bucket names follow the documented length restriction:

'Bucket names must be at least 3 and no more than 63 characters long.'

See https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
 2018-11-22 06:08:03 -06:00
Grégory Bataille
34ac5c72b9 feat(#1959): bucket in us-east-1 return None as LocationConstraint 2018-11-20 19:50:42 +01:00
Grégory Bataille
437eb892e2 feat(#1959): LocationConstraint us-east-1 is not accepted by the CreateBucket operation 2018-11-20 18:57:05 +01:00
Jordan Guymon
8e909f580a MockAWS implementation using botocore event hooks 2018-10-29 09:12:22 +01:00
Steve Pulec
27ca96519b Fix extra whitespace in s3. Closes #1844. 2018-10-15 01:37:38 -04:00
Jon Beilke
5b3b52752d explicitly check that lifecycle actions are not None when setting lifecycle 2018-10-04 10:25:16 -05:00
Jon Beilke
a1a8ac7286 check for None in lifecycle actions 2018-10-03 01:26:09 -05:00
Jon Beilke
9b5f983cb5 add action validation to set_lifecycle() 2018-10-03 01:11:11 -05:00
Jon Beilke
691a8722a8 formatting fix for flake8 due to extra spaces 2018-10-03 00:45:47 -05:00
Jon Beilke
ea4fcaa82a add support for NoncurrentVersionTransition, NoncurrentVersionExpiration, and AbortIncompleteMultipartUpload actions to s3 lifecycle rules 2018-10-03 00:40:28 -05:00
Jordan Upiter
756b5d6671 Add support for multiple delete markers on an s3 object 2018-09-11 10:41:01 -04:00
Steve Pulec
0309d1ec39
Merge pull request #1707 from Whamied/etag-no-quote-patch 
Added support for multipart upload completion with unquoted etags
 2018-07-19 10:05:07 -04:00