Commit Graph

66 Commits

Author SHA1 Message Date
Bert Blommers
11753adbf0
Techdebt: Replace sure with regular assertions in EMR (#6561) 2023-07-26 09:42:43 +00:00
steffyP
33c665fc21
EMR: fix handling of Properties for job flows (#6513) 2023-07-12 17:30:58 +00:00
Daniel Roschka
0588db704a
Remove dependency on pytz (#5749) 2022-12-09 22:56:08 -01:00
Bert Blommers
1a8ddc0f2b
Techdebt: Replace string-format with f-strings (for tests dirs) (#5678) 2022-11-17 21:41:08 -01:00
Brian Pandola
a49222fc15 Fix PytestRemovedIn8Warning Warnings 2022-10-27 19:46:47 -07:00
Bert Blommers
4f84e2f154
TechDebt - enable pylint rule redefined-outer-scope (#5518) 2022-10-04 16:28:30 +00:00
Bert Blommers
9e1d78d44e
TechDebt - remove warnings from tests (#5390) 2022-08-17 14:01:22 +00:00
Bert Blommers
3d913f8f15
MultiAccount support (#5192) 2022-08-13 09:49:43 +00:00
Brian Pandola
fff61af6fc
Fix TypeError when calling emr:RunJobFlow with AutoTerminationPolicy (#5285) 2022-07-02 17:03:54 -07:00
szopen321
dbcee3c196
Add instance type validation on add instance method (#5132) 2022-05-18 17:51:51 +00:00
Brian Pandola
626629ef82 Fix: EMR Steps should be returned in reverse order of creation
This is according to the AWS EMR documentation[1] and has been verified against a real AWS backend.

[1]:https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr.html#EMR.Client.list_steps
2022-03-19 22:51:53 -07:00
Bert Blommers
29d01c35bc
Update Black + formatting (#4926) 2022-03-10 13:39:59 -01:00
Bert Blommers
aa70ee254d
Remove deprecated decorators + boto dependency (#4378) 2022-01-18 14:18:57 -01:00
Deepesh J
9a8be5ce28
Add support for EMR AutoScalingRole (#4630) 2021-11-24 09:37:30 -08:00
Dominik Schubert
f717e494d3
Core - Use json module for error_json exception type rendering instead of jinja2 (#4587) 2021-11-17 15:19:03 -01:00
Bert Blommers
64e16d970a
EMR - Step state should be RUNNING, not STARTING (#4455) 2021-10-21 16:37:57 +00:00
Bert Blommers
14a69c7524
Techdebt: Enable pylint rules (#4432) 2021-10-18 19:44:29 +00:00
Bert Blommers
8f8ea45d32
EMR - mark tests as duplicated (#4332) 2021-09-23 13:27:55 +00:00
Bert Blommers
7693d77333
Remove Py2 support (#3915) 2021-07-26 07:40:39 +01:00
Brian Pandola
a95ca81e76
Support EMR clusters with multiple master nodes (#4019)
Reference: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha-launch.html
2021-06-18 06:04:21 +01:00
Brian Pandola
b0e2a750dc
Disallow termination of protected EMR job flows (#4015)
Error message verified against real AWS backend.
2021-06-17 12:20:45 +01:00
khneal
8b523c3fe1
EMR and SWF - add arn to response (#3873)
* emr: add ClusterArn to describe_cluster response

* emr: add ClusterArn to list_clusters response

* emr: add ClusterArn to put_auto_scaling_policy response

* emr: add ClusterArn to run_job_flow response

* emr: rename property "cluster_arn" to simply "arn"

* emr: generalize arn for account_id and region

* swf: add arn to list_domains response

* black reformat source code

* fix double import

* swf: require region on Domain object

Co-authored-by: Kevin Neal <Kevin_Neal@intuit.com>
2021-04-23 15:20:36 +01:00
Deepesh J
c31dffcc92
List instances Implementation in EMR (#3871)
* Implemented list instances in EMR

* removed import from tests

* make format

* fix W291 trailing whitespace

* removed to work for py2.7

* Storing only ec2_id and instance group in Fake instance

Co-authored-by: J <jdeepe@147dda1b0833.ant.amazon.com>
2021-04-23 08:03:06 +01:00
Brian Pandola
ac4a26f289
Add support for EMR-Managed Security Groups (#3824)
* Add support for EMR-Managed Security Groups

This covers the base case for EMR Clusters provisioned in a private subnet.

Ref: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-man-sec-groups.html

* Address PR comments

* Address PR comments
2021-04-02 15:34:02 +01:00
Brian Pandola
53a3e52c67
Fix: EMR ReleaseLabel validation does not respect semantic versioning (#3489)
Fixes #3474
2020-11-22 18:54:59 +00:00
jweite
5fe921c2bc
Added support for EMR Security Configurations and Kerberos Attributes. (#3456)
* Added support for EMR Security Configurations and Kerberos Attributes.

* Revised exception-raising test to work with pytest api.

* Added htmlcov to .gitignore; upgrading botocore to 1.18.17, per commit d29475e.

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-11-17 10:54:34 +00:00
Bert Blommers
273ca63d59 Linting 2020-11-11 15:55:37 +00:00
Bert Blommers
cb6731f340 Convert fixtures/exceptions to Pytest 2020-11-11 15:54:01 +00:00
Matěj Cepl
5697ff87a8 Back to Black 2020-11-10 14:12:38 +01:00
Matěj Cepl
ea489bce6c Finish porting from nose to pytest. 2020-11-10 08:25:05 +01:00
Matěj Cepl
77dc60ea97 Port test suite from nose to pytest.
This just eliminates all errors on the tests collection. Elimination of
failures is left to the next commit.
2020-11-10 08:23:44 +01:00
jweite
9bc6bded6e
EMR: Support for StepConcurrencyLevel. (#3351)
Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-10-02 14:07:13 +01:00
jweite
cd20668e9c
Support for autoscaling policies in run_jobflow, add_instance_group and list_instance_groups. (#3288)
Support for cluster_id parameter substitution in autoscaling policy cloudwatch alarm dimensions.
New operations put_autoscaling_policy and remove_autoscaling_policy support

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-09-23 11:21:45 +01:00
addomafi
c6eca18434 Reformat 2020-03-06 18:11:07 -03:00
addomafi
23771549c2 #2784 Implementing assertions for testcase with instance groups 2020-03-06 18:10:39 -03:00
addomafi
c8dfbe9575 #2784 Adding missing support for EbsConfiguration on EMR instance groups 2020-03-06 15:12:44 -03:00
addomafi
a9b0677667 #2784 Adding missing support for EbsConfiguration on EMR run_job_flow 2020-03-05 18:11:49 -03:00
Bert Blommers
7205ab7785 #1427 - EMR - Return start time of first step 2020-02-20 08:59:21 +00:00
Asher Foa
96e5b1993d Run black on moto & test directories. 2019-10-31 10:36:05 -07:00
Jordan Sanders
4cce4defac Support CustomAmiId in EMR (#2037) 2019-05-25 11:19:26 +01:00
Ash Berlin-Taylor
1b20f21a75 Escape EMR template fields to avoid invalid XML responses (#1467)
I had an EMR step that contained a `&` and this caused the ListStep call to fail.

I've added the `| escape` filter to handle it in this case and a few other cases that look like they could suffer the same fate.
2018-03-21 08:57:50 -07:00
Steven Cipriano
c4b9088bfc Add support for recursive emr settings
- Updates _RecursiveDictRef to not implement __getitem__, avoiding errors when using recursive settings for an emr job flow
2017-06-27 11:31:43 -07:00
Steve Pulec
0adebeed24 Merge #913. 2017-05-10 21:58:42 -04:00
Steve Pulec
3c0c4c2996 Fix tests for py3. 2017-02-23 22:28:09 -05:00
Steve Pulec
f37bad0e00 Lints. 2017-02-23 21:37:43 -05:00
Steve Pulec
81836b6981 Get standalone server mode working for all tests. 2017-02-23 19:43:48 -05:00
Steve Pulec
fde721bed7 Testing new version of decorator. 2017-02-15 22:35:45 -05:00
Steve Pulec
721d1b3a3f Merge pull request #736 from okomestudio/ts/botocore_update
Remove botocore version pinning
2016-11-06 09:50:28 -05:00
Taro Sato
5538b541a8 Fix a bug with map type not properly handling multiple entries 2016-10-31 11:29:39 -07:00
Taro Sato
484faa54c4 Implement filters and pagers for some EMR end points 2016-10-20 10:03:17 -07:00