Commit Graph

45 Commits

Author SHA1 Message Date
khneal
8b523c3fe1
EMR and SWF - add arn to response (#3873)
* emr: add ClusterArn to describe_cluster response

* emr: add ClusterArn to list_clusters response

* emr: add ClusterArn to put_auto_scaling_policy response

* emr: add ClusterArn to run_job_flow response

* emr: rename property "cluster_arn" to simply "arn"

* emr: generalize arn for account_id and region

* swf: add arn to list_domains response

* black reformat source code

* fix double import

* swf: require region on Domain object

Co-authored-by: Kevin Neal <Kevin_Neal@intuit.com>
2021-04-23 15:20:36 +01:00
Deepesh J
c31dffcc92
List instances Implementation in EMR (#3871)
* Implemented list instances in EMR

* removed import from tests

* make format

* fix W291 trailing whitespace

* removed to work for py2.7

* Storing only ec2_id and instance group in Fake instance

Co-authored-by: J <jdeepe@147dda1b0833.ant.amazon.com>
2021-04-23 08:03:06 +01:00
Brian Pandola
ac4a26f289
Add support for EMR-Managed Security Groups (#3824)
* Add support for EMR-Managed Security Groups

This covers the base case for EMR Clusters provisioned in a private subnet.

Ref: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-man-sec-groups.html

* Address PR comments

* Address PR comments
2021-04-02 15:34:02 +01:00
Brian Pandola
53a3e52c67
Fix: EMR ReleaseLabel validation does not respect semantic versioning (#3489)
Fixes #3474
2020-11-22 18:54:59 +00:00
jweite
5fe921c2bc
Added support for EMR Security Configurations and Kerberos Attributes. (#3456)
* Added support for EMR Security Configurations and Kerberos Attributes.

* Revised exception-raising test to work with pytest api.

* Added htmlcov to .gitignore; upgrading botocore to 1.18.17, per commit d29475e.

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-11-17 10:54:34 +00:00
Bert Blommers
273ca63d59 Linting 2020-11-11 15:55:37 +00:00
Bert Blommers
cb6731f340 Convert fixtures/exceptions to Pytest 2020-11-11 15:54:01 +00:00
Matěj Cepl
5697ff87a8 Back to Black 2020-11-10 14:12:38 +01:00
Matěj Cepl
ea489bce6c Finish porting from nose to pytest. 2020-11-10 08:25:05 +01:00
Matěj Cepl
77dc60ea97 Port test suite from nose to pytest.
This just eliminates all errors on the tests collection. Elimination of
failures is left to the next commit.
2020-11-10 08:23:44 +01:00
jweite
9bc6bded6e
EMR: Support for StepConcurrencyLevel. (#3351)
Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-10-02 14:07:13 +01:00
jweite
cd20668e9c
Support for autoscaling policies in run_jobflow, add_instance_group and list_instance_groups. (#3288)
Support for cluster_id parameter substitution in autoscaling policy cloudwatch alarm dimensions.
New operations put_autoscaling_policy and remove_autoscaling_policy support

Co-authored-by: Joseph Weitekamp <jweite@amazon.com>
2020-09-23 11:21:45 +01:00
addomafi
c6eca18434 Reformat 2020-03-06 18:11:07 -03:00
addomafi
23771549c2 #2784 Implementing assertions for testcase with instance groups 2020-03-06 18:10:39 -03:00
addomafi
c8dfbe9575 #2784 Adding missing support for EbsConfiguration on EMR instance groups 2020-03-06 15:12:44 -03:00
addomafi
a9b0677667 #2784 Adding missing support for EbsConfiguration on EMR run_job_flow 2020-03-05 18:11:49 -03:00
Bert Blommers
7205ab7785 #1427 - EMR - Return start time of first step 2020-02-20 08:59:21 +00:00
Asher Foa
96e5b1993d Run black on moto & test directories. 2019-10-31 10:36:05 -07:00
Jordan Sanders
4cce4defac Support CustomAmiId in EMR (#2037) 2019-05-25 11:19:26 +01:00
Ash Berlin-Taylor
1b20f21a75 Escape EMR template fields to avoid invalid XML responses (#1467)
I had an EMR step that contained a `&` and this caused the ListStep call to fail.

I've added the `| escape` filter to handle it in this case and a few other cases that look like they could suffer the same fate.
2018-03-21 08:57:50 -07:00
Steven Cipriano
c4b9088bfc Add support for recursive emr settings
- Updates _RecursiveDictRef to not implement __getitem__, avoiding errors when using recursive settings for an emr job flow
2017-06-27 11:31:43 -07:00
Steve Pulec
0adebeed24 Merge #913. 2017-05-10 21:58:42 -04:00
Steve Pulec
3c0c4c2996 Fix tests for py3. 2017-02-23 22:28:09 -05:00
Steve Pulec
f37bad0e00 Lints. 2017-02-23 21:37:43 -05:00
Steve Pulec
81836b6981 Get standalone server mode working for all tests. 2017-02-23 19:43:48 -05:00
Steve Pulec
fde721bed7 Testing new version of decorator. 2017-02-15 22:35:45 -05:00
Steve Pulec
721d1b3a3f Merge pull request #736 from okomestudio/ts/botocore_update
Remove botocore version pinning
2016-11-06 09:50:28 -05:00
Taro Sato
5538b541a8 Fix a bug with map type not properly handling multiple entries 2016-10-31 11:29:39 -07:00
Taro Sato
484faa54c4 Implement filters and pagers for some EMR end points 2016-10-20 10:03:17 -07:00
Taro Sato
b632df3aa4 Generate proper empty value for XML/JSON 2016-10-16 21:49:10 -07:00
Taro Sato
7cd404808b Better EMR coverage and boto3 request/response handling
This revision includes:

- A handler for requests for which content-type is JSON (from boto3).

- A decorator (generate_boto3_response) to convert XML responses to
  JSON (for boto3). This way, existing response templates for boto can
  be shared for generating boto3 response.

- Utility class/functions to use botocore's service specification data
  (accessible under botocore.data) for type casting, from query
  parameters to Python objects and XML to JSON.

- Updates to response handlers/models to cover more EMR end points and
  mockable parameters
2016-10-13 16:48:07 -07:00
Akira Chiku
7a11500171 Implement JSON responses for EMR (for Boto3) 2015-11-15 17:24:36 +09:00
Akira Chiku
91edbb21bf Make EMR multi-region 2015-07-12 19:32:16 +09:00
Shulman Evgeny
e498c95807 feature emr: set_termination_protection call implementation
feature emr: support for instance_groups param in run_jobflow
2015-07-08 10:26:58 +03:00
Steve Pulec
556394c68e No native dict comprehensions in py26. 2015-02-14 12:21:53 -05:00
Steve Pulec
95b1fa42b2 Add EMR clusters and tagging. 2015-02-14 12:00:14 -05:00
Shawn Falkner-Horine
632ea41d5f Tests: Tweaked two tests to be version-gated to Boto 2.8+. 2014-09-10 13:29:45 -07:00
Shawn Falkner-Horine
fd42d19f0e Tests: Tweaked two tests to be compatible with Boto 2.7, now-removed kwarg has its own version-gated test. 2014-09-10 13:02:55 -07:00
David Baumgold
eedb4c4b73 Support Python 3 using six 2014-08-28 10:57:43 -04:00
David Baumgold
4201e6b430 from __future__ import unicode_literals 2014-08-27 11:33:55 -04:00
Steve Pulec
8b278eb05d Clean up querystring logic. 2013-12-29 21:50:25 -05:00
Steve Pulec
62b72377bd Add visibletoallusers and normalizedinstancehours to EMR 2013-08-10 20:03:57 -04:00
Steve Pulec
1f83e6feea job_flow_role is not available in boto 2.7 2013-08-08 10:55:02 -04:00
Steve Pulec
e251fd8930 Add job_flow_role param or EMR 2013-08-08 10:40:29 -04:00
Steve Pulec
cea25e75c5 basic emr done 2013-08-07 20:32:29 -04:00