Commit Graph

92 Commits

Author SHA1 Message Date
Hugo Lopes Tavares
973264d940 Convert struct argument to bytestring to avoid errors with Python 2.7.6 2017-08-10 19:59:46 -04:00
Waldemar Hummer
ce2f3e6e2b fix receiving of messages from queues with a dot character in their name 2017-08-05 15:47:40 +10:00
Ferran Puig
e4f42d5880 Don't use exponential notation for SQS message timestamps 2017-07-03 16:17:01 +02:00
Luis Jimenez
898031b40c SQSResponse: include MD5OfMessageAttributes parameter only when there are message attributes 2017-06-29 09:27:31 -04:00
brett55
7033d142a0 - Bug fix for infinite loop when WaitTimeSeconds was set to 0
- Added 2 unit tests
2017-06-01 09:51:33 -06:00
Jack Danger Canty
6679def702 Python 2/3 compat for MD5 of SQS attributes 2017-05-22 18:11:59 -07:00
Jack Danger Canty
5e88b5d1b4 MD5 calculation of SQS message attributes
This implements the same MD5 hashing pattern as implemented in the Ruby
and Java AWS SDKs

Doesn't yet handle list types but if you're reading this you might be
surprised how easy that is to add. Give it a shot and if you get stuck
reach out to me for help.
2017-05-22 18:11:59 -07:00
Waldemar Hummer
a5727bf64a fix SQS message polling to abort after wait_seconds_timeout 2017-04-06 21:40:03 +10:00
michael_lerch
83084bf2af Prevent 100% cpu usage while SQS long polling on an empty queue
While using moto server with a test SQS client, I noticed significant CPU usage while the client was long polling. I narrowed this down to the `receive_messages` call of the SQS service sitting in a `while True:` statement with no work to be done, thus looping forever.

To produce this issue, I do:
```
$ python3 -m venv venv
$ . ./venv/bin/activate
(venv) $ pip install moto moto[server] boto3
Collecting moto
  Downloading moto-0.4.31-py2.py3-none-any.whl (303kB)
  --snip--
(venv) $ moto_server sqs &
[1] 31727
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
(venv) $ python3
Python 3.6.0 (default, Dec 24 2016, 08:01:42)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto3
>>> client = boto3.client('sqs', region_name='us-east-1', endpoint_url='http://127.0.0.1:5000');
>>> client.create_queue(QueueName='testing')
127.0.0.1 - - [16/Mar/2017 13:34:20] "POST / HTTP/1.1" 200 -
{'QueueUrl': 'http://sqs.us-east-1.amazonaws.com/123456789012/testing', 'ResponseMetadata': {'RequestId': '7a62c49f-347e-4fc4-9331-6e8e7a96aa73', 'HTTPStatusCode': 200, 'HTTPHeaders': {'content-type': 'text/html; charset=utf-8', 'content-length': '343', 'server': 'Werkzeug/0.12.1 Python/3.6.0', 'date': 'Thu, 16 Mar 2017 20:34:20 GMT'}, 'RetryAttempts': 0}}
>>> client.receive_message(QueueUrl='http://sqs.us-east-1.amazonaws.com/123456789012/testing', MaxNumberOfMessages=10, WaitTimeSeconds=10)
```

At this point the moto server will run at 100% cpu for 10 seconds until the request times out waiting for a message. If multiple clients are continuously reconnected (as in mocking a normal sqs worker setup) the server will sit at 100% cpu indefinitely.

This pull request adds a simple sleep statement to the SQS `receive_messages` call when there are no messages to process. In doing so, the loop will be limited to executing once per 0.001 seconds when the queue is empty. The CPU usage is nearly 0% after this change.
2017-03-16 13:43:45 -07:00
Seamus Cawley
bd2ff89bf1 Ensure SQS property WaitTimeSeconds is an integer 2017-03-13 13:56:55 +00:00
Steve Pulec
6d422d1f37 Add BaseModel to all models. 2017-03-11 23:41:12 -05:00
Steve Pulec
1709208872 First version of dashboard. 2017-03-11 22:45:42 -05:00
Steve Pulec
3b4ef2cf15 Merge in master. 2017-03-05 09:58:39 -05:00
Chris LaRose
56f9409ca9 Use request URL to generate SQS queue URLs; fixes #626 (#827) 2017-03-04 22:53:14 -05:00
Steve Pulec
f37bad0e00 Lints. 2017-02-23 21:37:43 -05:00
Steve Pulec
fde721bed7 Testing new version of decorator. 2017-02-15 22:35:45 -05:00
Steve Pulec
d3df810065 Generalize decorator code. 2017-02-12 00:22:29 -05:00
Brian Kruger
f212d70104 Use a sane aws sender_id from SQS. (#791) 2017-01-11 20:38:55 -05:00
Raghavendra D Prabhu
d07c646032 sqs: Use unix_time in place of time.time() (#787)
unix_time() from moto.core.utils is used as the time source through moto,
and it is identical to time.time() in output. Hence, using unix_time()
since it makes mocking easier during testing (when time is mocked out).
2017-01-11 20:35:56 -05:00
Steve Pulec
38021c6ef5 Test Boto 2.43.0 . 2016-11-06 10:29:50 -05:00
Raghavendra Prabhu
34a28b0fd5 Use region from ctor to set queue_arn.
Without this, any region passed to boto3.resource is ignored and unit
tests depending on it break.
2016-08-17 22:12:00 +01:00
Steve Pulec
ececa8009b Merge pull request #665 from macheins/fix-broken-sqs-queue-arn-attribute
Fix broken SQS QueueArn attribute
2016-07-24 09:28:47 -04:00
macheins
ef8b6c1f28 Fix broken format of SQS QueueArn attribute 2016-07-19 14:15:16 +02:00
Nuno Santos
8ffd4e51ec Fix retrieving inexistent queue errors when using boto3.
Handle 404 errors when trying to retrieve a SQS queue that does not
exist. Add get_queue and get_inexistent_queue tests for boto3.
2016-07-16 12:09:34 +01:00
Steve Pulec
ea43f41470 Merge pull request #625 from nfvs/fix_whitespace
Strip extra whitespace and newlines from SQS response templates.
2016-06-17 17:51:07 -04:00
Nuno Santos
68eeb5948b Strip extra whitespace and newlines from SQS response templates.
Fixes #623
2016-06-02 12:26:50 +02:00
Matt Long
9b7902018f Respect VisibilityTimeout when calling receive_message
Previously, receive_message would always use the queue's default
visibility timeout instead of the value passed as a query parameter when
calling the receive_message method on an SQS connection.
2016-05-31 15:13:49 -07:00
Steve Pulec
a600deb96a Fix merge conflicts and add EC2 Instance delete. Closes #576. 2016-04-28 09:22:31 -04:00
Scott Leishman
3fd33c03bf boto3 SQS set attributes fix. 2016-02-19 23:32:01 -08:00
Steve Pulec
63296049f1 More explicit SQS message checking. Closes #519. 2016-02-14 17:10:17 -05:00
Steve Pulec
705ec314a3 Cleanup different places using unix_time() 2015-11-27 14:14:40 -05:00
Ian Dees
27e7767883 Remove stuff that doesn't need to change 2015-11-09 20:19:51 -05:00
Ian Dees
97b7781c13 Include XML response template 2015-11-09 20:09:08 -05:00
Ian Dees
f8cbcfc098 How about in the SQSResponse object 2015-11-09 20:03:52 -05:00
Ian Dees
b732e116a4 Try the length of the body 2015-11-09 19:53:14 -05:00
Ian Dees
540ee79ad9 Put the size check in the queue 2015-11-09 19:51:17 -05:00
Ian Dees
5b2a724219 Check SQS message size 2015-11-09 19:29:08 -05:00
Andy Raines
76bce7954a Fixes #430: MD5 hashing should be done to the real body, not an escaped one 2015-10-05 14:14:56 +01:00
Steve Pulec
f72613cc47 Fix queue urls for other regions. Closes #411. 2015-09-19 09:18:16 -04:00
Dustin J. Mitchell
00e40c487e Return messages once they are gathered
If one or more messages are available, stop waiting and return them.
2015-08-31 11:27:42 -04:00
Jot
178d1c3a93 Added support for WaitTimeSeconds in SQS #392 2015-08-13 02:58:14 +02:00
Steve Pulec
519726a70a Cleaning up SQS for boto3. Closes #385. 2015-08-02 09:45:40 -04:00
Steve Pulec
53fff2eb84 Add UpdateStack functionality for Cloudformation. 2015-07-13 13:56:46 -04:00
Steve Pulec
6a0e8df189 Merge pull request #354 from silveregg/0.4.1-threadsafe
threadsafe fix
2015-05-29 23:34:55 -04:00
Steve Pulec
0195e04053 Allow deleting of SQS invisible messages. Closes #352. 2015-05-29 23:26:43 -04:00
Steve Pulec
6b44e41249 Add SQS purge_queue action. Closes #349. 2015-05-29 23:21:14 -04:00
Jeffrey Gelens
c2f2c16892 Fixed non threadsafe code 2015-03-16 13:13:40 +01:00
dreadpirateshawn
9affa7753d Performance: Leverage jinja2's Environment to cache compiled Templates. 2014-12-12 12:46:07 -08:00
Steve Pulec
16660ab72b Allow message passing to SQS queues in different regions from SNS. 2014-11-16 18:42:53 -05:00
Steve Pulec
aa7233a2db Fixes for multi-region SQS. 2014-11-16 17:57:46 -05:00