Jpleger/sqs by jeanpatrickleger · Pull Request #30 · Project-MONAI/monai-deploy-messaging

jeanpatrickleger · 2022-06-14T14:18:56Z

Fixes # .

Description

This is a pull request for AWS SQS ( Simple Queue Service ) support. This proposes an alternative option to using RMQ. The documentation about the activation of the plugin and its configuration can be found here : https://github.com/Project-MONAI/monai-deploy-messaging/blob/jpleger/SQS/src/Messaging/SQS/README.MD

Status

Ready/Work in progress/Hold

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
All tests passed locally by running ./src/run-tests-in-docker.sh.
Documentation comments included/updated.
User guide updated.
I have updated the changelog

Publish & close release only on main branch.

Target .NET Standard 2.1

* Implement unit tests for RabbitMQ plugin * Fix sonar code smells * Update PR template Signed-off-by: Victor Chang <vicchang@nvidia.com>

codecov · 2022-06-14T14:38:13Z

Codecov Report

Merging #30 (ed2f2b5) into release/0.1.0 (ffcf29a) will decrease coverage by 27.11111%.
The diff coverage is 53.31412%.

@@                   Coverage Diff                    @@
##           release/0.1.0         #30          +/-   ##
========================================================
- Coverage       93.77778%   66.66667%   -27.11112%     
========================================================
  Files                 19          36          +17     
  Lines                450        1065         +615     
  Branches              15          70          +55     
========================================================
+ Hits                 422         710         +288     
- Misses                24         343         +319     
- Partials               4          12           +8

Flag	Coverage Δ
unittests	`66.66667% <53.31412%> (-27.11112%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/Messaging/Common/Log.cs	`100.00000% <ø> (ø)`
.../Messaging/Configuration/ConfigurationException.cs	`50.00000% <ø> (+25.00000%)`	⬆️
src/Messaging/Messages/MessageBase.cs	`100.00000% <ø> (ø)`
...rc/Messaging/RabbitMq/RabbitMqConnectionFactory.cs	`0.00000% <0.00000%> (ø)`
src/Messaging/SQS/ConfigurationKeys.cs	`0.00000% <0.00000%> (ø)`
src/Messaging/SQS/Log.cs	`0.00000% <0.00000%> (ø)`
src/Messaging/SQS/QueueFormatter.cs	`0.00000% <0.00000%> (ø)`
src/Messaging/SQS/SQSMessagePublisherService.cs	`0.00000% <0.00000%> (ø)`
src/Messaging/SQS/SQSMessageSubscriberService.cs	`0.00000% <0.00000%> (ø)`
src/Messaging/Common/MessageConversionException.cs	`25.00000% <25.00000%> (ø)`
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffcf29a...ed2f2b5. Read the comment docs.

* Rename Message to Event used in the message body. * Add validation to events and validate event messages on all properties recursively. Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

- Add RunnerCompleteEvent - Rename TaskCompleteEvent to TaskUpdateEvent with new fields and changes. - Allow multiple inputs/outputs for TaskDispatchEvent - Add Accepted TaskStatus Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

* Ability to convert Message to JsonMessage<T> Signed-off-by: Victor Chang <vicchang@nvidia.com> * Add SubscribeAsync with a Func<> callback Signed-off-by: Victor Chang <vicchang@nvidia.com> * Update unit test for SubscribeAsync Signed-off-by: Victor Chang <vicchang@nvidia.com> * Use existing RabbitMQ connection if available, otherwise, create a new one. Signed-off-by: Victor Chang <vicchang@nvidia.com> * Fix code smells Signed-off-by: Victor Chang <vicchang@nvidia.com> * Breaking: IRabbitMqConnectionFactory to CreateChannel and grantee a single connection is used Signed-off-by: Victor Chang <vicchang@nvidia.com> * APIs to bind multiple topics to a single queue Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

* Rename RunnerCompleteEvent to TaskCallbackEvent with additional Metadata property. * Handle corrupted messages * Check message properties * Use RabbitMQ Timestamp * Declare exchanges are durable & !auto delete * Fix JsonProperty mapping * Include metadata in the TaskUpdateEvent Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

* Add intermediate storage for TaskDispatchEvent * Implement ICloneable for Storage Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

…ge (#22) Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

* RMQ connection support for SSL/TLS (no certificate check). * Refactored RabbitMqConnectionFactory to support TLS connection and Port number config. * Update on IRabbitMqConnectionFactory Summary. Co-authored-by: JP Leger <jpleger@amazon.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: JP Leger <jpleger@amazon.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

mocsharp and others added 14 commits April 5, 2022 16:45

Update release job

2bb7035

Skip publishing duplicate nuget packages

367882e

Requires manual trigger to deploy to NuGet.org

05376c0

Update ci.yml

a852ec7

Publish & close release only on main branch.

Update release job

da1c02b

Skip publishing duplicate nuget packages

bfafff0

Requires manual trigger to deploy to NuGet.org

529d9c5

Change target framework to .net standard 2.1

5fff089

Merge branch 'release/0.1.0' into vchang/netstandard

d1450c8

Merge pull request #2 from Project-MONAI/vchang/netstandard

776b217

Target .NET Standard 2.1

Update GitVersion.yml

f90bf2e

Update GitVersion.yml

649c656

Update ci.yml

fd4e2f6

[Breaking] Unit tests for RabbitMQ plugin (#4)

ffcf29a

* Implement unit tests for RabbitMQ plugin * Fix sonar code smells * Update PR template Signed-off-by: Victor Chang <vicchang@nvidia.com>

mocsharp and others added 15 commits June 14, 2022 09:41

Add subscription no requeue reject (#9)

5a314d3

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Add additional failure reasons (#11)

7fe7ce0

Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Update FailureReason.cs

2402d20

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Update TaskUpdateEvent.cs

19b141d

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

update task dispatch event

6436c66

Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Make credentials optional in TaskDispatchEvent.Storage (#17)

50c66fc

Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

change task status name

bdb5be7

Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Rename workflow instance id (#20)

f637151

Signed-off-by: jackschofield23 <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Intermediate Storage for TaskDispatchEvent (#21)

5f360a7

* Add intermediate storage for TaskDispatchEvent * Implement ICloneable for Storage Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Throw MessageConversionException on error converting JSON event messa…

64437ab

…ge (#22) Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

update taskupdate to include output artifacts

310d005

Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

jackschofield23 and others added 10 commits June 14, 2022 09:41

update task callback event (#24)

1761aad

Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

fix task callback and task update event (#25)

8f63ccd

Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

add payload id to task dispatch (#26)

b076491

Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

add export status (#28)

c167be8

Signed-off-by: Jack Schofield <jack.schofield@answerdigital.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Added SQS plug-in as RMQ alternative.

d0ba49e

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

SQS Support adde to messaging lib

bc71ebd

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Code analysis fixes.

4edabf0

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

remove unecessary folder.

8cdb394

Signed-off-by: JP Leger <jpleger@amazon.com> Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

problematic README.MD removed

75973f0

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

jeanpatrickleger force-pushed the jpleger/SQS branch from 4a642dc to 75973f0 Compare June 14, 2022 14:41

jeanpatrickleger added 3 commits June 14, 2022 09:43

README.MD added back.

fad266c

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Code smells corrections.

4d930e8

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Code smells corrections.

fa737a0

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

jeanpatrickleger requested a review from mocsharp June 14, 2022 15:54

jeanpatrickleger added 5 commits June 14, 2022 11:00

Code smells corrections.

9fe3d1b

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Code smells corrections.

674029f

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Code smells corrections.

47781e1

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Code smells corrections.

2e5ab53

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

Code smells corrections.

ed2f2b5

Signed-off-by: JP LEGER <jeanpatrick.leger@gmail.com>

mocsharp force-pushed the release/0.1.0 branch 2 times, most recently from e08d9a2 to 6b0d721 Compare June 27, 2022 22:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jpleger/sqs#30

Jpleger/sqs#30
jeanpatrickleger wants to merge 47 commits intorelease/0.1.0from
jpleger/SQS

jeanpatrickleger commented Jun 14, 2022

Uh oh!

codecov bot commented Jun 14, 2022 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jeanpatrickleger commented Jun 14, 2022

Description

Status

Types of changes

Uh oh!

codecov bot commented Jun 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Jun 14, 2022 •

edited

Loading