Transcript MSMQ 3.0 New Features

New Features of Message
Queuing in Windows
.NET Server 2003
Shai Kariv
Program Manager
MSMQ
Microsoft Corporation
Before We Start…


Win06: Tips & Tricks, later today!
MSMQ background:


MSMQ 1.0 in Windows NT 4.0
MSMQ 2.0 in Windows 2000



MSMQ 3.0 in XP and .NET Server 2003




Platform integration: AD, Clustering, MMC
Add-on services like Triggers
Internet messaging
1-to-many messaging
Integrating Triggers, extending Admin API
System.Messaging
Agenda

Reliable 1-N Messaging



Reliable Internet Messaging





Distribution Lists
Reliable Multicast
Messaging Over SOAP/HTTP
Firewalls and Load Balancing
Admin API
Deployment and Storage
Triggers
Distribution Lists
Overview



Software “fan-out”
Pub/Sub primitives
End-to-end semantics for




Delivery guarantees
Security
Transactions
Implementation: 1-1 “internally”
Distribution Lists
Public DLs


Registered in Active Directory
Leverage AD Group object



Programmable via ADSI calls
Manageable via AD MMC snap-ins
New format name: DL=<GUID>

Elements can be other DLs
Dim dest as New MSMQDestination
dest.FormatName = “DL=<GUID>”
Message.Send dest
Distribution Lists
Private DLs

Client-managed “DL”:



List of format names
Including new format names: DL, HTTP
Non-discoverable
Dim dest as New MSMQDestination
dest.FormatName =
“DIRECT=OS:host1\q”,”PRIVATE=…”
Message.Send dest
Distribution Lists
Transactions


Identical to queue transactions
Multiple transactions to same DL: InOrder delivery


If sent from the same computer
No in-order guarantee for messages
sent to different DLs

Even if DLs contain the same queues
Distribution Lists
Agenda

Reliable 1-N Messaging



Reliable Internet Messaging





Distribution Lists
Reliable Multicast
Messaging Over SOAP/HTTP
Firewalls and Load Balancing
Admin API
Deployment and Storage
Triggers
Reliable Multicast
Overview


Hardware “fan-out”: Efficient send to
multiple recipients
Expose and leverage IP multicast




MSMQ provides offline, persistency
Provide building blocks for Pub/Sub
Scenario: real-time market data push
(financial sector)
Protocol: Pragmatic General Multicast
Reliable Multicast
PGM: Pragmatic General Multicast

Joint Microsoft/Cisco effort



Reliable multicast transport



Public specification
Cisco routers provide optimization
Receiver either gets all transmission and
repair data
Or able to detect unrecoverable data loss
Members may join/leave at any time
PGM Data Flow
PGM host sends a multicast packet
R
Sender
S
R
Receiver
R
R
R
= Multicast ODATA
PGM Data Flow
A recipient sends (unicast) NAK
R
Sender
S
R
Receiver
R
R
R
Unicast NAK
PGM Data Flow
Router multicasts NCF in LAN
R
Sender
S
R
Receiver
R
R
R
= Multicast ODATA
PGM Data Flow
Router unicasts NAK back to sender
R
Sender
S
R
Receiver
R
R
R
PGM Data Flow
Sender multicasts repair data
R
Sender
S
R
Receiver
R
R
R
= Multicast ODATA
Reliable Multicast
Configuration and Security

Receiver configuration:



Sender configuration:



New queue property: MulticastAddress
A queue can join one multicast group
Max send rate
Max time the repair data is available
Security:


Existing authN/authZ (per queue)
Encryption only at application level
Reliable Multicast
Programming model

Receiver side:



New queue property: MulticastAddress
can be set programmatically…
When creating the queue or later on
Sender side:


New format name: MULTICAST=<IP>
No additional end-to-end semantics
Dim dest as New MSMQDestination
dest.FormatName = “MULTICAST=<address:port>”
Message.Send dest
Reliable Multicast
Agenda

Reliable 1-N Messaging



Reliable Internet Messaging





Distribution Lists
Reliable Multicast
Messaging Over SOAP/HTTP
Firewalls and Load Balancing
Admin API
Deployment and Storage
Triggers
Reliable Web Messaging
Goals





Same programming model
Use HTTP as transport
SOAP-based protocol
Secure messaging across firewalls
Support Network Load Balancing
Reliable Web Messaging
Programming model

Open queue, send message




Queue can be HTTP-addressed
New format name:
DIRECT=HTTPS://host/MSMQ/q
Open queue, receive message
New message properties leverage
SOAP support


SoapHeader, SoapBody
SoapEnvelope, CompoundMessage
Reliable Web Messaging
SOAP Reliable Messaging Protocol

SRMP: Specification for generalpurpose reliable messaging




Published specification
Extension of WS-Routing
Implemented by MSMQ 3.0
SOAP header is used to provide endto-end information needed for reliable
messaging
SRMP Message Elements
WS-Routing:
Properties
path
Stream
Services
Security
Receipts:
Delivery
Commitment
SOAP
Header
Stream
SOAP Body
SOAP
Body
SRMP Path, Propeties
<SOAP-ENV:Header xmlns=http://schema.xmlsoap.org/srmp
xmlns:rp="http://schemas.xmlsoap.org/rp/“>
<rp:path SOAPENV:mustUnderstand=“1”>
<rp:action> MSMQ:Message </rp:action>
<rp:to> http://server/something </rp:to>
<rp:id>
uuid:2288926@ac3fd49c-e7d5-4354-ba8d-3e13fc6f677c
</rp:id>
<rp:relatesTo>
uuid:2288926@ac3fd49d-e7d5-4354-ba8d-3e13fc6f677c
</rp:relatesTo>
</rp:path>
<properties SOAP-ENV:mustUnderstand="1“>
<expiresAt>20380119T031407</expiresAt>
<sentAt>20010509T130502</sentAt>
</properties>
</SOAP-ENV:Header>
SRMP Services
<services SOAP-ENV:mustUnderstand="1">
<durable/>
<deliveryReceiptRequest>
<sendTo>
http://www.we-love-books.org/receipts
</sendTo>
</deliveryReceiptRequest>
<commitmentReceiptRequest>
<sendTo>
http://www.we-love-books.org/deliverydone
</sendTo>
<negativeOnly/>
</commitmentReceiptRequest>
</services>
SRMP Streams
<stream SOAP-ENV:mustUnderstand="1">
<streamId>
uuid:84b9f5d0-33fb-4a81-b02b-5b760641c1d6
</streamId>
<current>1</current>
<start>
<sendReceiptsTo> http://sender/streamAcks/ </sendReceiptsTo>
</start>
</stream>
<stream SOAP-ENV:mustUnderstand="1">
<streamId>
uuid:84b9f5d0-33fb-4a81-b02b-5b760641c1d6
</streamId>
<current>7</current>
<previous>5</previous>
</stream>
SRMP Extensibility
MSMQ support for extensibility

MSMQ creates the SRMP elements


Receive-side applications can access
the message as XML document


When destination format name is URL
SoapEnvelope, CompoundMessage
Send-side applications can extend the
SOAP message

SoapHeader, SoapBody
Internet Messaging /
SRMP
Reliable Web Messaging
Redirecting SRMP messages

HTTP requests can be redirected



SRMP must support redirection
Redirecting SRMP messages requires
store and forward
MSMQ 3.0 supports store, redirect and
forward


Using XML files for “mapping”
Message content does not change
Reliable Web Messaging
Mapping file

<from>


External (known) name of the destination
<to>

Local resolution for the destination
<redirections>
<redirection>
<from>https://z.com/msmq/orders</from>
<to>https://server1/msmq/q1</to>
</redirection>
</redirections>
Reliable Web Messaging
Routing SRMP messages



Enterprise: MSMQ uses Active
Directory for dynamic routing
MSMQ does not use Active Directory to
resolve DIRECT format name
Redirect mapping allows
connectionless SRMP messaging

Useful for workgroup mode as well
Sender
MSMQ
host
Mapping table
Receiver
Internet Messaging /
Mapping
Agenda

Reliable 1-N Messaging



Reliable Internet Messaging





Distribution Lists
Reliable Multicast
Messaging Over SOAP/HTTP
Firewalls and Load Balancing
Admin API
Deployment and Storage
Triggers
Secure Web Messaging
Requirements




Do not allow direct TCP sessions from
Internet to Intranet
Do not expose names of internal
resources: computers, queues
Do not store sensitive data on
computers in the DMZ
Do not forward unauthenticated
messages from DMZ to Intranet
Secure Web Messaging
Topology
Internet
Intranet
DMZ
<sendReceiptsTo>
https://your_company.com/acks
https://my_company.com/orders
HTTP
Proxy
http://my_company.com/receipts
</sendReceiptsTo>
MSMQ
Server
MSMQ
Client
Authenticate
Map
Store
Forward
Firewall
https://ic/msmq/InQueue
Firewall
Reliable Web Messaging
NLB Overview

Network load balancing solutions:




Hardware (e.g. Cisco LocalDirector)
DNS round-robin
Windows Network Load Balancing
NLB assumes state-less services


Seamless for low QoS messaging apps
Special solution required for high QoS
messaging apps
Reliable Web Messaging
NLB: Low QoS messaging
Node1
My_MSMQ_App
Node2
NLB
My_MSMQ_App
DIRECT=HTTP://NLB_URL/MSMQ/q1
or
DIRECT=TCP:<NLB_IP>\q1
(works also on Windows 2000)
Node3
My_MSMQ_App
NLB Cluster
Reliable Web Messaging
NLB: High QoS messaging
Node1
Q1->client1\q1
Q2->client2\q9
MSMQ Client1
App_inqueue
NLB
Node2
Q1->client1\q1
Q2->client2\q9
MSMQ Client2
Node3
Q1->client1\q1
Q2->client2\q9
NLB Cluster
App_orders
Agenda

Reliable 1-N Messaging



Reliable Internet Messaging





Distribution Lists
Reliable Multicast
Messaging Over SOAP/HTTP
Firewalls and Load Balancing
Admin API
Deployment and Storage
Triggers
Admin API
Overview

Programmable control of MSMQ
objects:






MSMQ service
Application queues
Internal transmission queues
Local and remote
COM object model
Add-on available for MSMQ 2.0
Admin API
Extension to existing objects

MSMQApplication extensions includes:









ActiveQueues
PrivateQueues
DirectoryServiceServer
IsConnected
UsedQuota
Connect
Disconnect
Machine (can manage remote hosts)
MSMQQueue extension:

Purge
Admin API
New objects

MSMQManagement: base class for outgoing and target
queues





MSMQOutgoingQueueManagement






Machine
FormatName
MessageCount
QueueType (private, public, …)
NextHops (routing info)
EodGetSendInfo (transaction info)
EodResend (transactional protocol)
Pause
Resume
MSMQQueueManagement


UsedQuota
EodGetReceiveInfo (transaction info)
Admin API
Code example

Pause an outgoing queue on
“RemoteMachine”, inspect its content
and resume it.
Dim qmgmt as New MSMQManagement
qmgmt.Init Machine:=“RemoteMachine”,
PathName:=“TargetMachine\q”
Dim outqmgmt as MSMQOutgoingQueueManagement
Set outqmgmt = qmgmt
outqmgmt.Pause
MsgBox “messages in q:” & outqmgmt.MessageCount
outqmgmt.Resume
Agenda

Reliable 1-N Messaging



Reliable Internet Messaging





Distribution Lists
Reliable Multicast
Messaging Over SOAP/HTTP
Firewalls and Load Balancing
Admin API
Deployment and Storage
Triggers
Storage & Deployment
Enterprise deployment

Simplify MSMQ requirements in Active
Directory environment:



No MSMQ servers on domain controllers
MSMQ clients access Active Directory
directly via LDAP
Old requirements still apply for NT 4.0 and
Windows 2000 clients
Storage & Deployment
No deployment on DC
Existence &
Authentication
Any
Domain
Controller
Open
queue on
recipient
for send
MSMQ
Sender
Recipient
Storage & Deployment
Unlimited storage

Problem: MSMQ 1.0/2.0 message store:
2 GB limitation




Win64 doesn’t solve
Goal: lift the 2 GB limit
Implementation: Extend the current
MSMQ proprietary message store
Non-goal: Solve the 4 MB message size
limit
Agenda

Reliable 1-N Messaging



Reliable Internet Messaging





Distribution Lists
Reliable Multicast
Messaging Over SOAP/HTTP
Firewalls and Load Balancing
Admin API
Deployment and Storage
Triggers
Triggers
Overview


Problem: Generic queue listener code
is frequently rewritten
Solution:


High performance, built-in, 24x7 listener
service to invoke components on
message arrival
Invokes EXEs or COM components


.NET components need to be wrapped as COM
components
No need to call MSMQ API
Triggers
New features




Integrated setup
Integrated administration
Integrated clustering model
(active/active)
Message retrieval:



Peek
Receive
Transactional receive
Triggers
More Information…




Win06: Tips & Tricks, later today!
MSMQ web site:
www.microsoft.com/MSMQ
MSDN
Call-To-Action: Check out
System.Messaging in the .NET
Framework SDK
© 2002 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.