Inside Microsoft: Transferring Textual and Binary Data using SOAP and DIME

Transferring Textual and Binary Data using SOAP and DIME

Nov 22, 11:00 pm

Article Author: Vimal Ghosh V

Introduction

In many distributed applications there is a requirement for messaging. The data transfer could involve simple form posts or it could involve complex binary data exchange. In this article I’ll look at a method to pass binary and textual data over a network using Simple Object Access Protocol (SOAP) and Direct Internet Message Encapsulation (DIME). The process also involves Web Services Enhancements (WSE 2.0). The beauty of SOAP lies in its ability to encapsulate XML data within a SOAP message. This gives SOAP the flexibility to contain data from any XML schema, which is a powerful capability. The DIME specification defines a mechanism for packaging binary data with SOAP messages, so that when sending data that is not in XML format, applications need not be constrained by the SOAP specification. I will also discuss the way DIME is structured and explain some strategies that will allow you to efficiently transfer data using WSE. I will concentrate on building a sample application to demonstrate the capabilities of DIME and how it fits well with Web Services and how it can work closely with SOAP and WSE.

After reading this article you should:

Understand the basics of DIME

Be able to use WSE 2.0 to transfer data

Understand how SOAP messages work with DIME

Know how to extract DIME data from SOAP messages at the Web Server.

There is also a discussion on DIME and MIME (Multipurpose Internet Mail Extensions)

System Requirements

To run the sample application for this article you should have

IIS 5.0 or later running on Windows 2000 or later

The .NET Framework version 1.1 or later

VS .NET 2003

MSDE 2000 /SQL Server 2000

Web Service Extensions 2.0 Service pack 3.0.

You can download WSE 2.0 at http://msdn.microsoft.com/webservices/building/wse/default.aspx. For details on how to setup WSE 2.0 and its different capabilities please refer the online help manuals

Also it is assumed that the reader is comfortable with the basics on Web Services and messaging based on SOAP. Don’t worry about the details on how these things are implemented – I’ll will go little deeper when it is required during the course of this article. Again it’s all are laid out in perceptive of the article structure. But if you want to know about the details on these technologies please refer your MSDN manuals or official web sites.

Installing and Compiling the Sample Code

Included with this article is a link to download all of the SQL scripts, VB .NET code and Web Service code. To set it up, simply unzip the files onto your local drive. Look for the Readme.txt which will have all the information on how to setup the application. The application is extremely easy to follow once you’ve loaded everything from the zip file.

Data transfer possibilities and drawbacks

Let’s consider the options for transferring textual and binary data from a client application to a remote server. We all know industry is moving to an XML-centric world. When it comes to data transfer the obvious protocol is XML based. But you have to understand that there are plenty of financial and insurance industry related legacy systems which don’t speak XML, and they may never do so. So this reduces the options if want to interoperate with legacy systems.

In that case you may have to look at custom applications written to convert legacy formatted data to XML and back. But in that case you may be loosing the intrinsic efficiency of the legacy format, besides the overhead of the mechanism to serialize and deserialize the data. There are also situations where the act of serializing data into XML will cause other inefficiencies. In particular, there are cases where serializing data into XML is unwise because an efficient binary compression scheme is already in place. To quote an example, the instances of image files that can come in a number of different formats but tend to be transferred across the Web in mostly JPEG or GIF formats. Many images are large enough as it is, and the processing required to serialize the data to and from XML could give a horrendous performance loss. Although one could define a Web service method taking binary attachment as a parameter, this carries overhead since the data is encoded as xs:base64 or xs:hexbinary (embedded or referenced), adding lot of bloat to the message. Furthermore, the problem with encoding is that it requires decoding at the other end, thus making the process slow there too, and for referenced data, it is also inefficient for XML parsers to resolve references and extract the attachment while parsing the SOAP message.

SOAP with Attachments is the original W3C recommendation to define how SOAP messages are carried within a MIME "multipart/related" message, as a compound SOAP document. The first MIME part is the SOAP envelope, subsequent parts contain attachments. SOAP with Attachments addressed the problem of reducing payload overhead, but did not provide a formal way to associate attachments with the SOAP message because of the arcane requirements of MIME. Also, there are problems when dealing with SOAP messages which are digitally signed: Any changes to the data will invalidate the signature – this basically means you can’t convert the data to XML alongside a SOAP request in most cases when there is digital signature involved.

Introducing DIME

Enter Direct Internet Message Encapsulation ( DIME) as the solution. You can use the specifications of both DIME and WS-attachments to attach binary files and transfer them efficiently over the wire.

DIME is a lightweight, binary message format, defined in an Internet Engineering Task Force (IETF) specification that was submitted by Microsoft and IBM, for the encapsulation of application-defined payloads of arbitrary type and size into a single message construct. DIME addresses the difficulties involved in embedding binary data into XML documents. It is similar to MIME, but is an alternate binary message format that supports the compilation of a message with arbitrary types. Rather than a content type multipart/related it is packaged as application/dime so that those receiving the message can parse it accordingly. WS-Attachments describes how to encapsulate SOAP message in DIME

In the subsequent sections of this article you will see how you can generate a DIME message and pass it across to a Web Service. It is sometimes necessary for a Web service to send a large file of text or binary data, such as an image file, in a SOAP message. The DIME protocol solves this problem by defining a mechanism for placing the entire contents of the original file outside the SOAP envelope, eliminating the need to serialize the file into XML.

Basically, DIME is a light weight binary message format for sending and receiving messages, and is ultimately just a specification for including multiple binary records within a single package. It has the ability to contain any kind of data, including image files, SOAP messages, or even MIME messages. Specifications enforce no restriction on the size or format of the data. While sending the data one need not know the length of the total data being sent when using DIME. One should examine the DIME message to understand the information contained in each fields.

DIME Message Structure

A DIME message consists of one or more DIME records . A DIME record contains a payload described by a Type , a TypeFormat , a ContentLength , and an optional Id . The DIME record header stores information about these and other record and payload properties. The first record and last record in a DIME message are respectively marked with a MB ( Message Begin ) flagset and a ME ( Message End ) flagset. The minimum message length is one record, which is achieved by setting both the MB and the ME flag in the same record. The maximum number of DIME records that can be carried in a DIME message is unbounded. Figure 1 explains the layout in detail.

Figure 1. DIME message structure

The record layout format is as shown in Figure 2.

Figure 2. DIME record format

The order of transmission of the DIME record shown in Figure 2 above is shown to the octet level, with the bits of the octets listed along the top of the figure. The figure shows a group of octets, and the order of transmission of those octets is first left to right and then top to bottom, as they are read in English. You’ll deduce from the figure that the DIME record layout is in general has a length which is a multiple of 4 octets.

The fields in a DIME record as per the figure are

VERSION- The version mask is a 5 bit unsigned integer that must be set to the value of 1, i.e., binary 00001. So if you are developing a parser then the message field should conform to version 1

Message Begin (MB) – This field indicates the start of a DIME message and it is a 1 bit field

Message End (ME) – This field indicates the end of a DIME message and it is a 1 bit field

Chunk Flag (CF) – The chunked flag is a single bit that is set to 1 for the first record referencing chunked data as well as all following records that include the chunked data.

TYPE_T – The TYPE_T field value indicates the structure and format of the value of the TYPE field. The possible values include

Name	Value	Meaning
UNCHANGED –	0×00	Exclusively used by all records that are chunked beginning with 2nd chunked record and used through the last chunked record. When it is used the TYPE_LENGTH is required to be 0.
MEDIA_TYPE	0×01	Indicates the TYPE field contains a value conformed to items defined by RFC 2616
ABSOLUTE_URI	0×02	Indicates the TYPE field contains a value conformed to items defined by RFC 2396
UNKNOWN	0×03	Indicates that the type of the DATA field is unknown. The TYPE_LENGTH field is required to be 0. Receiving parser uses this only for storing data not for processing.
NONE	0×04	Indicates that no TYPE or DATA field is used. The values of the TYPE_LENGTH and DATA_LENGTH fields are required to be 0. This value can be used whenever an empty or terminating record is used.

RESRVD – This 4-bit integer always holds the value of 0×0.

OPTIONS_LENGTH – An unsigned 16 bit integer that specifies the length in octets of the OPTIONS field, excluding any padding used to achieve a 4 octet alignment of the OPTIONS field.

ID_LENGTH – The id length field is a 16 bit integer that indicates the length of the ID field in terms of octets, excluding any padding.

TYPE_LENGTH – An unsigned 16 bit integer that specifies the length in octets of the TYPE field excluding any padding used to achieve a 4 octet alignment of the TYPE field

DATA_LENGTH – The data length field is a 32 bit integer that indicates the length of the DATA field in terms of octets, excluding any padding. Since the data length is a 32-bit field, it specifies a maximum data size of 4 gigabytes (GB). This is a potentially limiting restriction on the size of the data that may need to be packed into a DIME data record. Fortunately, DIME has an excellent solution for avoiding the data size limitation “chunking. Please refer DIME specifications for further details.

OPTIONS – Its rarely used field. DIME has provisions for carrying additional information in a DIME record as option elements. A DIME record (including a record chunk) can carry zero or more such option elements each containing information about the payload or information which otherwise may be of benefit to a DIME parser.

ID – This is an identifier in the form of URI. This is required to uniquely identify the record

TYPE – This is an identifier in describing the type of the payload. Also the length of the TYPE field must be a multiple of 4 octets.

DATA – The DATA field carries the payload intended for the DIME user application. Any internal structure of the data carried within the DATA field is opaque to DIME

Further details on this can be found at the IETF site. Links given at the end of the article will give you enough information to explore the DIME specification in detail.

So far you have seen how DIME is organized, but not how to write using DIME. In fact this is pretty easy with .NET. I’ll now discuss on how you can make use of DIME in your client programs to upload data which includes binary and textual information.

Comparing DIME and MIME

MIME extends the format of Internet mail to allow non-US-ASCII textual messages, non-textual messages, multipart message bodies, and non-US-ASCII information in message headers. The SOAP Messages with Attachments specification basically indicates that you can package SOAP requests with related data in a MIME message. You’re probably aware that MIME allows you to package data records in single message, in a similar manner to DIME. So why was DIME was created when at first sight it looks like there was already a solution for the problem. In fact there are a number of reasons for arguing that DIME is technically superior – as explained in the following table:

MIME	DIME
Data record separation is by unique string identification. Requires scans through the entire message to find a map	The data record headers hold the information on the data boundary and length of the record
Message partitioning may break if record data contains similar separator string	Unique id identifies each record and record header holds information required
Memory allocation for message received is not straightforward. Incoming buffers will be allocated without knowledge of how much data is being received	Data length is known before hand and allocation can be done accordingly.
Specifications allow adding any MIME header fields that one want to a MIME message. This makes processing complicated especially for Web Service and SOAP based applications.	Specification has a small, fixed set of headers that provide the functionality required by a SOAP message.
Limits its content-type specifications to the text/html type of syntax	Supports both the MIME content-type mechanism, as well as content types specified with the URI mechanism.
Encoding of binary data is a requirement in many cases	Does not require any sort of encoding of binary data.

The MIME approach to sending SOAP messages with attachments is a technically feasible solution. However, supporting multiple specifications for achieving the same results is costly and potentially confusing. Due to its technical superiority, Microsoft is promoting DIME for binary attachments and future SOAP tools and platforms.

The Sample Application

The sample application that you are going to see is a simple File upload utility. The user can upload multiple files along with the filename to a Web server using DIME and SOAP. The application comprises a VB .NET client which consumes a Web service. The Web service is built with methods which help in the transfer of data between these applications and store the information in an SQL server database. In real scenario this setup could be extended to have data transfer between disparate applications. Client applications built on PDAs, Mobile phones and Laptop devices communicating with big enterprise applications is not a complex scenario any more.

Application Structure

Let’s examine the structure of the sample application under consideration. You will see how this setup can

Create a custom UI to get data from the user

Attach Binary information to SOAP Message

Create a Web Service application to receive data from the user

Detach binary information from the SOAP Message

Insert the data into a SQL Server 2000

The data received in the SQL Server database could be consumed by other applications. But these details are out of scope of this article. The pictorial representation of what you are going to see is as given in Figure 3.

Figure 3. Sample application architecture

Creating the Client Project

Create a VB.NET Windows application named DIMETest . Rename the form added by default to DIMETest.vb . Now you need to create a user interface. The client GUI mainly consists of a select screen where you can select different files from the hard disk and upload to a server. The details on the server side and the Web Services will be discussed in the subsequent sessions. Add labels, text boxes, on the form, and set the Text and Font properties so that the form appears as shown in Figure 4.

Figure 4. Selecting the files to upload

The GUI above looks very straightforward and self explanatory. You have five textbox fields and corresponding browse buttons. By clicking on each browse button the user can choose a file from the local disk and the path to the file will be displayed in the UI. You will be using the FileDialog control to help the user in choosing a particular file from the hard disk. Add a FileDialog control from the Toolbar to implement this feature. Finally you have two buttons named Upload and Close which would help in the upload of the data and terminating the application respectively. Now let’s look at how the browse button is implemented.

Private Sub BrowseButtonHandler(ByVal sender As System.Object, _ ByVal e As System.EventArgs) Handles _ Button1.Click, _ Button2.Click, _ Button3.Click, _ Button4.Click, _ Button5.Click Try With FileDialog ‘Show the Dialogbox for choosing the File .ShowDialog() ‘Based on the button set the File Name in the Textbox Select Case sender.name Case "Button1" TextBox1.Text = .FileName Case "Button2" TextBox2.Text = .FileName Case "Button3" TextBox3.Text = .FileName Case "Button4" TextBox4.Text = .FileName Case "Button5" TextBox5.Text = .FileName End Select End With Catch ex As Exception MessageBox.Show(ex.Message, "DIME Test Error") End Try
End Sub

The ShowDialog() function of the FileDialog control will popup the Open Dialog window which can be used to select a particular file. The function is written in such a way that all browse button logic can be handled in one single routine. The routine will identify a particular button click from the sender.Name property. Once the button is identified the corresponding textboxes will be filled with the full path of the file selected. Here FileName property of the FileDialog is used. The logic is implemented inside an exception handler routine which ensures that if in case there is an error it will be handled and notified to the user appropriately.

Uploading Data

Now let’s explore the Upload button. This button is actually used to upload data using a Web Service call. I’ll explain how this really works. Before discussing any more about how this is implemented you step back a little to understand how the capabilities of DIME is exposed to a .NET programmer. DIME will not be available unless you install WSE (Web Service Enhancements) 2.0. You might be wondering what this WSE is.

Web Services Enhancements for Microsoft .NET (WSE) is a supported add-on to Microsoft Visual Studio .NET 2003 and the Microsoft .NET Framework, and provides developers the latest advanced Web services capabilities to keep pace with the evolving Web services protocol specifications. With WSE 2.0, you can secure your messages using digital signatures and encryption, send attachments, route messages through intermediaries, and more – so it helps you in sending attachments with SOAP messages. WSE 2.0 is downloadable from the Microsoft site and a link is provided with the related links for this article.

Creating the Web Service

Let’s go back to the upload function UploadButton_Click . If you observe closely you have an object created in the function that looks like this.

Dim DimeProxy as New DimeService.DIMEWse

It is basically the Web Service object proxy. How do you get this? You need to add a Web Reference. But if you want a Web Reference then you need to have a Web Service already in place. But did you create one? Yes you are right, you did not create one, but this was done deliberately in order to avoid deviating from the task at hand. For now let us assume that you have added a Web Reference of a Web Service named DIMEService, which is located at http://localhost/DIMEService/DIME.asmx . You can add a reference by using the VS IDE menu item Project -> Add Web Reference. You will come back to the Web Service implementation part a little later. Or you could use the code download from the zip file attached which would contain the source code for the Web Service implementation.

Once you have added the Web Service you need to perform some important steps to keep it working for your requirements. The point to note here is that if you have previously installed the WSE Settings tool on the client, Add Web Reference will automatically generate a separate proxy class ending with Wse that already inherits from the proper WSE class. In that case, you can just use the DIMEWse class as the service proxy. If you have not done that then follow the steps given below

First of all find out the References.vb file which is created by the Visual Studio when you add the Web Reference. The References.vb is located by expanding the Web References in your Solutions Explorer as shown in Figure 5.

Figure 5. Finding the References.vb file

From the project add a Reference to the Microsoft.Web.Service2 Namespace. You need to have WSE 2.0 installed in the system to get this reference. In order for a Web service or a client application to receive and handle DIME messages, WSE must be installed. Also, after adding a Web reference to the DIME-based Web service, you must modify the proxy class in the References.vb file so that it inherits from the Microsoft.Web.Services2.WebServicesClientProtocol class in WSE. For example, in the References.vb the proxy class needs to be changed to inherit from Microsoft.Web.Services2.WebServicesClientProtocol as shown below.

Public Class DIMEWse Inherits Microsoft.Web.Services2.WebServicesClientProtocol

Consuming the Web Service

Now you have the full setup on the Client to consume the Web Service. Now let’s continue on the functionality implemented in UploadButton_Click. The first few lines talks about creating DIME attachments with the help of WSE routines.

Try
   ‘Create an Array to hold the DIME Attachments
   Dim DIMEAttachments As Dime.DimeAttachment ()
   ‘Call the Private function to return the DIME Array
   DIMEAttachments = CreateDIMEAttachments ()
   …

Here you create an array of DIME attachments because you could add more than one DIME attachment for Upload through the UI. So the first line is defining an array of DIME attachments. Dime.DimeAttachment is a class defined in Microsoft.Web.Services2.Dime Namespace. Essentially it specifies the header and payload data for sending and receiving DIME attachments with SOAP messages.

The array created is filled by a custom routine named CreateDIMEArray() . If you examine how this is implemented you will know that the function creates DIME attachments using the filename stored in each textboxes. It calls the function CreateDIMEAttachment() to create the DIME attachment and add it to the array of attachments as given below. The process will be repeated for all the textboxes.

…

‘Collect the File Information

‘Choose the items that has information

If TextBox1.Text <> "" Then
   Counter += 1
   ReDim Preserve DimeAttachmentArray(Counter)
   DimeAttachmentArray(Counter) = CreateAttachement (TextBox1.Text)

End If

…

Now its time to checkout the important function in the application CreateDIMEAttachment() . Here you will come to know how WSE helps us in creating DIME attachments with the physical files.

Private Function CreateAttachement (ByVal FilePath As String) As _ Dime.DimeAttachment Try ‘To store the Real DIME Attachment Dim TempDimeAttachment As Dime.DimeAttachment Dim FileName As String If FilePath <> "" Then FileName = FilePath.Substring(FilePath.LastIndexOf("") + 1) TempDimeAttachment = New Dime.DimeAttachment(GetMIMEType(FileName), _ Dime.TypeFormat.MediaType, FileName) ‘To make the Id unique the GUID is appended to the File Name ‘This is required for store different DIME attachments when passed ‘along With the SOAP message TempDimeAttachment.Id = FileName & "|" & Guid.NewGuid.ToString Return TempDimeAttachment End If Catch ex As Exception …

If you study the DimeAttachment object little closely you will come to know that it takes in three parameters as shown below. It basically specifies the header and Payload data for sending the DIME attachments.

TempDimeAttachment = New Dime.DimeAttachment(GetMIMEType(FileName), _
                                  Dime.TypeFormat.MediaType, FileName)

TempDimeAttachment.Id = FileName & "|" & Guid.NewGuid.ToString

First parameter is the MIME type of the file as a string. This information comes from a small utility function GetMIMEType() , which polls the registry to find out the MIME type of the file that is been attached. The HKEY_CLASSES_ROOT section of the registry will be polled to get the MIME type corresponding to the extension of the file. Check out the article by Todd Davis for more information on how this works.

The next parameter used is the DIME Type format. It specifies the possible formats for record payload types. This enumeration has an attribute that allows a bitwise combination of its member values. It takes in values like

AbsoluteUri -An absolute URI.

MediaType -A media type format.

None -No format is specified.

Unchanged – The same as the format for the preceding record.

Unknown – The format is not known.

In your case you use the MediaType format. The last parameter is the path of the file that needs to be attached.

There is one complication before you can use the attachment: Notice that in the above setup the user could use the same file to upload multiple times. So you need a unique Id to identify the attachment. Obviously there is no better choice than the Guid.NewGuid() routine. You create a unique Id by concatenating the filename with the Guid separated by a ~|’ (pipe). This means the web service can identify the DIME attachment.

In the background a DIME record was created and it has been attached with the proper payload information. Likewise the DIMEAttachment will be created for all the files chosen for upload.

You are now at a point where you have all the necessary information about the files to be attached. The remaining task is to embed the attachments that you have created with the SOAP message and send that to the Web service. For this the upload routine is equipped with the following.

Dim DIMEAttachments As Dime.DimeAttachment () …
‘Create an Object of the Web Service proxy object Dim DimeProxy As New DimeService.DIMEWse Dim CurrentRequestConxt As SoapContext Dim Counter As Short Dim ReturnFlag As Boolean ‘Get the current Request Context from the Proxy object CurrentRequestConxt = DimeProxy.RequestSoapContext Dim DIMEEnumerator As System.Collections.IEnumerator = _ DIMEAttachments.GetEnumerator () While DIMEEnumerator.MoveNext () CurrentRequestConxt.Attachments.Add (DIMEEnumerator.Current) End While Dim ErrorString as String ‘Call the Web Service method to upload the DIME Attachments ReturnFlag = DimeProxy.UploadFile (ErrorString) If ReturnFlag = True Then MessageBox.Show ("Upload of DIME Attachment was Successful", _ "DIME Test") Else MessageBox.Show ("Upload of DIME Attachment Failed" & _ ControlChars.CrLf & ErrorString,"DIME Error") …

First of all you get the SoapContext from the Web Service proxy object created earlier. Then each of the DimeAttachment s from the array is added to the SoapContext s attachments collection. You will repeat this for all the elements of the array which has some files attached to it. You have now completed attaching binary information to the SOAP message. Now let’s scrutinize what would have happened to the SOAP message.

You are going to look at an example where a SOAP request is being sent along with a binary file. The SOAP message details here are cut down a little for the sake of clarity.

1 0 0 0000000000000

010 0000000101001

00000000000000000000000010110110

http://schemas.xmlsoap.org/soap/envelope/

<envelope>
    <body>
      <…href= Doc1|13213143535/>
   </body>

</envelope>

…………………..Record       ..

0 0 1 0000000000110

001 0000000001010

00000000000000001111111111111111

Doc1|13213143535

application/msword
     <204 K of binary data>
     …Record        

0 1 0 0000000000000

000 0000000000000

000000000000000000011000111110000

…

The above markup represents a DIME message with data records separated for clarity by record markings. The fixed portion of the headers is represented in binary format, so that you can see the specific values of the flag fields. The ID, Type, and Data are simply displayed as text where appropriate.

You can see the ID of the binary attachment in this case is Doc1|13213143535 . This is the ID specified in the Data Record ID field of the binary portion of your DIME. Again this is created by the GUID routine that you saw bit earlier.

The ID length is non-zero, so an application will be able to determine that this data record has an ID, and it can make the comparison with the value indicated by the href attribute in the SOAP message. The type is specified with the media type flag (the leading 001 in the second row of this data record). This means that you are using a content type mechanism for identifying the sort of data you have. In this case the type field contains application/msword to indicate what kind of data is in your data field.

The second and last data record has the End Message flag set (the 1 in the second bit of the first line) and the Chunked Flag is not set (the 0 immediately following the End Message flag). Notice that there is no ID or Type field, since it was specified in the first record in the sequence. The End Message flag lets us know that this is the end of your DIME message.

The only thing left out now is to send this SOAP message to the Web Service. You will use the Web service method UploadFile to transfer the contents to the Web Service end.

  ReturnFlag = DimeProxy.UploadFile (ErrorString)
  If ReturnFlag = True Then
       MessageBox.Show ("Upload of DIME Attachment was Successful", _
          "DIME Test")
  Else
      MessageBox.Show ("Upload of DIME Attachment Failed" & _
         ControlChars.CrLf & ErrorString,"DIME Error")
  …

The Web service method would return appropriate response to the operation you carry out. If it returns a ~True’ it means that the data has been successfully stored at the server side. Otherwise a false is returned. If the value returned is ~False’ then the ErrorString can be checked to find out about the error.

Now you have actually completed everything with regard to the Client Application. You have created the File Upload Utility and used a Web Service to upload that information to the Server. But what I have not covered is the actual Web Service implementation. Let’s do that next.

DIME Service

In this section you’ll see how to create a Web Service to extract the information from the SOAP message and update the database. If you are creating the project as you go along you should use VS .NET to create the Web Service and change the name of the project to DIMEService . Also, change the name of default Service1.asmx to DIME.asmx

Next you need to configure WSE so that you could use DIME Messages. Even with WSE installed on the ASP.NET Web server, there are few additional DIME-specific configurations that need to be made to the ASP.NET application in order to use attachments with DIME. After creating the Web service project in Visual Studio .NET, a reference to the Microsoft.Web.Services2.dll assembly needs to be added to the project. Now you can use the WSE Setting 2.0 Menu item under Solution explorer to configure the SOAP extensions so that it would work for DIME messages and attachments. The property dialog is shown in Figure 6.

Figure 6. Configuring SOAP Extensions

Please check both the checkboxes so that the system works the way you want it to work. Basically this operation will create an entry for the SOAP extension in the Web.Config file

<configuration>
    <system.web>
      …
        <webServices>
            <soapExtensionTypes>
           <add type="Microsoft.Web.Services2.WebServicesExtension,
               Microsoft.Web.Services2, 
              Version=2.0.0.0, 
               Culture=neutral,
               PublicKeyToken=31bf3856ad364e35"
                priority="1" group="0" />
              </soapExtensionTypes>
        </webServices>
    </system.web>

</configuration>

The prime idea of these extensions is to have the capability of modifying the SOAP messages. When a client calls a Web method from the server, the request is serialized in to a SOAP message and sent over the network as a SOAP request. The SOAP request is deserialized at the server side. The server then composes a SOAP message, serializes it, and it sends it over the network as SOAP response. The SOAP response is deserialized at the client side before the SOAP message is received by the client application. Therefore, serialization and deserialization occur both at the client side and the server side. You could use the SOAP extension to perform various operations, before and after the serialization phases. You will see how you get the DIME attachments in the next sections. Before that you need to add a Web method so that your client could make use of it.

You can rename the default Hello World method or you could create a new method. As you can see the only method exposed by the Web Service is UploadFile . Let us implement the same

<WebMethod()> _ Public Function UploadFile(ByRef ErrorString As String) As Boolean Try Dim SoapContxt As SoapContext = RequestSoapContext.Current ~Verifies that a valid Soap Request is in place. ~Otherwise error out If SoapContxt Is Nothing Then Throw New ApplicationException("Only SOAP requests are permitted.") End If Dim DIMEAttachments As Attachments.AttachmentCollection Dim DIMEAttachment As Attachments.Attachment DIMEAttachments = SoapContxt.Attachments For Each DIMEAttachment In DIMEAttachments StoreFileInDatabase(DIMEAttachment) Next Return True Catch ex As Exception ErrorString = ex. Message Return False End Try End Function

You should ensure that the SoapContext returns something and throw an exception otherwise. If there is no value that implies some problem in the message that was passed.

Then you will use the StoreFileInDatabase() function to save the data to the database. In the code download you will find a table named DIME as part of the SQL script which is used to store the file information. The table is created in the Northwind database and details are given in the accompanying Readme.txt . This is the only table that you will make use of in the entire application.

If you examine the StoreFileInDatabase function, you would know that it is doing the exact opposite to the client application.

Private Function StoreFileInDatabase(ByVal DIMEFile As _ 
                                     Attachments.Attachment)

…
      DimeStream = DIMEFile.Stream
      Dim FileData() As Byte = New [Byte](DimeStream.Length) {}
      DimeStream.Read(FileData, 0, DimeStream.Length)                
     ‘Got the file Contents
      FileContents = FileData
      ‘Got the file Name
      TempFileInfo = DIMEFile.Id.Split("|")
     ‘First Array element corresponds to the Orginal File Name
      FileName = TempFileInfo(0)
      ‘The MIME type of the file which can be used for
      ‘Streaming the file in an ASP.NET environment
      FileType = DIMEFile.ContentType
      ‘Store the file Contents to the Database
      Dim ParmeterArray() As SqlParameter = New SqlParameter(2) {}
      Dim Dbbroker As New DataBroker
      ‘ File Name of the File 
      ParmeterArray(0) = New SqlParameter(&quot;FileName", _
                                   SqlDbType.VarChar, 50)
      ParmeterArray(0).Value = FileName
      …
      Dbbroker.OpenConnection()
      ‘Call the helper class method to store the contents to the database
      Call Dbbroker.ExecuteQuery("InsertFile", ParmeterArray)

Essentially that is the heart of the whole process to get the DIME message out and extract the binary information. The filename is derived by using the Split() function. Once you have the raw data you can make use of a Database Helper class DataBroker to update the database with the relevant information. This utility class is a lightweight version of the Microsoft supplied Database application blocks. This utility class will have functions which would help us in interacting with the database. Here the point to note is that you are actually storing the entire contents of the file rather that just the file path. A stored procedure named InsertFile will be used to do this.

Let’s discuss that part of the function which extracts out the DIME message that you have passed from the client.

      DimeStream = DIMEFile.Stream
      Dim FileData() As Byte = New [Byte](DimeStream.Length) {}
      DimeStream.Read(FileData, 0, DimeStream.Length)                
     ‘Got the file Contents
      FileContents = FileData

The routine is very straight forward and self explanatory. First of all you need to create a System.IO.Stream Object to hold the raw binary data that you are extracting from the SoapContext . Recall the attachments are each tagged with a unique key. So you can use the same key to reference the data in the DIME attachment collection. The cool thing is that the SoapContext s allow you to stream the data and hence store it in a Stream object. From the stream object it is very easy to read back the raw data. With the data received the code can send out a response flag as True (or False if something went wrong).

… For Each DIMEAttachment In DIMEAttachments StoreFileInDatabase(DIMEAttachment) Next Return True Catch ex As Exception ErrorString = ex. Message Return False End Try

Conclusion

In this article, I explained the Direct Internet Message Encapsulation (DIME) specification, which defines a lightweight, binary message format that can be used to embed binary data into SOAP messages. You looked at a sample VB .NET application that used the DIME to attach binary and textual data. You also saw how you can extract out this information at the Web Service end. Together you found how Web Services Enhancements for Microsoft .NET offers the most complete solution for sending and receiving attachments with SOAP messages. The discussion also touched briefly on DIME’s superiority over MIME.

In a real secure world you could have added more security with the current WSE extensions and could have adopted a transactional database uploading mechanism which takes care of data inconsistencies. You could also look at WSDL Extension for SOAP in the DIME specification to define XML elements and attributes for use in Web Service Description Language (WSDL) documents. All this can be taken up as future work and food for thought. So keep DIMEing.

Permanent Link

by Feature Articles