What is DICOM Storage Commitment Service and why is it needed
Storage Commitment Data Flow
All together SCM is pretty straight forward. All we need to do is to send a list of the instances and get back a reply saying which are in the PACS database and which are not, and that’s exactly how it works. Well, almost.
The above diagram that I made hopefully explains it all. On the left there’s the SCM request. It is sent using N-ACTION command with a dataset that contains:
- A Transaction UID, identifying this commit request
- A referenced SOP Sequence, DICOM Tag (0008,1199) with a list of SOP Class UID’s and SOP Instance UID’s we request to commit.
- The transaction UID from the request.
- A list of succeeded instances
- If not all were ok, a list of failed instances
The Timing of the Storage Commitment Result
Getting the SCM result can sometimes be tricky. Maybe a similar design paradigm to the one described earlier led to this. Maybe, the SCP can’t answer immediately and it needs to think about it, queue the request for some batch process, check the database, compose a reply, queue it for sending and so on. Instead of just getting the results immediately in a response, DICOM lets the SCM SCP the freedom to decide when and how to answer. The SCP should send us back a N-EVENT REPORT command with the result and this result can arrive in one of three ways:- The SCP can Send the N-EVENT-REPORT on the same association that the SCU initiated or
- The SCP can start another association to the SCU and send the N-EVENT-REPORT, or
- The next time the SCU starts an association the SCP can send the N-EVENT-REPORT immediately after the association negotiation phase.
Implementing DICOM Storage Commit SCU with RZDCX
Because the result may be coming on another association, we better have an accepter running. We don’t have to but it’s a good idea because some SCP’s will not do it any other way. In DCXACC there’s a callback named OnCommitResult that hands out the Transaction UID’s and the succeeded and failed instances lists. We can run this accepter on a different process, on a different thread or on the same thread as you’ll see in the example. To send the request you can either call CommitFiles or CommitInstances. If you call the first, RZDCX will open each file, extract the SOP Class UID and SOP Instance UID and build the request dataset. If you use CommitInstances than you have to provide the list. CommitFiles is handier though because you’re not going to delete these files before you got the commit result anyhow. These two methods will not wait for the result and hang out immediately. There’s also CommitFilesAndWaitForResult and its pair CommitInstancesAndWaitForResult that wait for a while before hanging out and gives you the same out parameters as OnCommitResult does. If your PACS support that, these would be easier. Decent PACS should have a flag that controls this behavior for every AE Title and let you select between the ways that the results are sent back to the SCU. Here’s a single threaded example. What is done here is to set an accepter and start it, then send the request, then wait for the result on the accepter. I don’t recommend doing it this way but it kind of cool as an example. The best way I think is to have the accepter run independently on another thread or process that if you implement Storage SCP (which you probably do in order to get instances back on Q/R) also handles incoming C-STORE’s.C# Test Code
The example code this time is directly from my nUnit test suite. You can download the sources from this link. All these examples do basically the same: Create some DICOM Test Files, Save them to disk, Send them using C-STORE, Check that they were stored with Storage Commitment and wait for the result.
public void CommitFilesSameThread()
{
// Create
test files
String
fullpath = "SCMTEST";
Directory.CreateDirectory(fullpath);
CommonTestUtilities.CreateDummyImages(fullpath,
1, 1);
// Send
test files
string
MyAETitle = System.Environment.GetEnvironmentVariable("COMPUTERNAME");
DCXREQ
r = new DCXREQ();
string
succeededInstances;
string
failedInstances;
r.Send(MyAETitle, IS_AE, IS_Host,
IS_port, fullpath + "\\SER1\\IMG1",
out succeededInstances, out failedInstances);
Assert.That(failedInstances.Length
== 0);
Assert.That(succeededInstances.Length
> 0);
// Commit
files and wait for result on separate association for 30 seconds
SyncAccepter
a1 = new SyncAccepter();
r.CommitFiles(MyAETitle, IS_AE,
IS_Host, IS_port, fullpath + "\\SER1\\IMG1");
a1.WaitForIt(30);
if
(a1._gotIt)
{
//
Check the result
Assert.True(a1._status,
"Commit result is not success");
Assert.That(a1._failed_instances.Length
== 0);
DCXOBJ
obj = new DCXOBJ();
obj.openFile(fullpath + "\\SER1\\IMG1");
string
sop_class_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopClassUid).Value.ToString();
string
instance_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopInstanceUID).Value.ToString();
Assert.AreEqual(a1._succeeded_instances,
sop_class_uid + ";" + instance_uid
+ ";");
}
else
Assert.Fail("Didn't get commit result");
/// Cleanup
Directory.Delete(fullpath,
true);
}
Here’s the sync accepter:
class SyncAccepter
{
public
bool _gotIt = false;
public
bool _status = false;
public
string _transaction_uid;
public
string _succeeded_instances;
public
string _failed_instances;
public
DCXACC accepter;
public
string MyAETitle;
public
void accepter_OnCommitResult(
bool
status,
string
transaction_uid,
string
succeeded_instances,
string
failed_instances)
{
_gotIt = true;
_status = status;
_transaction_uid =
transaction_uid;
_succeeded_instances =
succeeded_instances;
_failed_instances =
failed_instances;
}
public
SyncAccepter()
{
accepter = new DCXACC();
accepter.OnCommitResult += new IDCXACCEvents_OnCommitResultEventHandler(accepter_OnCommitResult);
MyAETitle = System.Environment.GetEnvironmentVariable("COMPUTERNAME");
accepter.WaitForConnection(MyAETitle, 104, 0);
}
public
bool WaitForIt(int
timeout)
{
if
(accepter.WaitForConnection(MyAETitle, 104, timeout))
return
accepter.WaitForCommand(timeout);
else
return
false;
}
}
And here’s the example that waits for the results on the
same association.
public void CommitFilesAndWaitForResultOnSameAssoc()
{
bool
status = false;
bool
gotIt = false;
String
fullpath = "SCMTEST";
Directory.CreateDirectory(fullpath);
CommonTestUtilities.CreateDummyImages(fullpath,
1, 1);
string
succeededInstances;
string
failedInstances;
string
MyAETitle = System.Environment.GetEnvironmentVariable("COMPUTERNAME");
DCXREQ
r = new DCXREQ();
r.OnFileSent += new IDCXREQEvents_OnFileSentEventHandler(OnFileSent);
r.Send(MyAETitle, IS_AE, IS_Host,
IS_port, fullpath + "\\SER1\\IMG1",
out succeededInstances, out failedInstances);
DCXOBJ
obj = new DCXOBJ();
obj.openFile(fullpath + "\\SER1\\IMG1");
string
sop_class_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopClassUid).Value.ToString();
string
instance_uid = obj.getElementByTag((int)DICOM_TAGS_ENUM.sopInstanceUID).Value.ToString();
string
transactionUID = r.CommitFilesAndWaitForResult(MyAETitle, IS_AE, IS_Host,
IS_port, fullpath + "\\SER1\\IMG1",
5, out
gotIt, out status, out
succeededInstances, out failedInstances);
Directory.Delete(fullpath,
true);
Assert.True(status,
"Commit result is not success");
Assert.That(failedInstances.Length
== 0);
Assert.AreEqual(succeededInstances,
sop_class_uid + ";" + instance_uid
+ ";");
}
And here’s the common test utilities class in case you need
it
using System;
using
System.Collections.Generic;
using
System.Text;
using rzdcxLib;
using System.IO;
namespace rzdcxNUnit
{
class CommonTestUtilities
{
public static string
TestPatientName
{
get
{ return "John^Doe";
}
}
public static string
TestPatientID
{
get
{ return "123765";
}
}
public static string
TestStudyInstanceUID
{
get
{ return "123765.1";
}
}
public static string
TestSeriesInstanceUID
{
get
{ return "123765.1.1";
}
}
///
/// Create to series with 4 images each of test images
///
///
Root directory to put
the files in
///
public static unsafe List<String>
CreateDummyImages(String path)
{
return
CreateDummyImages(path, 4, 2, "", false);
}
public static unsafe List<String>
CreateDummyImages(String path, int numSeries, int
numImagesPerSeries)
{
return
CreateDummyImages(path, numSeries, numImagesPerSeries, "",
false);
}
public static unsafe List<String>
CreateDummyImages(String path, int numSeries, int
numImagesPerSeries, String suffix)
{
return
CreateDummyImages(path, numSeries, numImagesPerSeries, suffix, false);
}
///
/// Create a set of test images
///
///
Root directory to put
the files in
///
How many series
to create
///
How
many image files per series
///
filename suffix to
use
///
public static unsafe List<String>
CreateDummyImages(String path, int numSeries, int
numImagesPerSeries, String suffix, bool long_uid_names)
{
List<String> filesList = new
List<string>();
const
int ROWS = 64;
const
int COLUMNS = 64;
const
int SAMPLES_PER_PIXEL = 1;
const
string PHOTOMETRIC_INTERPRETATION = "MONOCHROME2";
const
int BITS_ALLOCATED = 16;
const
int BITS_STORED = 12;
const
int RESCALE_INTERCEPT = 0;
DCXOBJ
obj = new DCXOBJ();
/// Create an element pointer to place in the object for every
tag
DCXELM
el = new DCXELM();
/// Set Hebrew Character Set
el.Init((int)DICOM_TAGS_ENUM.SpecificCharacterSet);
el.Value = "ISO_IR
192";
/// insert the element to the object
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.Rows);
el.Value = ROWS;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.Columns);
el.Value = COLUMNS;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.SamplesPerPixel);
el.Value = SAMPLES_PER_PIXEL;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.PhotometricInterpretation);
el.Value =
PHOTOMETRIC_INTERPRETATION;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.BitsAllocated);
el.Value = BITS_ALLOCATED;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.BitsStored);
el.Value = BITS_STORED;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.HighBit);
el.Value = BITS_STORED - 1;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.PixelRepresentation);
el.Value = 0;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.WindowCenter);
el.Value = (int)(1 << (BITS_STORED - 1));
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.WindowWidth);
el.Value = (int)(1 << BITS_STORED);
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.RescaleIntercept);
el.Value = (short)RESCALE_INTERCEPT;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.RescaleSlope);
el.Value = 1;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.GraphicData);
el.Value = "456\\8934\\39843\\223\\332\\231\\100\\200\\300\\400";
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.PixelData);
el.Length = ROWS * COLUMNS *
SAMPLES_PER_PIXEL;
el.ValueRepresentation = VR_CODE.VR_CODE_OW;
ushort[]
pixels = new ushort[ROWS
* COLUMNS];
for
(int y = 0; y < ROWS; y++)
{
for
(int x = 0; x < COLUMNS; x++)
{
int
i = x + COLUMNS * y;
pixels[i] = (ushort)(((i) % (1 << BITS_STORED)) -
RESCALE_INTERCEPT);
}
}
fixed
(ushort* p = pixels)
{
UIntPtr
p1 = (UIntPtr)p;
el.Value = p1;
}
obj.insertElement(el);
// Set
identifying elements
el.Init((int)DICOM_TAGS_ENUM.PatientsName);
el.Value = CommonTestUtilities.TestPatientName;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.patientID);
el.Value = TestPatientID;
obj.insertElement(el);
String
study_uid = "123765.1";
el.Init((int)DICOM_TAGS_ENUM.studyInstanceUID);
el.Value = CommonTestUtilities.TestStudyInstanceUID;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.StudyID);
el.Value = 1;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.sopClassUid);
el.Value = "1.2.840.10008.5.1.4.1.1.7";
// Secondary Capture
obj.insertElement(el);
for
(int seriesid = 1; seriesid <= numSeries;
seriesid++)
{
String
series_uid = study_uid + "." +
seriesid;
el.Init((int)DICOM_TAGS_ENUM.seriesInstanceUID);
el.Value = series_uid;
obj.insertElement(el);
string
series_path = "";
if
(long_uid_names)
series_path = path + "\\" + series_uid;
else
series_path = path + "\\SER" + seriesid;
Directory.CreateDirectory(series_path);
el.Init((int)DICOM_TAGS_ENUM.SeriesNumber);
el.Value = seriesid;
obj.insertElement(el);
for
(int instanceid = 1; instanceid <=
numImagesPerSeries; instanceid++)
{
String
instance_uid = series_uid + "." +
instanceid;
el.Init((int)DICOM_TAGS_ENUM.sopInstanceUID);
el.Value = instance_uid;
obj.insertElement(el);
el.Init((int)DICOM_TAGS_ENUM.InstanceNumber);
el.Value = instanceid;
obj.insertElement(el);
/// Save it
///
String
filename = "";
if(long_uid_names)
filename = series_path
+ "\\" + instance_uid + suffix;
else
filename = series_path
+ "\\IMG" + instanceid + suffix;
obj.saveFile(filename);
filesList.Add(filename);
}
}
return
filesList;
}
// The unsafe
keyword allows pointers to be used within the following method:
public static unsafe void Copy(byte* pSrc,
int srcIndex, byte[]
dst, int dstIndex, int
count)
{
if
(pSrc == null || srcIndex < 0 ||
dst == null
|| dstIndex < 0 || count < 0)
{
throw
new System.ArgumentException();
}
//int
srcLen = src.Length;
//int
dstLen = dst.Length;
//if
(srcLen - srcIndex < count || dstLen - dstIndex < count)
//{
// throw new System.ArgumentException();
//}
// The
following fixed statement pins the location of the src and dst objects
// in
memory so that they will not be moved by garbage collection.
fixed
(byte*
pDst = dst)
{
byte*
ps = pSrc;
byte*
pd = pDst;
//
Loop over the count in blocks of 4 bytes, copying an integer (4 bytes) at a
time:
for
(int i = 0; i < count / 4; i++)
{
*((int*)pd)
= *((int*)ps);
pd += 4;
ps += 4;
}
//
Complete the copy by moving any bytes that weren't moved in blocks of 4:
for
(int i = 0; i < count % 4; i++)
{
*pd = *ps;
pd++;
ps++;
}
}
}
}
}
Summary
Let’s wrap it all up:- There’s DICOM service called Storage Commitment (SCM) that gets a list of instances and gives back which are stored with our peer and can be safely deleted from our local disk and which are not and we better send them over again.
- The SOP Class UID of Storage Commitment is 1.2.840.10008.1.20.1
- The request is sent via N-ACTION with the well known SOP Instance UID
- The result is received via N-EVENT-REPORT
- The result can come on a separate association.
- If we want to give the PACS a chance to send the result on the same association, we should at least wait for it to come for couple of seconds after sending the request.
Q&A
Q: Should the list of instances in the commit request be identical to the group of instances we sent in the association that stored the files?
A: No. In DICOM there's no contextual meaning to the association.Q: If some files failed to commit, should I send all files again or just the ones that failed?
A:I would recommend sending just the one that failed.Q: What if I didn't get the result?
A: Send a Storage Commit Request again.Q: How can I know the reason for the failure from the commit result?
A: You can't but from the C-STORE command response you can. There's a status there and sometimes additional explanation attributes. Read the log.
Why not to implement it the way described above
One last thing I owe you. You can say that the command succeeded but the file is dead, ha? After all, we are in a hospital right, the operation was successful but the patient died? anyone? Never mind. Implementations that just store the file on disk, say OK and later parse it and find out that it’s wrong and can’t actually process it simply don’t have any way to tell the client what was the problem with the file. They lose the only chance to respond properly in the C-STORE Response. If you fear that processing the registration of a new instance during the C-STORE will delay the communication so you better review your registration process. Maybe your DICOM parser is too slow or your database connection is not optimized.<- Prev - Modality Performed Procedure Step
Pixel Data - Next ->
Excellent article.
ReplyDeleteThank you.
Ariel T.
Hello,
ReplyDeleteThanks for this great tutorial. I would like to hear your opinion about the following case:
I am implementing a DICOM router, it receives a full study very fast in a local device, and then uploads it to a repository on the cloud.
So, I am wondering, when is the best moment to confirm the arrival of the image/s? at router or cloud repository? What to answer when the images are still in the router buffer?
Thanks in advance,
Jaime O.
Hi Jamie
DeleteThats an interesting question.
I would suggest to check what IHE has to say about it. The key to search would be 'shared image archive' in the XDS-I integration profile.
I'll do this check as well and re-reply.
The way I would do it is to reply success once it's stored on the proxy regardless of the upload progress.
Otherwise, if you reply failed, the modality will send the study again and this is not what you want.
Regards
Roni