Batch converting Neurolucida DAT file format to XML file format

 The issue: I have been writing some scripts in Matlab that permit the direct import of the contour and marker data from the delineations users draw with Neurolucida 360 software. For that purpose, I have been using the option of saving the Neurolucida delineation files in the XML format. Neurolucida 360 permits the user to save their data in several different formats, among them the native binary format (DAT), a standard XML format and an ASC (ASCII type) format. I use XML format because of good documentation (links here and here) provided by MBF. However, Neurolucida 360 saves by default the tracing data in their native binary (DAT) format, so most users will (un)consciously primarily save their data in that format. MBF does not give the key for deciphering this format directly, but attempts were made to reverse engineer the key (link here and here). There is also code available for converting between different tracing formats (NLMorphologyConverter). Unfortunately, this software has not been updated since 2016, and MBF file format has changed since then, so when I tried to use this software, I could only get the marker positions converted, but not the contours. 

Diagnose procedure: After a lot of searching in the MBF website, I found something interesting in their list of NL 360 updates. In 2019, there was included a data2xml batch converter in their program. If you scroll down to the section regarding the version 2019.1.2 updates, you will read about such a converter. 

Solution: You can use this batch converter by following these steps.

1. Open NL 360. You will notice that on the upper right corner of the main interface menu there is an empty text box (marked by a red arrow in the figure under).This is what the MBF website calls a "quick launch box".


2. Type "Convert" on the mentioned field. The autofill function of the box will suggest the option "Convert Files". Choose that option.


3. A new menu will appear with the title "DAT to XML converter". 


4. Choose your Source (folder containing the original DAT files) and Destination (folder that will receive the XML files) folders in the menu. Press the Convert button on the lower left corner to start the process. 


5. After some seconds, the process will be concluded.


Notes: Unfortunately, This conversion is not always effective. I noticed a surprising problem regarding NL360 dat files. If you look closely at a list of DAT files in the figure of the windows file explorer shown under, you will notice that some DAT files are small (some tens of KB size), while others are very big (above 1 MB). 


After some testing, I realized that the small dat files will get correctly converted into xml files, while the big ones will not. The resulting xml files will have a normal heading and the beginning of the main body of data will be normal, but at some point it shows the data in a string of hexadecimal equivalents (that take a lot of space in the text file). The interesting (strange) part is that these files can still be opened normally using NL 360. I tried to find if there is any pattern in the content of such files, but could not find any visible difference between these and the small ones. I guess it is some sort of bug in the saving process. I contacted MBF to understand better the reason behind these bug (!), and will write a post when I solve this mystery. Fortunately, after browsing among the files produced by different NL360 users, I realize that the proportion of "bugged" to "healthy" DAT files is relatively low, so this problem may only occur rarely.  







. 


 

Kommentarer

Legg inn en kommentar

Populære innlegg fra denne bloggen

Opening Zeiss czi file format mosaic files in Matlab using bio-formats

  The issue: I often write scripts in Matlab to batch image processing tasks, and for that purpose I need to load confocal or axioscan Zeiss czi format image files. I use an open source package called Bio-formats . However, I frequently get two kinds of problems when using this library in Matlab: a) working memory problems (Matlabs Java memory capacity gets filled when the files are in the order of the GB size); and b) stitching problems. This post will only focus on the second issue (I will write later a separate post for the first). The problem is that, for a long time, when I tried to open czi image files of confocal images that were tiled, the bio-formats pack would open the tiles individually, but it would not stitch them as a single image.     Diagnose procedure:  I tried to download different versions of the bio-formats library, because according to internet forums like forum.image.sc , the latest version of bio-formats had that problem fixed. The v...
Les mer

Possible solution for problems opening confocal files from Zeiss LSM 880 in Neurolucida 360 environment

  The issue : One of the regular users of Neurolucida 360 has noticed that some image files produced by the Zeiss LSM 880 (confocal microscope using ZEN black software) do not open. This happens rarely, but she had five examples of medium sized (tens to hundreds of MB sized, mosaics with multiple z-levels) files that NL360 could not open.  Diagnose procedure : I tried to open the same files in Fiji and by using Matlabs bformats pack (v7.0.0) and in both cases, I got error messages concerning corrupted data bytes. Nevertheless it was possible to open those files in ZEN blue lite version. That made me hypothesize that there are a few corrupted data bytes in the file, but that ZEN lite can bypass those data points. That lead me to the think that if I tried to save those files again using the "Save as" menu, the files would be saved with the corrected bytes version, and that would make the files whole again. Upon testing this procedure, the files were fixed and could open normal...
Les mer

Opening multileveled TIFF images in Matlab

  The issue : this is a simple issue. I generally open single-leveled TIFF files in Matlab by simply using the function:    data = imread('example.tif'); However, if the tif file in question is multi-layered (3D), this command will only open the top layer and ignore the rest.  Solution : There are two possible solutions for this problem. 1. Using only one command (tested in MatLab R2023, documentation refers to being first released with R2020b):        data  = tiffreadVolume('example.tif'); The numerical array data will therefore have 3 dimensions (x,y,z). I got it automatically in the format uint8 when I tried. Between the two solutions, this one is the fastest. 2. Using multiple lines (maybe for versions prior to MatLab R2020b):      info = imfinfo('example.tif');   data = zeros(info(1).Width,info(1).Height,numel(info)); %preallocates the data array.   data = uint8(data);  %converts the array from double to uin...
Les mer