This topic contains 4 replies, has 2 voices, and was last updated by  Ryan McBee 4 years ago.

A slow filter

  • Some background: I’m currently in flux with my Iguana implementations. I’m still using VMD files with an endpoint of database on my channels. I had been doing customizations for clients in 4.x in each VMD file, but now I’m moving to a single VMD for all my implementations and doing that custom work in filters. I run accross a scenario where my filter is really throttling my throughput from about 12 messages per second to one. This wouldn’t be a killer except I’m doing a lot of testing with this client that involves bulk processing. I think it’s the codemap that is slowing me down as I have the second fucntion at many other sites to address a problem in our database design not allowing nulls.

    Is there a more efficient way to do this mapping? Should I just have a string of “If, then” statements to handle this?

    require 'node'
    require 'codemap'
    
    function main(data)
       local msgIn, msgType=hl7.parse{vmd='TRACE_VMD_GLOBAL.vmd', data=data}
       local msgOut=hl7.message{vmd='TRACE_VMD_GLOBAL.vmd',name=msgType}
       
       siteMapping(msgIn, msgOut)
       queue.push{data=msgOut:S()}
    end
     
    function siteMapping(msgIn,msgOut)
       msgOut:mapTree(msgIn)
       --Added custom Mappings for Sites 3/4/2013
       --They want to display short terms instead of codes
       siteMap = codemap.map({['008']='ABC',['002']='MAIN',['003']='CVS'},'UNKNOWN')
       msgOut.MSH[4][2] = msgIn.MSH[4][1]
       msgOut.MSH[4][1] = siteMap[msgIn.MSH[4][1]]
    
       --Sending a space instead of a null for Guarantors that don't have them.
       for i = 1, #msgIn.GT1 do
          if msgIn.GT1[i][3][1][1]:S()=='' then
             msgOut.GT1[i][3][1][1][1]= ' '
          end
          if msgIn.GT1[i][3][1][2]:S()=='' then
             msgOut.GT1[i][3][1][2]= ' '
          end
       end
    end

    M.R. McBee

    Hey Ryan – sorry meant to reply to this earlier – I reckon one optimization would be to shift this line:

    siteMap = codemap.map({[‘008′]=’ABC’,[‘002′]=’MAIN’,[‘003′]=’CVS’},’UNKNOWN’)

    To be outside of the function so that it only get’s invoked once when the script starts up. That may make a difference or not…

    How big is the vmd file and how many GT1 segments?

    Where is the “siteMap” function defined? I think there are quite a few variables missing that could slow things down further.

    I took your advice on moving the sitemap code out. I doesn’t seem to haveI helped much. But ‘ve done some similar mappings on more servers and I don’t see the performance hit. This isn’t our oldest server, but it’s no spring chicken. I’m willing to chalk it up to that since this is the only place I’ve seen this issue and it doesn’t actually cause a problem during normal operation.

    M.R. McBee

    Hey Ryan,

    It’s probably best to follow it up with support – you guys are entitled to it – it’s a specific problem – I am sure it’s something about the script or the environment.

    Other things I would suggest are trying the same code on a different server to eliminate that as a variable and putting some logging in there to see what might be the slow parts.

    Thanks Eliot, I really do appreciate it. The next time I do something with this customer or this server, will pursue it with support. Like I said, it’s not presenting a tangible issue and I have some other dragons that need slaying. But I will investigate this further when do some work on this particular server and I’ll let eveyrone know what I find.

    M.R. McBee

Tagged: 

You must be logged in to reply to this topic.